Jump to content

Wrong channel names on scanning


supertack

Recommended Posts

Hi...I've noticed recently a bug wich affects the scanning process, in both DVBViewer built-in scanner and the full version of transEdit MMC !

 

For example, when trying to scan some transponders on Nilesat (7°W), I got this picture :

 

scanfq3.jpg

 

As you can see in red, channel names are reported as services followed by PID numbers. I tried other softwares and my home receiver in the same conditions (without moving miy dish, some LNB) , the names are shown correctly, so the bug comes from DVBViewer...

 

But I noticed something else...when analysing the same transponder using TransEdit MMC, I see that names are reported correctly (unfortunatly only on analyse, not at scan) as you can see below :

 

testzp6.jpg

 

I noticed that during scanning, the SDT case take a long time to become green, so I tried also to add more time to scanner without success.

 

I experimented also this bug on same transponders on Hotbird, Astra & Hispasat...

 

My configuration :

 

Hauppauge WinTV Nova-S-Plus with the latest driver

DVBViewer Pro 3.5.0.100 (last version)

TransEdit MMC full (last version)

Pentium IV @ 2.4 GHz, 768 MB DD-RAM, GeForce FX 5200 (128 MB)

Classic motorized dish, no DisecQ, 1 LNB (0.2 dB)

Signal quality near to 95% on this transponder

 

Thanks for help !

Edited by supertack
  • Like 1
Link to comment

..fiddling with the time out won't help. It's a real bug that can be easily reproduced. The reason has to examined. The scanning engine doesn't read the SDT though there is a normal density of sections. The analyzer's parsing engine is working ok :bye:

 

 

 

 

 

ps.

 

@supertack, could you identify the other transponders that you've mentioned ?

Link to comment

Thanks Derrick for your fast answer !

 

Well, I don't remember very well the exact transponders, but many ot them are actually on Nilesat...

 

I downloaded an updated list from Joshy's fun and tried to scan Nilesat again, and after checking Lyngsat I saw that I missed my channel names on many transponders. That's all what I can tell you now.

 

But I remember that I've got a correct channel list few weeks ago, using the same version of TransEdit MMC :bye:

 

Other transponders with same problem on Nilesat :

 

12303 H 27500 3/4

 

12149 H 27500 3/4 (some channels)

 

Maybe Nilesat is transmitting with new parameters ? But anyway, other softwares still giving the correct channel names...

 

NB: when scanning Nilesat, just scan the transponders marked in lyngsat as "Egypt" beams. Others beams are for middle east only.

 

I hope there is a fix for this bug ? Because it could affect so many interesting channels on other sats.... :bye:

Link to comment

Ok, that revealed it - see attached image:

 

- TS header, payload_unit_start_indicator set (4 bytes)

- pointer field (= 0, 1 byte)

- section with table_id 0x72 (ST, stuffing table, 8 bytes) !!!

- section with table_id 0x42 (SDT actual)

 

Same for all packets with payload_unit_start and a 0x42 table.

 

The scanner's section parser expects 0x42 right after the pointer field. Dunno whether the NileSat method is ETSI compliant or not. There is no hint that something like this may happen. The specs say:

 

5.2.8 Stuffing Table (ST)

The purpose of this clause (see table 11) is to invalidate existing sections at a delivery system boundary e.g. at a cable head-end. When one section of a sub_table is overwritten, then all the sections of that sub_table shall also be overwritten (stuffed) in order to retain the integrity of the section_number field.

...and about stuffing:

 

Stuffing may be performed by filling each remaining byte of the TS packet with the value "0xFF". Consequently the value "0xFF" shall not be used for the table_id. If the byte immediately following the last byte of a section takes the value of "0xFF", then the rest of the TS packet shall be stuffed with "0xFF" bytes. These bytes may be discarded by a decoder. Stuffing may also be performed using the adaptation_field mechanism.

...and nothing more. The parser only handles these "recommended" kinds of stuffing. At least the NileSat method is quite unusual. :bye:

Zwischenablage01.png

Link to comment

Euh...may someone explain to me ??? It is a bug from DVBViewer ? Because, as I said, there is no problem with other softwares...

 

 

I'll try to find some problematic transponders on the sats you requested : Hotbird and Astra

 

Thanks, this forum is really fantastic ! :bye:

Link to comment

Well, so maybe Nilesat is not using a non-standard method...but why other popular software works correctly ?

 

That includes also my old Drake, a standard home demodulator built in 2001... :bye:

Link to comment
The parser only handles these "recommended" kinds of stuffing. At least the NileSat method is quite unusual.

 

The stuffing table is an overwritten table that is invalidated and should be ignored. The FFs are part of this don't care table and should not be considered as stuffing in the sense of your 2nd cited passage. Whether this is unusual or not, it is imho allowed and no violation of the standard.

 

That's why I (my personal opinion :bye: ) come to the conclusion that the parser should be fixed. Take the parser from the analyzer :bye:

Link to comment
It is a bug from DVBViewer ?

This is not one of those movies for simple minded people that leave no doubt who is good and who is bad. :bye:

 

The stuffing table is an overwritten table that is invalidated

It doesn't look like it ever has been another table. I would expect a different length and some other content than 0xFF, because the purpose of the ST is to invalidate a sub_table (BAT or SDT in this case) by overwriting the table_id with 0x72, and nothing more. But all possible sub_tables are still present in this stream (SDT actual, SDT other, BAT). IMO they simply abuse the ST for ordinary stuffing.

 

Nevertheless there are two basic questions arising from it:

 

1) May a single payload_unit contain different sub_tables (with different table_ids)?

 

2) How shall the parser handle unexpected table_ids (let's say, table_id 0x00 within a SDT stream)? Shall it try to read the length of this unknown section and skip it? That's what the analyzer does. Or regard it as corrupted data and discard the whole payload_unit? That's what the scanner does, thus being more crash-proof.

 

Of course it is possible to add a simple "if 0x72 after pointer_field then skip stuffing_table" statement. But I prefer to solve issues like this more profoundly.

 

Take the parser from the analyzer

...and kick all devices with hardware PID filter. Yes, why not? :bye:

Link to comment
It doesn't look like it ever has been another table. I would expect a different length and some other content than 0xFF
who cares?

 

May a single payload_unit contain different sub_tables (with different table_ids)?
why not?
Within TS packets of any single PID value, one section is finished before the next one is allowed to be started
Link to comment

Something to add, for better understanding:

 

In many cases it's a question of balance, not of bug/fix. Software should reject as much as possible, and tolerate as much as necessary. Many crashes and all these buffer overflow exploits can only happen because software is too tolerant, processing data that is "out of bounds". Sometimes things don't work because software is too restrictive.

 

Hard to understand for users... well, go to your kitchen and try to balance a wooden spoon on a knife edge. Then run with let's say 2.5 gsps (gigasteps per second), without losing the spoon. After that, you'll know what it's like to be a developer. :bye:

Link to comment
Have a look at Hotbird 12322 H. The scanner finds only some of the channel names, the analyzer at least more. So there is something to do...

..yes, indeed :bye:

 

here's what tsreader finds (same as transedit's analyzer):

 

 

 

 

hmm, I don't see the problem. There are 2 different sections with 0x42, a long one (length 0x3f3) and a short one (0x30). Both are alternating and start at the beginning of a ts_packet_payload with stuffing at the end.

Link to comment

Hotbird 12322 is a different issue.

 

As you can see in the Analyzer, there are two descriptors in the SDT for the channels in question. A service_name_descriptor, containing the provider and service name, as it should be, and the scanner reads them. But it is followed immediately (in the same descriptor_loop) by a multilingual_service_name_descriptor, that specifies only two languages, but no names. The scanner reads this descriptor too, thus replacing the already retrieved names by empty strings.

 

So I have to tell the scanner not to read a name if it has already got one for a channel. The Analyzer is not affected because it lists the content of the second descriptor separately.

 

This is an example for being too tolerant, in contrast to the NileSat issue.

Link to comment

Ok, as a first result, try TransEdit 3.0.1.2 -> members area, beta section.

 

Hotbird 12322 H is ok now, NileSat don't know... don't expect too much, it's just a test version. I made the parser more tolerant, got a nice crash (endless loop) on a Hotbird transponder, made it a bit less tolerant again... as I said, balancing. Astra 19°/28° and Hotbird full scan without crash -> uploaded, though I'm not happy with certain aspects yet.

 

@Derrick: This version is supposed to record *all* PIDs of a transponder, if you select the whole PID list in the Analyzer, even those that are not visible. With other words, selecting all disables filtering. Try...

Link to comment

The scan is perfect in both Hotbird & Nilesat using new version of TransEdit !!! :bye:

 

fixbs0.jpg

 

Thanks Griga you're a genious ! And thanks for Derrick for your help too !

 

Now I think there is something to do with the DVBViewer built-in scanner, as no everybody is using TransEdit MMC (and they are wrong !) :bye:

 

 

Thanks again...I'll tell you if I find similar problem on other satelites I'll scan today... I hope that there is no crash as you said !

Edited by supertack
Link to comment

Stop joking. It's a very serious matter :bye:

 

I've digged a bit deeper in consideration of the relation between sections and payload units. The ISO specs say:

 

When at least one section begins in a given Transport Stream packet, then the payload_unit_start_indicator shall be set to 1 and the first byte of the payload of that Transport Stream packet shall contain the pointer. When no section begins in a given Transport Stream packet, then the payload_unit_start_indicator shall be set to 0 and no pointer shall be sent in the payload of that packet.

...which means, a payload_unit may contain sections with different table_ids, if they all start in the same TS packet with payload_unit_start_indicator = 1.

 

Even the Analyzer may fail to read a section, if *three* sections start within a packet, and the table_ids of the first two sections do not belong to evaluated tables (like 0x72, the NileSat stuffing table). In this case the Analyzer will skip the first section, and, after having encountered the second table_id, discard the rest, including subsequent packets with payload_unit_start_indicator = 0. I think I better change that too. :bye:

Link to comment

Well, I tried to use the new version of TransEdit on few sat I can receive. Fortunatly, it was no crash during scan, but I noticed some "services" again... :bye: (probably because of bad reception conditions) .However, TransEdit is more sensitive than ever and I'm really satisfied !

 

If this may help you, you can find my scan result attached below in .ini format.

 

Thanks again !

 

Scan_17_11_06.rar

 

 

@ Griga : could you add a new option in TransEdit ? It would be helpful for us to choose our proper category during export (and not only Provider, Scan Date or Frequency). When I export, I always choose Scan Date, and after I modify the .ini file for converting the scan date to satellite name, as in root section. But this takes a lot of time, specially with big lists. And it's the only way if I want to obtain this result :

 

88av6.jpg

Edited by supertack
Link to comment
Even the Analyzer may fail to read a section, if *three* sections start within a packet, and the table_ids of the first two sections do not belong to evaluated tables (like 0x72, the NileSat stuffing table). In this case the Analyzer will skip the first section, and, after having encountered the second table_id, discard the rest, including subsequent packets with payload_unit_start_indicator = 0. I think I better change that too.

Maybe you have to repeat this. I don't quite understand.. ..of course there could be more than 2 sections starting in one and the same ts_packet. Theoretically speaking at least. What could be the use of this other than bothering stressed developers, I don't know :bye: You have to think of sections that can be short by default, e.g. like some ecms. Further the section rate has to be quite high which means a single section in a ts_packet is a real waste of bandwidth. There might be other reasons..

Link to comment
but I noticed some "services" again...

There are always "service xxxx" entries, because the SDT contains no service_name_descriptor for some channels. Always compare it with other sources (DVB, not LyngSat) before posting it as a bug.

 

could you add a new option in TransEdit ?

I don't understand what you want. Please try again in the Plugins or Feature Requests Section of this forum, since it is not bug-related.

 

I don't quite understand.. ..

Examining the code revealed that the Analyzer's section parser is restricted in the same way as previously the one of the scanner, affecting the third (scanner: second) and following sections. I've changed it, though I don't expect much practical relevance.

Link to comment

..something else. Dunno whether it fits here ;)

 

Apparently there are streams that don't contain a SDT but a service descriptor 0x48 is used in the PMT. No action needed, just for information. It's only a data stream anyway..

 

 

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Unfortunately, your content contains terms that we do not allow. Please edit your content to remove the highlighted words below.
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...