BBC HD Fixer

[Griga]

Yesterday we discovered that the AC3 stream broadcasted by BBC HD is buggy. The PMT (Program Map Table) says it's a Private 1 Stream, but the PES start codes are 00 00 01 C0, as for MPEG audio. Should be 00 00 01 BD. However, the content is AC3.

 

This bug (I would never have expected something like this from BBC ) prevents some software from detecting and processing the AC3 stream correctly, because it's not ISO and DVB compliant.

 

BBC_HD_Fixer.exe (attached below) replaces the wrong AC3 PES startcodes in BBC HD TS recordings by the correct ones. Drag & Drop the TS file on the program icon or on the BBC HD Fixer window. The program works "in place", which means, it doesn't copy the file while changing it.

 

BBC_HD_Fixer is freeware. You may distribute it as you like, but not change it. It is provided "as it is". You use it at your own risk.

 

<edit>

Tool removed, since BBC HD is broadcasting AC3 with correct start codes now.

</edit>

 

<edit>

Tool re-uploaded since BBC HD has returned to the bad old habits AC3 PID set to 2329 as default.

</edit>

 

<edit>

Tool updated, since BBC HD has changed the AC3 PID to 5501. Writes the (entered) AC3 PID to the file BBC_HD_Fixer.ini in order to remember it.

 

If you store BBC_HD_Fixer.exe in the DVBViewer Pro/GE installation directory (where DVBViewer.exe is, recommended), it will create the ini file in the DVBViewer configuration folder, otherwise in its own directory. In this case you will have to store the tool in a directory granting write access (e.g. my documents), since Windows (particularly Vista) doesn't want applications to write to the program files directory.

</edit>

BBC_HD_Fixer.zip

Edited May 17, 2009 by Griga

[zarqy]

thanks, saves me doing it manually

[nomis1804]

This is brilliant! Thanks so much

[Raptor-75]

It seems BBC is not the only channel with this problem. I have to use this fixer to get AC3 playback on recordings from the Swedish channel SVT HD. The AC3 PID is 617.

 

I assume this doesn't work while recording is still going on, which is a shame. Any chance a workaround for the bad stream could be included in DVBViewer?

[clumpco]

Is this problem still around?

I ask because any BBC HD .ts recorded by DVBViewer (not sure of my version, I'm at the office) says that it is AC3 5.1 but only plays back the L&R channels.

TIA

[Griga]

Is this problem still around?

No. They are using the correct PES start codes for AC3 now (00 00 01 BD). I'll remove the tool.

 

I ask because any BBC HD .ts recorded by DVBViewer (not sure of my version, I'm at the office) says that it is AC3 5.1 but only plays back the L&R channels.

Check View -> Filters -> DVBSource. The property page shows the detected format. If it is 5.1 resp. 3/2, but only two channels are played, I'd assume a decoder or soundcard (configuration) problem.

[clumpco]

If it is 5.1 resp. 3/2, but only two channels are played, I'd assume a decoder or soundcard (configuration) problem.

 

Hi Griga, thanks for the reply.

If I look at the ac3 filter while receiving BBC HD I see all six channels with activity on the input side and only two on the output (normal as I only have two speakers on my computer).

If I record the programme and then play it back in MPC it shows as AC3 5.1 but only has "input" to the AC3 filter on the L&R channels.

This is a bit of a problem for my HD torrent clients who are wanting their 5.1 :-)

 

I though that DVBViewer did a direct stream copy to disk? How can it lose 4 channels?

 

I will investigate further.

 

[uPDATE]

I feel so stupid... Media Player Classic had its internal AC3 filter set up to downmix to stereo. Disabled the filter.

Note that I use MPC to demux the audio for my recode to 720p so my recodes only had stereo :-(

What an idiot!

 

Thanks for the help

Edited August 5, 2008 by clumpco

[youngbob]

No. They are using the correct PES start codes for AC3 now (00 00 01 BD). I'll remove the tool.

Check View -> Filters -> DVBSource. The property page shows the detected format. If it is 5.1 resp. 3/2, but only two channels are played, I'd assume a decoder or soundcard (configuration) problem.

I've been using DVBViewer for some time now and if I record the BBC HD stream my player (Popcorn Hour) think that two audio channels are included. If I open the file in tsremuxer it report an error in stream but allows me to remux OK which plays fine. Could you please email me the BBC Fixer and I'll try it with the files as remuxing 14Gb files takes a while!

Thanks

Steve

[potnoodles]

Yesterday we discovered that the AC3 stream broadcasted by BBC HD is buggy. The PMT (Program Map Table) says it's a Private 1 Stream, but the PES start codes are 00 00 01 C0, as for MPEG audio. Should be 00 00 01 BD. However, the content is AC3.

 

This bug (I would never have expected something like this from BBC ) prevents some software from detecting and processing the AC3 stream correctly, e.g. DVBViewer Pro when playing a BBC HD recording, because it's not ISO and DVB compliant.

 

BBC_HD_Fixer.exe (attached below) replaces the wrong AC3 PES startcodes in BBC HD TS recordings by the correct ones. Drag & Drop the TS file on the program icon or on the BBC HD Fixer window. The program works "in place", which means, it doesn't copy the file while changing it.

 

BBC_HD_Fixer is freeware. You may distribute it as you like, but not change it. It is provided "as it is". You use it at your own risk.

 

<edit>

Tool removed, since BBC HD is broadcasting AC3 with correct start codes now.

</edit>

 

The problem has returned

http://www.DVBViewer.info/forum/index.php?showtopic=30726

 

MPC reports incorrect audio track data (although it is AC3 ) and PS3 shows no audio data.

Using the (removed) tool fixes the problem for MPC and PS3

 

regards

[Griga]

The problem has returned

Indeed. I've updated and re-uploaded the tool (see first post).

 

It's really incredible. Shame on BBC. It should follow the ISO and EBU (European Broadcasting Union) specifications. This is not on NileSat or some other far-out satellite position, right? The attached hex dump shows the wrong stream_id 0xC0 (fourth byte in the orange part). The first bytes after the PES header are 0B 77, which is the AC3 sync word.

 

From ISO/IEC 13818-1

 

stream_id 110x xxxx: ISO/IEC 13818-3 or ISO/IEC 11172-3 or ISO/IEC 13818-7 or ISO/IEC 14496-3 audio stream number x xxxx

...which means, 0xC0 is reserved for MPEG and AAC audio.

 

From ETSI TS 102 366 V1.1.1, (EBU) Technical Specification, Digital Audio Compression (AC-3, Enhanced AC-3) Standard

 

A.3.1 Stream_type

The value of stream_type for an AC-3 elementary stream shall be 0×06 (indicating PES packets containing private data).

A.3.2 Stream_id

The value of stream_id in the PES header shall be 0×BD (indicating private_stream_1).

[potnoodles]

Indeed. I've updated and re-uploaded the tool (see first post).

 

It's really incredible. Shame on BBC. It should follow the ISO and EBU (European Broadcasting Union) specifications. This is not on NileSat or some other far-out satellite position, right? The attached hex dump shows the wrong stream_id 0xC0 (fourth byte in the orange part). The first bytes after the PES header are 0B 77, which is the AC3 sync word.

 

From ISO/IEC 13818-1

...which means, 0xC0 is reserved for MPEG and AAC audio.

 

From ETSI TS 102 366 V1.1.1, (EBU) Technical Specification, Digital Audio Compression (AC-3, Enhanced AC-3) Standard

 

Thanks Griga,

 

It is indeed the broadcast on astra/eurobird 28 available to the UK HD Freesat boxes - I wonder if that has anything to do with it.

 

I also wonder if its a deliberate attempt at stopping copying and distribution of the HD broadcasts. I.E you can watch the broadcast OK but the recordings are useless??

 

Anyway thanks once more

and best regards

[AndreNewman]

Indeed. I've updated and re-uploaded the tool (see first post).

 

Thanks, I just found this tool while trying to work out what BBC have changed, registered here just to get it.

 

It works well but I'm trying to script with it as I have > 1TB (and growing) of BBCHD that needs fixing! I can't find any way to tell it to exit when it has finished fixing the file such that the script can do the next file, clicking the exit box kinda misses the point. I have adapted my old VLC script now the PS3 won't play program streams with AC3 again :-(

 

Cheers

 

Andre

[Griga]

as I have > 1TB (and growing) of BBCHD that needs fixing!

 

Version 1.0.1 (see first post, re-download) accepts up to two command parameters. If one of them is -c (doesn't matter which one) and the other one a path\filename, the tool does the fixing silently. It doesn't show up, doesn't display error messages and terminates automatically after the job is done.

[AndreNewman]

Version 1.0.1 (see first post, re-download) accepts up to two command parameters. If one of them is -c (doesn't matter which one) and the other one a path\filename, the tool does the fixing silently. It doesn't show up, doesn't display error messages and terminates automatically after the job is done.

 

Works perfectly, many many thanks :-))

 

In case it's useful to anyone this is my (very crude) script, it runs on a MythTV Linux server under crossover, finds all the recordings from channel 7940 (BBCHD) tacks a .ts on the end, runs it through BBC HD Fixer then changes the filename back to the original.

 

Currently thrashing through 2x 1TB disks about half full of BBC HD, should be done in a few hours.

 

Cheers Griga.

 

#!/bin/sh

for a in 7940*.mpg; do

echo $a

mv "./$a" "./$a.ts"

/home/andre/cxoffice/bin/wine /home/andre/BBC_HD_Fixer.exe -c "./$a.ts"

mv "./$a.ts" "./$a"

done

exit

[earto]

i get lost sync while reading the ts packets error. any ideas??

[Griga]

Corrupted TS packet structure, most likely due to a high CPU load while recording, preventing Windows from flushing its hard disk cache in time. More buffering in DVBViewer (see recorder options) may help.

 

TSPlayer (see members area -> tools) is able to fix a corrupted packet structure. Try a TS -> TS conversion. However, there is data missing in the file, and there is no way to reconstruct it.

[Rob100]

Version 1.0.1 (see first post, re-download) accepts up to two command parameters. If one of them is -c (doesn't matter which one) and the other one a path\filename, the tool does the fixing silently. It doesn't show up, doesn't display error messages and terminates automatically after the job is done.

 

Hi, my plan is to use this tool to process BBC HD files automatically after recording with the recording service.

 

I want to use this as part of a script, but the problem I'm having is that the command "returns" before it's finished. I can see the task still running in the task manager and my HDD is flashing away so I know it's working, but would like it to not to "return" until it has finished and even better return an exit code.

 

Any chance please?

[Griga]

I want to use this as part of a script, but the problem I'm having is that the command "returns" before it's finished.

It depends on the type of command and the script language. For VBScript see here (bWaitOnReturn parameter):

 

http://msdn.microsoft.com/en-us/library/d5fk67ky(VS.85).aspx

[dvbviewser]

BBC HD has recently changed PIDs, so this excellent tool doesn't work any more. Could you update it please? Many thanks in advance!

 

As it seems they will never be compliant again, might it be possible for the app to read the PIDs from a text file to save future recompiles?

[Derrick]

To fix the ac3_stream_id you could use a tool called TSDoctor. It's more flexible and also has a cutting function with preview. Though, sometimes it's rather slow..

[dvbviewser]

To fix the ac3_stream_id you could use a tool called TSDoctor. It's more flexible and also has a cutting function with preview. Though, sometimes it's rather slow..

 

Hey, thank you for the super quick reply. Unfortunately the results play back jerkily, with sound dropouts every second or so, and it's *very* slow. (Mustn't it be doing a lot more than fixing the audio since it takes so long?)

[Derrick]

Does the DVBViewer play the original recording fluently? If not, there might be stream errors. Check the discontinuity counter (view->filters->dvb source)

[dvbviewser]

Does the DVBViewer play the original recording fluently? If not, there might be stream errors. Check the discontinuity counter (view->filters->dvb source)

 

It seems fine. It's only a short clip (3 mins) and there is 1 continuity error. I'll have a go with a longer clip in case there was something odd about that one.

 

Thanks for replying so quickly again.

[dvbviewser]

It seems fine. It's only a short clip (3 mins) and there is 1 continuity error. I'll have a go with a longer clip in case there was something odd about that one.

 

Thanks for replying so quickly again.

 

Update: I tried a different clip, about 10 mins/1500MB, and it works fine. The only problem is, where BBC HD Fixer takes 15 seconds on this file, TS Doctor takes...6 minutes! This isn't a viable solution I'm afraid. But thanks for the pointer. Hopefully HD Fixer will be updated at some point.

Edited May 16, 2009 by dvbviewser

[Griga]

BBC HD has recently changed PIDs, so this excellent tool doesn't work any more.

You can enter the new AC3 PID 5501 in the tool's window, can't you?

 

However, as BBC HD seems to adhere stubbornly to bad habits I will upload a new version soon that will save the entered PID in an INI file.

[dvbviewser]

You can enter the new AC3 PID 5501 in the tool's window, can't you?

 

However, as BBC HD seems to adhere stubbornly to bad habits I will upload a new version soon that will save the entered PID in an INI file.

 

Doh! What an IDIOT! But thanks for organising an INI file. Fantastic!

[Griga]

BBC_HD_Fixer updated. See first posting.

 

Please tell me: Are the BBC HD guys just stupid, or do they want to hinder recording purposely? I'm asking that because ITV HD is doing something similar by hiding the H.264 stream as private data.

[JMS]

As far as I can receive / check it beside ITV HD no other UK FTA station uses AC3. So the wrong PES ID on BBC HD could in fact be a bug. On the other hand since ITV HD is like our private stations I strongly believe that messing with the PSI data is with full intent. In general: changing PSI data itself and shifting PIDs from week to week seems to be quite common on UK stations.

 

Jochen

[Derrick]

At least ITV HD or better service 10510 is a different matter. It's a red_button_button_service which is based on OpenTV's middleware. But even without opentv it's possible to watch all the freetv services. That's what counts

[dvbviewser]

Many thanks for the updated app!

 

The BBC claim it's a bug and that they aim to be standards compliant--but they said that before Christmas! There's a blog entry I'll find later, but I'm a bit pressed for time now.

 

Thanks again.

[uglyned]

I've always assumed that on ITV's part it's bloody-mindedness.

 

ITV are the UK's oldest non-subscription commercial TV station and Sky have been around making people pay for rubbish TV for a good while now. Some years ago, ITV and Sky fell out.

 

So when ITV were involved in setting up Freesat, a non-subscription satellite service which was competition for Sky, they introduced their first HD channel. However, in a first attempt to prevent Sky viewers from viewing ITV HD and therefore increase the value of Freesat, they hid ITV HD away behind a 'red button'.

 

Now, with new versions of Sky firmware, Sky subscribers can watch the channel but as it doesn't appear on the EPG, they can't record it.

[dvbrewer]

I've always assumed that on ITV's part it's bloody-mindedness.

 

So when ITV were involved in setting up Freesat, a non-subscription satellite service which was competition for Sky, they introduced their first HD channel. However, in a first attempt to prevent Sky viewers from viewing ITV HD and therefore increase the value of Freesat, they hid ITV HD away behind a 'red button'.

There was no bloody mindedness involved with ITV HD, it just could not run a FTA HD channel on Eurobird so it 'hides' using the red button. When it moves to Astra 2D hopefully it will be a normal HD channel.

[dvbviewser]

Many thanks for the updated app!

 

The BBC claim it's a bug and that they aim to be standards compliant--but they said that before Christmas! There's a blog entry I'll find later, but I'm a bit pressed for time now.

 

Right, one machine rebuild later , the official answer. Post 90 at

 

http://www.bbc.co.uk/blogs/bbcinternet/200...og_level.html#P

 

"Dear Sir

 

Thank you for drawing this to our attention and we apologise for the disruption that this has caused to your reception.

 

The BBC does advise viewers wanting to receive our services via satellite either to do so using Sky or Freesat receivers. As we do not test reception of our services on any other satellite receivers, we cannot promise glitch-free reception on receivers not made for the Sky or Freesat platforms.

 

Nevertheless we do aim to comply with recognised broadcast standards such as those from the DVB and we can confirm that our current broadcasts are not compliant in the way that you have indicated. You are correct that Sky and Freesat receivers are agnostic to this particular signalling and that is why this was not picked up in our testing of our coders following a recent upgrade.

 

We are working with our system suppliers and will aim to correct this signalling as soon as possible. However unfortunately this will not be until the new year as we are heading into the Christmas 'lockdown' where we make no changes to our platforms to ensure they remain stable."

 

Well, they can aim to comply, but it's obviously waaaay down the list!

[mdonahuerib]

Hi, would any kind soul be willing to build this little app for Mac OS X? If not feasible, might it be possible to add a "File add" button or menu item? I tried running the app via Crossover (Wine) and it started up fine ... but since there was no drag and drop interaction between the Windows app and the OS, I am stuck.

 

Thanks in advance and regards,

 

Mark

[Griga]

Hi, would any kind soul be willing to build this little app for Mac OS X?

Sorry, no Mac programmers around here...

 

but since there was no drag and drop interaction between the Windows app and the OS, I am stuck.

What about passing the path\filename as commandline parameter?

[mdonahuerib]

Sorry, no Mac programmers around here...

 

 

What about passing the path\filename as commandline parameter?

 

Sorry if I missed any doc on this, but, I didn't know BBC HD Fixer accepted a command line parameter. Syntax?

 

Thanks!

[Griga]

Well, dragging & dropping a file on an app or shortcut implies commandline processing, because Windows passes the dropped file as commandline parameter to the app. Just the path\filename (wrapped in quotation marks, if it contains blanks), nothing more.

 

Yo may also enter a path\filename as commandline parameter in the properties of a (Windows) BBC_HD_Fixer shortcut (rightclick it...) by appending it to the EXE path in the "Target" input line.

 

Please read here (some posts above) how a Linux guy made it work:

 

http://www.DVBViewer.info/forum/index.php?...st&p=223870

[mdonahuerib]

Thanks, Griga ... worked a treat! I had seen the shell script you refer to but never connected the dots. I found I could run the command in Crossover, appending the file name, and voila!

 

One suggestion/request ... any chance you can allow files with an .mpg extension to be processed? My ts files all have the "wrong" extension.

 

Thanks again!

[fac13]

I think the recent (Aug 5th) BBC HD encoder change may have fixed this issue; can anybody confirm this?

Edited August 11, 2009 by fac13

[Griga]

Currently the BBC HD AC3 stream ID is correct (0xBD). Let's see if they stick to it... it had already been corrected in the past, but then they relapsed