Jump to content
merkin stare

Buffer overflow on 4k hevc

Recommended Posts

merkin stare    0
merkin stare

I am having an issue with high bitrate 4k transmission via dvb-s2 (70+ mb/s).

When viewing live, there is many buffer overflows in DVBSource filter (latest version).

This issue occurs in both DVBViewer and Transedit (latest version).

 

Here is a ts dump of the transmission (MD5 3a 66 b4 09 dd ec 4b 12 36 d9 11 54 bd 35 5f 3c):

https://drive.google.com/file/d/0B0489aMeZ10HRzdzQVhRUlZMRm8/view?usp=sharing

 

I have no issue viewing this transmission live with other dvb-s/s2 viewing software, such as SmartDVB.

I have no issue with File Playback of the recording of the transmission in DVBViewer.

 

So it seems the issue occurs from Tuner/Capture filter of hardware >> DVBSource.ax filter.

I have added buffer option in DVBSource.ini to increase buffer, but issue still persists

I have tried many different tuners/drivers (Prof7500/Qbox2/QboxCI/TBS6903), but issue still persists.

 

Since issue does not occur on File Playback in DVBViewer, but also does not occur with other software, than issue must be related somewhere in filter graph from Tuner/Capture >> DVBSource filter.

 

Share this post


Link to post
nuts    51
nuts

You could try to increase Max. Queued Audio: TV/Radio in DVBSource filter or set it to 0 to disable this function. 

Here are some further information about DVBSource filter:

http://www.DVBViewer.com/griga/DVBSource E/DVBSource.html

 

 

Share this post


Link to post
merkin stare    0
merkin stare

I have adjusted those settings in the past with high bitrate mpeg4/h264 with some success.

But with hevc/h265 the issue persists no matter what setting I adjust.  

 

I think the high bitrate is contributing to this issue, but again its no issue in other dvb software that i have tested with.

 

I should also mention that I am using the latest LAV filters for decoding and EVR for rendering, but data is not even making it that far downstream in the filter graph so I doubt that the decompression/rendering filters are at fault here.

Edited by merkin stare

Share this post


Link to post
nuts    51
nuts

Hm ok. And it works fine for hevc recordings with DVBViewer and DVBSource or do you use LAV Sourcefilter for file playback? 

 

P.S. Your hardware=?

Share this post


Link to post
Tjod    291
Tjod

In short, you system is at it's limit if you playback hevc with 4k resolution.

 

With live TV the the codec has to be as fast as the data arrive from the TV card. With file playback it can be a little bit slower.

Zitat

overflows in DVBSource

the codec (PC) is to slow, therefore DVB Source has to through data away because there are more data arriving from the TV card then the decoder can process.

 

About other DVB Software, the DVBViewer is a 32 bit application and need 32 bit codecs.

http://www.DVBViewer.tv/forum/topic/56578-64-bits-version/

And LAV Filter (hevc) codec is more effective ion the 64 bit version.

 

Try the HEVC Lentoid decoder, it should work better on a slow PC.

Share this post


Link to post
nuts    51
nuts

No 32bit Decoder is doing 70mb/s 4K HEVC in software (realtime) and it is working for file playback ... 

Share this post


Link to post
merkin stare    0
merkin stare
20 hours ago, nuts said:

Hm ok. And it works fine for hevc recordings with DVBViewer and DVBSource or do you use LAV Sourcefilter for file playback? 

 

P.S. Your hardware=?

yes, recordings work fine.  i do not use LAV Source filter.

exactly what do you want to know about my hardware other than what was already posted?

 

20 hours ago, Tjod said:

In short, you system is at it's limit if you playback hevc with 4k resolution.

 

With live TV the the codec has to be as fast as the data arrive from the TV card. With file playback it can be a little bit slower.

the codec (PC) is to slow, therefore DVB Source has to through data away because there are more data arriving from the TV card then the decoder can process.

 

About other DVB Software, the DVBViewer is a 32 bit application and need 32 bit codecs.

http://www.DVBViewer.tv/forum/topic/56578-64-bits-version/

And LAV Filter (hevc) codec is more effective ion the 64 bit version.

 

Try the HEVC Lentoid decoder, it should work better on a slow PC.

Perhaps you missed my comment on SmartDVB working without issue.  I use the same decompression codecs and renderer.  The ONLY difference is the source filters.

If this was hardware issue (slow PC) than the same issue would occur with other dvb viewing software.

I am using a gtx960 with DXVA2.  When viewing live 4k hevc high bitrate feeds with SmartDVB the cpu usage is <10%.  All decoding is done in GPU.

 

The filter graph looks something like this:

Tuner >> Capture >> Source >> DeMux >> Decoder >> Renderer

 

The ONLY filter changing in the graph stream between DVBViewer and SmartDVB is the Source filter.  And since this issue ONLY occurs with the combination of LIVE TV and DVBViewer the issue most likely is occurring between the Capture and DVBSource filters.

 

Share this post


Link to post
craig_s    22
craig_s
Am 12.9.2017 um 18:51 schrieb merkin stare:

I am using a gtx960 with DXVA2.

 

I am using a GTX960 with DXVA2 and 4k HEVC live TV is running like a charm. May be you only have not the right settings in LAV Video Decoder.

While 4k HEVC live TV is running the filter graph should look like this:

 

uc?id=0ByfqFwPSU1vzRGZvcGNDRDlZSVk

 

 

Then klick on "LAV Video Decoder". While 4k HEVC live TV is running set everything like here:

If you changed anything here you have to restart playback (or simply restart DVBViewer).

If everything works right you should see your GTX960 as "Active Hardware Accelerator"- but only -> while 4k HEVC live TV is running.. :)

 

uc?id=0ByfqFwPSU1vzOE1uMGFidzdJc1k

Share this post


Link to post
nuts    51
nuts

DXVA2 is working for @merkin stare ...  

 

I think he is runnig out of RAM for LiveTV, but I have no idea why this happens only for LiveTV and not for file playback. :( 

@merkin stare Could you see any differences between LiveTV and file playback in RAM usage or CPU load? You are using 2gb or 4gb version of GTX960? 

 

P.S. Any chance to receive high bitrate HEVC 4K @ Astra 19,2? 

 

Share this post


Link to post
merkin stare    0
merkin stare

Hi,

I appreciate the feedback but I made a coupe mistakes.

1. I am actually running GTX950 (2GB)

2. File playback of high bitrate 4k hevc recording DOES also produce buffer errors. (I was playing low bitrate 4k hevc by accident)

 

So let me sum up the current situation:

High bitrate 4k hevc (like sample in original post) produces buffer errors in DVBSource filter during live playback and during recording playback in DVBViewer

- CPU usage 25-30%

- RAM usage 1.59GB

- LAV active decoder: dxva2n

High bitrate 4k hevc plays perfectly fine during live playback and during recording playback in SmartDVB 

- CPU usage 5-10%

- RAM usage 1.57GB

- LAV active decoder: dxva2n

Low bitrate 4k hevc produces NO buffer errors in DVBSource filter during live playback and during recording playback in DVBViewer 

- CPU usage 3-8%

- RAM usage 1.47GB

- LAV active decoder: dxva2n

 

@nuts - I cannot receive Astra 19,2.  What is the highest bitrate 4k hevc transmission you can receive?

Did you try to playback the file I uploaded?

Share this post


Link to post
nuts    51
nuts

Hm ok. What about high bitrate 4K HEVC recording playback in DVBViewer without DVBSource?

I am asking this because DVB Source usually doesnt make problems.

After your last post it could also be an issue with the video renderer (i.e. running out of GPU RAM with DVBViewer).

You are using EVR or EVR Custom? Did you tried both or madVR video renderer?

 

 

 

 

Share this post


Link to post
craig_s    22
craig_s

Ok I did not test 4k on DVBViewer since a longer period - just tried and there seem to be some new issues now:

Astra UHD, SES UHD, Fashion and Pearl 4k - all live have Buffer overflows! That's new and did not happen some months ago.

 

Also new, any recorded UHD .ts (10-bit, 24-60 fps) shows heavy buffer overflows during playback. But only using DVBSource. LAV Splitter plays them clean.

I have no Idea what happened meanwhile. Unfortunately I have no time to check much at the moment. May be more time some days later..

Share this post


Link to post
craig_s    22
craig_s

..the defect was quickly found, the LAV nightly I used (0.70.2-7) had a wicked bug: no hardware acceleration with DVBSource !?! Things happen...:blink:

Now on 0.70.2-74 hardware acc. is back again -> 4k file and live flawlessly with DVBSource as ever since, puh!

 

Suggestion for @merkin stare

 

CPU usage 5-10% is barely ok but  25-30% is too much.

I have 1-2% (both, file and live TV, i7-4790K) which is a fast CPU but even with slow ones 4k (10-bit) should not go over 6-10% using a GTX 950/960.

There must be something wrong in your system, if it's not the LAV Nightly ..?

 

-> always and again and again.. check in "LAV Video Decoder" settings if GTX950 is shown as "Active Hardware Accelerator" during live/playback!

(see image above).

If there is "avcodec" and "none" hardware acceleration is not active.

Edited by craig_s

Share this post


Link to post
merkin stare    0
merkin stare

@craig_s - but did you try playing the high bitrate recording I shared in the op?

 

Again, I have no issue with live or recording with SmartDVB.  I do not change codecs or settings when testing between apps.  For this reason issue must be related specifically to DVBViewer.

Share this post


Link to post
merkin stare    0
merkin stare

@nuts - Where is the setting to use a different source filter for file playback?

Share this post


Link to post
craig_s    22
craig_s
vor 7 Minuten schrieb merkin stare:

Where is the setting to use a different source filter for file playback?

 

Options > directX > DVBViewer Filter - uncheck .ts

 

Yes your 4ktest-1_0001.ts runs well with LAV Splitter and crappy with DVBSource. Why ..?

Edited by craig_s

Share this post


Link to post
craig_s    22
craig_s
vor 28 Minuten schrieb merkin stare:

Again, I have no issue with live or recording with SmartDVB.  I do not change codecs or settings when testing between apps.  For this reason issue must be related specifically to DVBViewer.

 

64-Bit Vers. of SmartDVB? DVBViewer is a 32-Bit app. In 64-Bit 4k might run even without hardware acceleration depending on your CPU.

Share this post


Link to post
merkin stare    0
merkin stare
51 minutes ago, craig_s said:

Options > directX > DVBViewer Filter - uncheck .ts

thanks

 

51 minutes ago, craig_s said:

Yes your 4ktest-1_0001.ts runs well with LAV Splitter and crappy with DVBSource. Why ..?

Its crappy with DVBSource because DVBSource is buggy and cannot handle very high bitrate content.  That is the whole point of this thread.

 

33 minutes ago, craig_s said:

64-Bit Vers. of SmartDVB? DVBViewer is a 32-Bit app. In 64-Bit 4k might run even without hardware acceleration depending on your CPU.

I am using 32-bit version of SmartDVB.

Share this post


Link to post
nuts    51
nuts

 

vor einer Stunde schrieb merkin stare:

Again, I have no issue with live or recording with SmartDVB.  I do not change codecs or settings when testing between apps.  For this reason issue must be related specifically to DVBViewer.

Yeah and we are trying to find out what is going on. ;) 

 

But if its working for both of you with LAV Source instead of DVB Source in DVBViewer it seems to be related to DVB Sourcefilter.

I will try to reproduce it and report the issue to Griga. 

Share this post


Link to post
Derrick    62
Derrick
vor 14 Minuten schrieb merkin stare:

thanks

 

Its crappy with DVBSource because DVBSource is buggy and cannot handle very high bitrate content.  That is the whole point of this thread.

 

I am using 32-bit version of SmartDVB.

How high is the bitrate of your service? I never had  problems with high rates and the source filter. Must be another reason. Could you upload a sample?

Share this post


Link to post
merkin stare    0
merkin stare
2 minutes ago, Derrick said:

How high is the bitrate of your service? I never had  problems with high rates and the source filter. Must be another reason. Could you upload a sample?

sample is in the first post.  its ~70mb/s

even reports of the Octagon SF4008 choking on these feeds over here.

Share this post


Link to post
nuts    51
nuts
vor 4 Minuten schrieb Derrick:

How high is the bitrate of your service? I never had  problems with high rates and the source filter. Must be another reason. Could you upload a sample?

Post 1. ;)

Share this post


Link to post
Derrick    62
Derrick

Yes,my fault.

 

Share this post


Link to post
craig_s    22
craig_s
vor 3 Stunden schrieb merkin stare:

Its crappy with DVBSource because DVBSource is buggy and cannot handle very high bitrate content. That is the whole point of this thread.

 

Additional to Derricks " I never had problems with high rates and the source filter"

- ok, overall bitrate 72,7 Mbit/s of your sample is high but I have a 75,8 and a 79,1 Mbit/s HEVC here running clean with DVBSource as if as if nothing had happened. Media Info shows nearly the same video parameters.

 

If DVBSource had failed in the past it was often due to audio tracks. So I cut the first 6 sec of your sample with TS Doctor and removed the mp2 track.

Now DVBSource played the sample clean but did not find any audio track (LAV does). May be this helps the developers?

Share this post


Link to post
merkin stare    0
merkin stare

ok I switched to LAV splitter and recording plays perfect (~7% CPU/1.54GB RAM).  There is definitely something wrong with DVBSource filter.

How do we bring this to the developers attention?

Share this post


Link to post
craig_s    22
craig_s

They are not living on Mars like in hardware forums :) and read more here then one would think. But it needs time...

Share this post


Link to post
merkin stare    0
merkin stare

@craig_s - can you share samples or mediainfo xml of your other high bitrate hevc recordings?

Share this post


Link to post
craig_s    22
craig_s

They are no recordings but demo samples. One of them is here http://4kmedia.org/sony-camping-in-nature-4k-demo/

The other is the SDR Vers. of the same video. Maybe download nearby there?

Just found it  by googl "sony camp hdr". www.4ktv.de from where I had them does not host them any more.

 

p.s. regard that HDR looks like flat colors without HDR TV.

Edited by craig_s

Share this post


Link to post
Griga    359
Griga
3 hours ago, craig_s said:

If DVBSource had failed in the past it was often due to audio tracks.

 

If expected streams are missing try to increase Tweaker.exe -> Search Depth in MB (German: Suchtiefe in MB). E.g. a sample starting with  a contiguous 6 MB chunk of video data may prevent DVBViewer from finding the subsequent audio data.

Share this post


Link to post
merkin stare    0
merkin stare
4 hours ago, craig_s said:

If DVBSource had failed in the past it was often due to audio tracks. So I cut the first 6 sec of your sample with TS Doctor and removed the mp2 track.

Now DVBSource played the sample clean but did not find any audio track (LAV does). May be this helps the developers?

 

I can confirm this behavior.  The posted transport stream in the op has two audio pids:

1. 0x01E2 (EAC3)

2. 0x01E3 (MPEG Audio)

 

I made three different files from the original file:

1. Stripped both audio pids from the transport stream

2. Stripped pid 0x01E2 from the transport stream

3. Stripped pid 0x01E3 from the transport stream

 

The only files that plays WITHOUT buffer overflows in DVBSource filter is the transport stream with BOTH audio pids stripped from the original file.

 

All three files play perfect with LAV Splitter.

 

52 minutes ago, Griga said:

 

If expected streams are missing try to increase Tweaker.exe -> Search Depth in MB (German: Suchtiefe in MB). E.g. a sample starting with  a contiguous 6 MB chunk of video data may prevent DVBViewer from finding the subsequent audio data.

The problem is once DVBViewer "finds" the audio data buffer overflows will start to occur in DVBSource filter.

Share this post


Link to post
merkin stare    0
merkin stare
4 hours ago, craig_s said:

They are no recordings but demo samples. One of them is here http://4kmedia.org/sony-camping-in-nature-4k-demo/

The other is the SDR Vers. of the same video. Maybe download nearby there?

Just found it  by googl "sony camp hdr". www.4ktv.de from where I had them does not host them any more.

 

p.s. regard that HDR looks like flat colors without HDR TV.

I downloaded a couple demos.  They play perfect, but they are mp4 containers and therefore are using LAV Splitter, not DVBSource. 

I do not know how to force DVBViewer to use DVBSource filter for mp4 container.

 

Thanks for sharing.

Share this post


Link to post
craig_s    22
craig_s
vor 8 Stunden schrieb merkin stare:

but they are mp4 containers and therefore are using LAV Splitter, not DVBSource.

 

Sorry, my fault

 

vor 9 Stunden schrieb Griga:

If expected streams are missing try to increase Tweaker.exe -> Search Depth in MB (German: Suchtiefe in MB). E.g. a sample starting with  a contiguous 6 MB chunk of video data may prevent DVBViewer from finding the subsequent audio data.

 

Ok, 'increase Tweaker.exe -> Search Depth in MB' solved a lot.

I increased only from 6 to 8 MB and DVBSource finds now the aac track in all these professional demos:

 

60fps HEVC 10-bit 61,8 Mb/s .ts

http://4kmedia.org/lg-daylight-hdr-uhd-4k-demo/

or non-HDR, here I checked Italy, China, Brazil..:

http://4kmedia.org/tag/samsung/page/2/

Play them by DVBSource - no audio found with Search Depth 6 MB.

 

 

But unfortunately this is no solution for merkin stare's video.

 

vor 9 Stunden schrieb merkin stare:

The problem is once DVBViewer "finds" the audio -> data buffer overflows will start to occur in DVBSource filter.

 

Also my cutting or 1 (of 2) audio track removing experiments do not help - with Search Depth ~18 MB DVBSource finds now the audio in the modified samples but also the buffer overflows are back.

 

Edited by craig_s

Share this post


Link to post
craig_s    22
craig_s

p.s.

Playing the above linked demos DVBSource is 'riding on the edge',

- I would say a maximum of 'Allocated Buffers' = 1000 is too less for such high HEVC bitrate:

 

DVBS_on_the_edge.thumb.png.34815da63e17ad1ddd44df8e46ff3fe5.png

 

At the moment 'Queued Video Buffers' want to hop over 1000 buffer overflows will start to occur.

..possible to increase 'Allocated Buffers'?

 

 

I have some 120 Mb/s (!) H.264 samples here which show 'only' -

Allocated Buffers = 1000

Queued Video Buffers ~ 970

 

'Normal' Astra UHD recordings ~ 24 Mb/s:

Allocated Buffers ~ 730

Queued Video Buffers ~ 430

 

720p recordings 12,8 Mb/s:

Allocated Buffers ~ 410

Queued Video Buffers ~ 370

 

Edited by craig_s

Share this post


Link to post
Griga    359
Griga
1 hour ago, craig_s said:

I would say a maximum of 'Allocated Buffers' = 1000 is too less for such high HEVC bitrate:

 

Maybe it's rather the buffer size. The DVBSource is using a default size of 8 kb per buffer, yielding a maximum of 8 MB that can be buffered. However, there are already special mesaures for the Lentoid HEVC Decoder because it requires buffers that can hold the maximum PES packet size that occurs in the stream (see here, Miscellaneous section).

 

Here DVBViewer & DVBSource & Lentoid Decoder play almost all UHD HEVC live streams from Astra 19° and 28° East fluently without buffer overflows on a PC with i7 CPU and without hardware accelerated HEVC decoding (-> Geforce GTX 660, limited support for 8 bit HEVC, none for 10 bit) if I set the buffer size to 1024 = 1 MB per buffer. Even the sample above is played without buffer overflows if I set the buffer size to 2048 = 2 MB, but the decoder seems to have some difficulties to decode it correctly without artifacts.

 

Now the question is if enlarged buffers also have a beneficial effect if other decoders are used. That's what I'm currently investigating. Unfortunately I can't reproduce what you are reporting because I don't have appropriate hardware. LAV without hardware accelerated HEVC decoding is just a disaster ;) However, the Cyberlink HEVC Decoder also seems to work better with enlarged buffers. Buffer overflows occur if I try to play the sample above with the default buffer size. With 1 MB it's ok.

Share this post


Link to post
craig_s    22
craig_s

Lentoid with buffer size to 2048 = 2 MB plays the Sams.demos (52Mb/s) ok (~60% CPU). Below 2048 Artifacts coming from the picture bottom.

Plays also merkin stare's video (72Mb/s) fluently (~60% CPU) with sound but wrong colors + artifacts.

 

Buffer size to 2048 has no influence on hardware accelerated LAV.

Edited by craig_s

Share this post


Link to post
merkin stare    0
merkin stare

As mentioned in the op, I increased the buffer size in DVBSource.ini, but overflows still occur in DVBSource filter with both live and recorded high bit rate content I shared in the op.

 

Here is a schedule of the high bitrate 4k transmissions that have this issue:

https://support.directv.com/dtv-programming/4k-events

 

I think we should not overlook the fact that buffer overflows only occur if there is Audio and Video streams present.  No buffer overflow issues when the audio pids are removed.

Share this post


Link to post
Griga    359
Griga
17 minutes ago, craig_s said:

Buffer size to 2048 has no influence on hardware accelerated LAV.

 

It isn't activated for LAV. DVBViewer checks the decoder and sets a flag for DVBSource if it recognizes the Lentoid decoder.

Share this post


Link to post
craig_s    22
craig_s
vor 6 Minuten schrieb merkin stare:

I think we should not overlook the fact that buffer overflows only occur if there is Audio and Video streams present.  No buffer overflow issues when the audio pids are removed.

 

YES ! Please don't forget that Griga.

 

 

vor 1 Stunde schrieb Griga:

Here DVBViewer & DVBSource & Lentoid Decoder play almost all UHD HEVC live streams from Astra 19° and 28° East fluently without buffer overflows on a PC with i7 CPU and without hardware accelerated HEVC decoding

 

Here (GTX960, i7-4790K, buffer size 2048 ) Lentoid plays SES UHD live stream flawlessly at only 25% CPU.

But SES UHD live stream is peanuts compared to what merkin stare is watching!

Share this post


Link to post
Griga    359
Griga
8 hours ago, craig_s said:

But SES UHD live stream is peanuts compared to what merkin stare is watching!

 

Well, seems to be a bit blown up without much behind it, thus defying the purpose of HEVC. With 70 MBit/s one could as well use the good old MPEG2 for 4k. The provider should use a better encoder ;) The 20 MBit/s HDR streams on Astra 10715 H are more challenging for software decoding, and compared to the 10000 x 1700 VR Demo on Astra 11612 H merkin is watching peanuts, regarding the video resolution :)

 

Share this post


Link to post
merkin stare    0
merkin stare
25 minutes ago, Griga said:

 

Well, seems to be a bit blown up without much behind it, thus defying the purpose of HEVC. With 70 MBit/s one could as well use the good old MPEG2 for 4k. The provider should use a better encoder ;) The 20 MBit/s HDR streams on Astra 10715 H are more challenging for software decoding, and compared to the 10000 x 1700 VR Demo on Astra 11612 H merkin is watching peanuts, regarding the video resolution :)

 

There is no denying that 70mb/s is overkill for hevc, but that is a different topic of conversation for a different time.  Also the encoders have nothing to do with this issue.

The provider is renting a full transponder and has chosen to use the full bandwidth of the transponder during the event.  We cant assume why they made that decision, but the engineers who designed the encoder and the provider using the encoder is definitely smarter than us.

 

Here are the facts:

1.  SmartDVB and other software plays the content without issue

2.  DVBViewer source filter chokes unless audio pids are removed from the stream

 

This screams bug with DVBViewer Source filter and all I here are unfounded excuses.

 

Do you have a bitcoin address?  Perhaps we can start a fund to get you a video card with full hevc decoding.  Its surprises me that you don't have one for testing.  I would love to be the first to donate to the cause.

Edited by merkin stare

Share this post


Link to post

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now


×