Original stream limited to 3Mb/s on a 500Mb/s upload link

[Victor Safta]

Hello,

 

I am trying to stream my DVB-C for personal use using DVBViewer Media Server with the best possible quality. So I do not want to transcode.

For some reason, whenever I connect to the media server and try to stream original streams, the video is always buffering.

I check the upload traffic on the server and I see it never gets more than 3 Mb/s.

 

On the other hand, if I transcode good quality, the traffic get way above 15 Mb/s but the CPU gets overloaded on the server.

 

My question is why the direct stream is limited to 3Mb/s? Because I have good bandwidth both on the server and on the client, I would like to use the original streams.

 

Thank you for your help.

 

My configuration is:

Server side: Windows 10-v1803, i3 2Ghz, 4GB Ram - 1600 Mhz, DVBViewer Media Server 2.1.2.0, Astrometa tuner

Upload link: 500 Mb/s

 

Client side: Kodi with DVBViewer add-on, SIPTV on smart tv, VLC....

Download link: 50Mb/s

Edited October 14, 2018 by Victor Safta
insert attachment

[Griga]

The only reasons I can think of is security software, that continuously wants to analyze the output, and/or the port. Transcoded streams are delivered by the web server (default port 8089), original streams by the live stream server (default port 7522) - see Media Server Options (svcoptions.exe) -> Web/UPnP.

 

As test you can also let the web server deliver the original stream by changing the port number in the URL.

[Victor Safta]

14 hours ago, Griga said:

The only reasons I can think of is security software, that continuously wants to analyze the output, and/or the port. Transcoded streams are delivered by the web server (default port 8089), original streams by the live stream server (default port 7522) - see Media Server Options (svcoptions.exe) -> Web/UPnP.

 

As test you can also let the web server deliver the original stream by changing the port number in the URL.

I tested both ports (8089, 7522).

I had to manually open 7522 in the firewall and forwarded in the router.

 

The result is the same on both. No more than 3 Mb/s.

[Griga]

You are streaming over the internet? You should have mentioned it.

 

11 hours ago, Victor Safta said:

I had to manually open 7522 in the firewall and forwarded in the router.

 

That's an absolute no-no because the live streaming server is not protected at all. Forwarding the port in the router opens the door for the whole world, and there are search engines that make it easy to detect media servers. Only the web server provides a password protection. Hence the web server port should be the only one that is forwarded in the router.

 

11 hours ago, Victor Safta said:

The result is the same on both. No more than 3 Mb/s.

 

There is no bandwidth limitation in the Media Server. In my home network there are no problems with original 20 MBit/s UHD streams. Maybe someone else who has more experience with streaming over the internet than me has an idea what is going on in your case.

 

[Victor Safta]

28 minutes ago, Griga said:

You are streaming over the internet? You should have mentioned it.

 

 

That's an absolute no-no because the live streaming server is not protected at all. Forwarding the port in the router opens the door for the whole world, and there are search engines that make it easy to detect media servers. Only the web server provides a password protection. Hence the web server port should be the only one that is forwarded in the router.

 

 

There is no bandwidth limitation in the Media Server. In my home network there are no problems with original 20 MBit/s UHD streams. Maybe someone else who has more experience with streaming over the internet than me has an idea what is going on in your case.

 

2

 

Thank you Griga.

I closed down 7522 and still trying to figure out the issue.

 

I use transcoding for now and I get a reasonable result but I still want to fix the direct stream.

 

PS. Direct streaming on LAN is perfect so the server is ok.

I checked all the permissions and limitations in the firewall. Everything looks good.

[Griga]

Which format are you using for transcoded TV?

 

Maybe there is setting for a format (mime type) dependent bandwith limitation involved somewhere... the mime type of the original transport stream is "video/MP2T".

[Victor Safta]

54 minutes ago, Griga said:

Which format are you using for transcoded TV?

 

Maybe there is setting for a format (mime type) dependent bandwith limitation involved somewhere... the mime type of the original transport stream is "video/MP2T".

 

You mean in my router configuration?

Or the Media Server?

[Griga]

The Media Server neither contains a bandwidth limitation nor a setting for it.

[Victor Safta]

On 10/16/2018 at 12:48 PM, Griga said:

The Media Server neither contains a bandwidth limitation nor a setting for it.

 

I ran some more tests.

I checked all networks settings/limitations on the firewall and on the router.

 

It seems the problem lies with the Media Server.

I recorded media with DVB and direct streamed it with no problem through the Media Server. The stream goes to 20Mb with no problem. The same port is used: 8089

Each time I go back to streaming Live Tv... the stream goes back down to 3Mb.

 

I will try to reinstall the server and get back with more info.

 

Thank you.

[Griga]

56 minutes ago, Victor Safta said:

Each time I go back to streaming Live Tv... the stream goes back down to 3Mb.

 

But this does not happen within your home network, does it? So it comes down to the question: What makes the difference between streaming across LAN/WLAN and the internet? From the Media Server's point of view there is no difference. There are no explicite data rate limitations depending on the target.

 

However, there is a difference between file and live streaming (or processing in general): Data from a file can be read on demand (pull mode). Live data has to be processed as it comes because the data source can't be influenced (push mode). DVB card drivers don't deliver data continuously, but in chunks. If an application can't process it or pass it on in time, buffer overflows and data loss occur.

 

The difference between "original" and transcoded streaming is FFmpeg being involved (DMS passes a URL to a FFmpeg process -> FFmpeg receives live data via localhost from the DMS -> transcodes it -> passes it back through a pipe / shared memory mechanism -> DMS sends it to the client). Maybe there is more buffering (= more delay) involved altogether.

 

[Griga]

Buffering seems to be the key for solving this problem, but different than I thought...

 

https://blogs.technet.microsoft.com/networking/2017/05/08/windows-network-performance-suffering-from-bad-buffering/

http://cygwin.1069669.n5.nabble.com/Limited-Internet-speeds-caused-by-inappropriate-socket-buffering-in-function-fdsock-winsup-net-cc-td131649.html

 

The "original" live streaming calls a network library function that uses the mentioned SO_SNDBUF option to set up a send buffer for the server. Transcoded live streaming doesn't.

 

However, I'm not clear on the scope of this Networking Blog yet. The guy writes about TCP (which includes HTTP), internet and Windows 10. What about other Windows versions? What about LAN/WLAN? Additional investigations / tests are required, and I need you as tester.

 

[Griga]

P.S.

 

1 hour ago, Griga said:

However, I'm not clear on the scope of this Networking Blog yet. The guy writes about TCP (which includes HTTP), internet and Windows 10. What about other Windows versions?

 

https://msdn.microsoft.com/en-us/library/windows/desktop/bb736549(v=vs.85).aspx

 

Quote

Dynamic send buffering for TCP was added on Windows 7 and Windows Server 2008 R2. By default, dynamic send buffering for TCP is enabled unless an application sets the SO_SNDBUF socket option on the stream socket.

 

[Victor Safta]

2 hours ago, Griga said:

However, I'm not clear on the scope of this Networking Blog yet. The guy writes about TCP (which includes HTTP), internet and Windows 10. What about other Windows versions? What about LAN/WLAN? Additional investigations / tests are required, and I need you as tester.

 

2

 

Thank you for your help, Griga.

I am more than willing to test anything.

Even more, I can grant you full access to the whole infrastructure if you want to take a look. 

[Griga]

Check your PMs...

[Griga]

The reply that I got as PM from @Victor Safta:

 

Quote

Wow... This is magic

 

Instant change with this one.

SD is running flawlessly.

HD is buffering now and then but it's working.

I can see a limit around 13Mb which is far more than the previous 3Mb.

I think my best HD requires just a little more in order to be perfect.

 

I copy it here because I think the effect of not using a static send buffer for live streaming over the internet (thus enabling the dynamic TCP send buffering of Windows) is of public interest.

 

A throughput increase from 3 MBit/s to 13 MBit/s is quite something... however, not enough for reliable HDTV streaming. What is the expected maximum data rate for media streams in your setup? Please note that some of the bandwidth is eaten by the HTTP/TCP protocol overhead etc. How much do you achieve with file streaming (recorded TV)?

 

Please also note the influence of the DMS Options -> Web/UPnP -> Live Stream Server settings on the data rate: Ticking "Remove video filler data" and unticking all additional streams may lower the data rate by some percent.

 

Regarding the DMS code, there are only two options left: I can influence the maximum size of buffered data within the DMS (currently 8 MB = 4.4 seconds of 15 MBit/s data) and the size of the chunks that are passed to the Windows TCP/IP stack (currently 8084 bytes = 43 TS packets)..

 

[trudeh]

@Griga

Wir sind ja hier eigentlich im deutschen Bereich des Forums and mei inglisch makes mi so isy nobadi after...

 

Wenn ich das jetzt grob verstanden habe geht es um eine Begrenzung bei einem transcodierten Stream übers Internet.

Ich hab auch teilweise Probleme bei Streamen, die nach diversen Test und einem Tarifwechsel nicht an einem zu geringen Upload liegen können.

Wenn ich z.B. ein 3600 Profil gewählt habe, geht die Datenrate teilweise nicht über 2500 raus, was dann natürlich zu Haklern und Abbrüchen führt.

Ist das Problem vergleichbar und gibt es dazu bald eine Lösung?

 

[Griga]

34 minutes ago, trudeh said:

Wenn ich das jetzt grob verstanden habe geht es um eine Begrenzung bei einem transcodierten Stream übers Internet.

 

No. This topic is about the data rate of untrancoded (original) streams.

 

36 minutes ago, trudeh said:

Wir sind ja hier eigentlich im deutschen Bereich des Forums 

 

I've moved this topic to the English section. I didn't notice that it was placed wrongly. Thanks for the hint.

 

[Victor Safta]

On 10/19/2018 at 5:52 PM, Griga said:

 

A throughput increase from 3 MBit/s to 13 MBit/s is quite something... however, not enough for reliable HDTV streaming. What is the expected maximum data rate for media streams in your setup? Please note that some of the bandwidth is eaten by the HTTP/TCP protocol overhead etc. How much do you achieve with file streaming (recorded TV)?

 

 

 

I believe the data rate required for my HD streaming should not be more than 20Mb.

This is what I get when streaming recordings of my HD source.

 

Currently, my HD streaming is reasonable with very rare buffering on the client side.

 

[Griga]

On 10/21/2018 at 9:21 AM, Victor Safta said:

I believe the data rate required for my HD streaming should not be more than 20Mb. 

This is what I get when streaming recordings of my HD source.

 

Now I found time to analyze what happens if the web server (port 8089) sends a large file >= 64 kb. The underlying network library allocates a 64 kb buffer for reading 64 kb chunks from the file and additionally asks Windows by using the SO_SNDBUF socket option to use a static 64 kb TCP send buffer.

 

This is a bit surprising because the static send buffer is what I've lately kicked for live streaming, yielding a significantly increased data rate. A possible interpretation of the results is, that sending a large file via the internet is working well with a static TCP buffer because data can be read on demand whenever sending something is possible. In contrast, live streaming rather requires dynamic buffering because data must be passed on whenever it comes. You can't wait until the data can be sent or free buffer space is available.

 

However, that's just a guess. Finding out more about it would require more experimental testing. There are several layers involved that make things complicated: The DMS code, the underlying network library that encapsulates the Windows TCP/IP stack, the network adapter and its driver, the router, your internet connection and your provider with their way of handling data transfers... all this makes it hard to judge why certain measures yield certain results.

 

[Victor Safta]

UPDATE: On top of the streaming problems I started to experience bad upload speeds. After checking everything including my ISP, I decided to factory reset my Mikrotik router.

After the factory reset everything is fine, including the HD broadcasting.

 

Unfortunately, I still don't know exactly what caused the issues, but for sure it was related to my router.

 

I have been using the server for more than one week now and I must say that the DVBViewer Media Server is a very reliable solution.

Thank you for your help!

[Arne Stoffregen]

Hello @ all,

i have the same problem. On LAN it works, but over a link with higher latency (internet) a cant get more then 3-4Mbits throughput.

Server is Windows Server 2008R2 and client Windows 10.

My workaround at this moment is, to tunnel port 8089 and 7522 over ssh, which works well. That means there is no problem with the bandwith or with my router.

@Griga What was your solution you sent to Victor Safta?

 

 

[Griga]

31 minutes ago, Arne Stoffregen said:

What was your solution you sent to Victor Safta?

 

https://www.dvbviewer.tv/forum/topic/61806-original-stream-limited-to-3mbs-on-a-500mbs-upload-link/?do=findComment&comment=477002

 

It will be part of the upcoming next Media Server release.