FFMPEG with QSV Decoding and Encoding Live Streams

[Josezkaos]

As I commented this issue in another thread dedicated to NVIDIA transcoding, I'll create a new one for this as I think is interesting.

 

I've recorded a short clip from a DVB-T stream with DVBViewer:

 

http://www.mediafire.com/?aoe7406d87h3k

 

If QSV Decoder would work with live streams, this would be enought:

Quote

 

ffmpeg -hwaccel qsv -vcodec h264_qsv -i dvbt_Stream.ts -vcodec h264_qsv -vf deinterlace_qsv -y test.ts

 

 

 

But that fails.

 

Somehow, if the stream is copied and piped into another ffmpeg ... it works:

Quote

 

ffmpeg -i dvbt_Stream.ts -vcodec copy -acodec copy -f mpegts -| ffmpeg -hwaccel qsv -vcodec h264_qsv -f mpegts -i pipe: -vcodec h264_qsv -vf deinterlace_qsv -y test.ts

 

 

 

Stream is copied, decoded with H264_QSV and then encoded with H264_QSV, all under HW Acceleration.

 

But this workaround is not valid for Recording Services:

Quote

 

Cmd=-i "{infile}" -vcodec copy -acodec copy -f mpegts -| ffmpeg -hwaccel qsv -vcodec h264_qsv -f mpegts -i pipe: -vcodec h264_qsv -vf deinterlace_qsv -y "{outfile}"

 

 

 

Seems the pipe lunches another instance which can't reach RS output pipe ({outfile}).

 

Why use QSV on Decoder and Encoder for transcoding live streams?

 

1. Decoded frames are in Video Memory, not in System Memory this way, so Encoding runs way more faster saving time moving those decoded frames between memories.

2. Deinterlacing with QSV avoids using CPU, once again faster and saving CPU for other processes running.

 

Does anyone know how to get RS to transcode live streams using QSV for Decoding and Encoding?

 

Thanks in advance (specially to Griga and majstang who replied in the other thread).

[majstang]

Hi Josezkaos,

 

although the FFMPEG stuff is kind of a blank area for me (not much testing done in my case) my question is how are you trying to launch it through RS? Via a task? If so the right placeholders has to be used.

 

Christian tells us:

Quote

{DVBVIEWER_PATH} can be used as placeholder for the DVBViewer / Recording Service installation path (including the trailing backslash) in the task filename, working directory and in parameters, e. g. {DVBVIEWER_PATH}ffmpeg.exe. Please note that placeholders must be uppercase!

 also the right placeholder for the infile is {SOURCE_FILE}. Outfile I dont know. 

 

Also search for Transcoded Streaming in this thread http://www.DVBViewer.tv/forum/topic/19629-recording-service-beta/?do=findComment&comment=431016

Edited January 9, 2018 by majstang

[Josezkaos]

The ffmpegpref.ini is used on live streams:

 

Quote

[PIPE TEST]

Cmd=-i "{infile}" -vcodec copy -acodec copy -f mpegts -| ffmpeg -hwaccel qsv -vcodec h264_qsv -f mpegts -i pipe: -vcodec h264_qsv -vf deinterlace_qsv -y "{outfile}"

maxWidth=1920
maxHeight=1080
MimeType=video/mpeg
Ext=.ts
Delay=2000

 

I'm using VLC as player, and in FFMPEGPREF.INI the {infile} and {outfile} are used in all prefs available by default in RS.

 

The other examples are adapted for the recording uploaded to mediafire, but the result is the same as using a live stream from RS ("http://localhost:1212/upnp/channelstream/3713217958821953687%7ClaSexta%20HD%20%28spa%29.ts").

 

 

[majstang]

Ok, check out this german thread

[Josezkaos]

I figured out what's the real reason why this command is not working in RS:

 

Quote

 

ffmpeg -i dvbt_Stream.ts -vcodec copy -acodec copy -f mpegts -| ffmpeg -hwaccel qsv -vcodec h264_qsv -f mpegts -i pipe: -vcodec h264_qsv -vf deinterlace_qsv -y test.ts

 

It's not (only) about the pipe, it's the HWACCEL option.

 

FFMPEG supports hardware acceleration, but when it comes to QSV it needs to render a DirectX device first.

 

As RS is running in a Windows Service context, DirectX is not available (DirectX 11 does under some limitations, but FFMPEG does not use it to access QSV devices) and therefore FFMPEG cannot create an instance of DirectX QSV Devices to decode video when called from RS.

 

In short, what we need is to fix the live stream so H264_QSV decoder can handle it (that's solved using the "-vcodec copy -acodec copy" or "-c copy" and a pipe to another FFMPEG instance), and to find a context where FFMPEG can render a DirectX QSV Device and use it instead of Windows Service Context.

 

I'm developing a simple Java app which performes like man in the middle, launching FFMPEG from Windows User Context so FFMPEG can access to DirectX QSV Devices ... it seems to work, but it still needs some fixes and tunning.

 

By the momment, I'm able to decode, deinterlace and encode using only QSV hardware acceleration as well as inject it into RS.

 

I'll keep you all up with news.

 

 

[majstang]

Yeah, the MS introduced Service context security limitations is a huge PITA! Im using a named pipe solution running scripts unrestricted for 8 years now. Have been suggested that solution being built into the RS/DMS equally as long, but no luck so far. I can understand it cuz its not the cleanest of solutions Although MS has API instructions/documentation themselves how to do something like it. Good luck to you and your nifty endeavours (they are very interesting and fun to follow)

[Josezkaos]

Here it is a little comparaison...

 

This one is using RS default transcoding WEBM HD 3600Kbps (aka: 720p, 25fps):

 

 

CPU using 70-80%, GPU at 0%, 

 

This one is using my http server (which calls FFMPEG using QSV) at 3M (aka: 1080p, 50fps):

 

CPU doesn't change activity (stays at 33%), GPU jumps to 40-60%.

 

Video stream plays flawlessly 

[janee]

Hi Josezkaos,

nice project!!! This is very interessting.

I never got it working. Maybe you can share  the commandline for ffmpeg. I use a Kaby Lake CPU with Iris Plus Graphics 640.

Thanks!

 

e.g If I try this:

ffmpeg -analyzeduration 1500k  -i "http://127.0.0.1:7522/upnp/channelstream/transcode/2359890582721931325%7CDas%20Erste%20HD.ts" -vcodec copy -acodec copy -f mpegts -| ffmpeg -hwaccel qsv -vcodec h264_qsv -f mpegts -i pipe: -vcodec h264_qsv -vf deinterlace_qsv -y "c:\temp\output.ts"

 

I got errors.

Spoiler

[NULL @ 0000023965c9cc40] missing picture in access unit with size 3
Past duration 0.987221 too large
[NULL @ 0000023965c9cc40] missing picture in access unit with size 3
    Last message repeated 1 times
Past duration 0.987221 too large
[NULL @ 0000023965c9cc40] missing picture in access unit with size 3
Past duration 0.987221 too large
    Last message repeated 1 times
[NULL @ 0000023965c9cc40] missing picture in access unit with size 3
Past duration 0.987221 too large
Error while filtering: Cannot allocate memory
Failed to inject frame into filter network: Cannot allocate memory
Error while processing the decoded data for stream #0:0
Conversion failed!
av_interleaved_write_frame(): Broken pipe
Error writing trailer of pipe:: Broken pipe
frame=  654 fps= 56 q=-1.0 Lsize=   12471kB time=00:00:14.41 bitrate=7089.4kbits/s speed=1.22x
video:12102kB audio:416kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
Conversion failed!

 

 

[Josezkaos]

Thanks!

 

Anyway, it is a very simple project .... actually, it just generates the FFMPEG call out of Windows Service Context (whis is the one uses RS) and returns the encoded video to RS.

 

That means, it adds some seconds of delay... But in the other hand, you get efficent HW encoding/decoding/deinterlacing/scaling.

 

My Intel HD Graphics 6000 seems not to support deint+scaling (runs of memmory ... but both works if used alone).

 

To use this project you need:

1 - a new FFMPEGPREF for RS

Quote

[FFMPEGSERVER 3M H264 DEINT]
Cmd=-re -f mpegts -i http://localhost:5566/RSFfmpeg?inputstream={infile}&ffmpegpref=3MDeintH264 -c copy -f mpegts -y "{outfile}"
maxWidth=1920
maxHeight=1080
MimeType=video/mpeg
Ext=.ts
Delay=1
 

2 - Configure the HTTPServer (don't panic, just to indicate a port number, FFMPEG path, and a new properties file path)

 

I've attached the jar, a properties file and even a .Bat to make it launch on windows startup (up to you).

 

You have to keep in mind just one thing: To decode using FFMPEG and QSV, it's needed to use the QSV decoder ... so the FFMPEG call, has a QSV codec.... If you want to use it with MPEG2, H264 and HEVC channels you'll need to create 3 different prefs into the NEW propperties file and use h264_qsv, hevc_qsv, mpeg2_qsv and the RS FFMPEGPref file.

 

Hope it helps!!!

RSFFMPEGServer.zip

[CiNcH]

Quite interesting. Never thought of the none-working hwaccel to be a problem of DirectX not being available to a service.

 

I wonder whether this can more easily and more efficiently be implemented in RS/DMS directly!?

 

Also good to see deinterlace_qsv and scale_qsv in action finally. It would however be desirable to use them in parallel of course. Coming closer to full QSV pipeline...

[Josezkaos]

On 22/1/2018 at 1:22 PM, CiNcH said:

Coming closer to full QSV pipeline...

 

And here it is ...

 

Quote

##H264 TO H264 720P(1Mbps)
1M720pH264=-hwaccel qsv -vcodec h264_qsv -f mpegts -i pipe\: -vcodec h264_qsv  -vf deinterlace_qsv,scale_qsv=1280:720,hwdownload,format=nv12 -b:v 1M -minrate 250K -maxrate 1M -bufsize 2M -y -acodec aac -ab 64k -ac 2 -f mpegts pipe\:

 

This one added to the RSFFMPEGServer's ffmpegprefs.properties and this one to RS ffmpegprefs.properties:

 

Quote

[FFMPEGSERVER 1M HD 720P]
Cmd=-re -f mpegts -i http://localhost:5566/RSFfmpegQSV?inputstream={infile}&ffmpegpref=1M720pH264 -c copy -f mpegts -y "{outfile}"
maxWidth=1920
maxHeight=1080
MimeType=video/mpeg
Ext=.ts
Delay=1

 

(the server doesn't take any resolution params yet ... I'll add it when I have a little time to do it).

[Bernhard Berger]

I am kind of struggling with this setup..:

 

14.02.18 14:39:49.399 TDVBHTTPClient       Client           Kodi/17.6 (Linux; Android 6.0.1; MIBOX3 Build/MOB31Z) Android/6.0.1 Sys_CPU/armv8l App_Bitness/32 Version/17.6-Git:20171114-a9a7a20
14.02.18 14:39:49.399 TDVBHTTPClient ($045C89F0) DoFlashStream    chid=2359890676884057735&preset=24
14.02.18 14:39:49.399 TDVBHTTPClient ($045C89F0) DoFlashStream    192.168.8.111 Kodi/17.6 (Linux; Android 6.0.1; MIBOX3 Build/MOB31Z) Android/6.0.1 Sys_CPU/armv8l App_Bitness/32 Version/17.6-Git:20171114-a9a7a20
14.02.18 14:39:49.399 Converter Start cmd  -re -f mpegts -mpegts_original_network_id 1 -mpegts_transport_stream_id 1041 -mpegts_service_id 11911 -mpegts_service_type 1 -metadata service_name="RTL HD Austria" -i http://localhost:5566/RSFfmpegQSV?inputstream=http://127.0.0.1:7522/upnp/channelstream/transcode/2359890676884057735%7CRTL%20HD%20Austria.ts&ffmpegpref=1M720pH264 -c copy -f mpegts -y "\\.\pipe\Output{DDB8CCD4-AB45-42F2-9D1C-FAEB6E896D83}"
14.02.18 14:39:49.399 SetStandbyBlock      webserver
14.02.18 14:39:49.399 TRecordingEngine     AddReference     webserver: 1
14.02.18 14:39:49.493 FFMPEG               ffmpeg version N-87353-g183fd30 Copyright (c) 2000-2017 the FFmpeg developers
  built with gcc 7.2.0 (GCC)
  configuration: --enable-gpl --enable-version3 --enable-cuda --enable-cuvid --enable-d3d11va --enable-dxva2 --enable-libmfx --enable-nvenc --enable-avisynth --enable-bzlib --enable-fontconfig --enable-frei0r --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libfreetype --enable-libgme --enable-libgsm --enable-libilbc --enable-libmodplug --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenh264 --enable-libopenjpeg --enable-libopus --enable-librtmp --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs --enable-libxvid --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib
  libavutil      55. 76.100 / 55. 76.100
  libavcodec     57.106.101 / 57.106.101
  libavformat    57. 82.101 / 57. 82.101
  libavdevice    57.  8.101 / 57.  8.101
  libavfilter     6.105.100 /  6.105.100
  libswscale      4.  7.103 /  4.  7.103
  libswresample   2.  8.100 /  2.  8.100
  libpostproc    54.  6.100 / 54.  6.100
Option metadata (add metadata) cannot be applied to input url http://localhost:5566/RSFfmpegQSV?inputstream=http://127.0.0.1:7522/upnp/channelstream/transcode/2359890676884057735%7CRTL%20HD%20Austria.ts&ffmpegpref=1M720pH264 -- you are trying to apply an input option to an output file or vice versa. Move this option before the file it belongs to.
Error parsing options for input file http://localhost:5566/RSFfmpegQSV?inputstream=http://127.0.0.1:7522/upnp/channelstream/transcode/2359890676884057735%7CRTL%20HD%20Austria.ts&ffmpegpref=1M720pH264.
Error opening input files: Invalid argument
14.02.18 14:39:49.508 TFFMPEGLiveTVConverter OnTerminated 

 

[Josezkaos]

It says you're placing output options before "-i".

 

Place this:

-mpegts_service_id 11911 -mpegts_service_type 1 -metadata service_name="RTL HD Austria"

after the "-c copy" and try again.

[janee]

He is probably using the DMS instead of Recording Service Beta.

The DMS optimize the ffmpeg parameter, so Josezkaos 's "hack" won't work.

 

[Bernhard Berger]

vor 1 Minute schrieb janee:

He is probably using the DMS instead of Recording Service Beta.

The DMS optimize the ffmpeg parameter, so Josezkaos 's "hack" won't work.

 

 

correct. Did not realize that this won't work on DMS 

[kamakamadaun]

Hi.

 

I have run into a similar issue. I don't need live decoding but I am trying to run a post recording task where I transcode/remux the recorded ts file to mkv using h264_qsv (I need to transcode because chromecast 2 doesn't work properly with interlaced content). I get the same problem with hardware decoding/encoding not being available.

Could I use this for this case too? It is not obvious to me how it should be called in that case. Or is there an easier work-around?

 

In the tasks.xml file I found 

<AsUser>0</AsUser>

But I cannot find it documented anywhere so I don't know if it has anything to do with this. I tried setting it to 1 but then the task seems not to run at all.