[FFmpeg-devel] [PATCH] libavcodec/qsv.c: Re-design session control and internal allocation

[Hendrik Leppkes]

On Wed, Oct 14, 2015 at 11:47 AM, Hendrik Leppkes <h.leppkes at gmail.com> wrote:
> On Wed, Oct 7, 2015 at 7:07 PM, Ivan Uskov <ivan.uskov at nablet.com> wrote:
>> Hello wm4,
>>
>> Wednesday, October 7, 2015, 7:40:45 PM, you wrote:
>>
>> w> There's no automagic way to get this done.
>>
>> w> Hardware accelerators like vaapi and vdpau need the same thing. These
>> w> have special APIs to set an API context on the decoder. Some APIs use
>> w> hwaccel_context, vdpau uses av_vdpau_bind_context().
>>
>> w> The hwaccel_context pointer is untyped (just a void*), and what it means
>> w> is implicit to the hwaccel or the decoder that is used. It could point
>> w> to some sort of qsv state, which will have to be explicitly allocated
>> w> and set by the API user (which might be ffmpeg.c).
>> So  if  I will implement something like ffmpeg_qsv.c (using ffmpeg_vdpau.c as
>> example)   and   extend  the  hwaccels[]  into  ffmpeg_opt.c  by corresponded
>> qsv entries it  would  be the acceptable solution, correct?
>>
>> w> For filters there is no such thing yet. New API would have to be
>> w> created. For filters in particular, we will have to make sure that the
>> w> API isn't overly qsv-specific, and that it is extendable to other APIs
>> w> (like for example vaapi or vdpau).
>> Ok,   if   VPP  could be the  issue  I  would  like  to  get  working  direct
>> link qsv_decoder-qsv_encoder first.
>>
>
> Libav has a patch that does exactly this, allow direct QSV->QSV
> transcoding with help of some utility code in ffmpeg.c/avconv.c
> You might want to look at that instead of re-inventing it. That'll
> help make everyones live easier, as I'll just merge it when the time
> comes, and the codebases don't diverge too drastically.
>

This functionality has been merged now. It works for some samples.
You can try to use it with a command line like this:

ffmpeg -hwaccel qsv -c:v h264_qsv -i h264.ts -c:v h264_qsv output.mkv

This will transcode using a QSV->QSV pipeline, no copying to system
memory, and about 2.5x faster on my IVB laptop.

However, its broken on a lot of more complex H264 files, you'll get
errors like get_buffer() failed - this is because our qsvdec behaves
rather strangely, and instead of buffering input data when needed, it
buffers output surfaces, which is not ideal, since it bloats up the
memory usage an the number of surfaces required explodes into
infinity.
I know how to fix it - just restore the decoder to the same buffering
model as it originally used, buffer input data instead of output
surfaces. Unless someone else wants to fix it, I'll simply do it in
the next week or so.

- Hendrik