[FFmpeg-devel] [PATCH] https://trac.ffmpeg.org/ticket/7030 fix

Fri Jul 13 13:07:52 EEST 2018

> From: ffmpeg-devel [mailto:ffmpeg-devel-bounces at ffmpeg.org] On Behalf
> Of Mark Thompson
> Sent: Thursday, July 12, 2018 7:46 AM
> To: ffmpeg-devel at ffmpeg.org
> Subject: Re: [FFmpeg-devel] [PATCH] https://trac.ffmpeg.org/ticket/7030 fix
> 
> On 04/07/18 05:07, Li, Zhong wrote:
> >> -----Original Message-----
> >> From: ffmpeg-devel [mailto:ffmpeg-devel-bounces at ffmpeg.org] On
> Behalf
> >> Of Hendrik Leppkes
> >> Sent: Wednesday, July 4, 2018 2:15 AM
> >> To: FFmpeg development discussions and patches
> >> <ffmpeg-devel at ffmpeg.org>
> >> Subject: Re: [FFmpeg-devel] [PATCH]
> >> https://trac.ffmpeg.org/ticket/7030 fix
> >>
> >> On Tue, Jul 3, 2018 at 10:55 AM Li, Zhong <zhong.li at intel.com> wrote:
> >>>
> >>>> -----Original Message-----
> >>>> From: ffmpeg-devel [mailto:ffmpeg-devel-bounces at ffmpeg.org] On
> >>>> Behalf Of Yuri Palich
> >>>> Sent: Friday, March 2, 2018 7:55 PM
> >>>> To: ffmpeg-devel <ffmpeg-devel at ffmpeg.org>
> >>>> Subject: [FFmpeg-devel] [PATCH] https://trac.ffmpeg.org/ticket/7030
> >>>> fix
> >>>>
> >>>> From 7056d06412214bd601afb4b423d20ab6100e6ac5 Mon Sep 17
> >> 00:00:00
> >>>> 2001
> >>>> From: palich2000 <y.palich.t at gmail.com>
> >>>> Date: Fri, 2 Mar 2018 11:48:22 +0000
> >>>> Subject: [PATCH] https://trac.ffmpeg.org/ticket/7030 fix
> >>>>
> >>>> ---
> >>>>  libavcodec/qsvenc_h264.c | 15 +++++++++++++++
> >>>>  1 file changed, 15 insertions(+)
> >>>>
> >>>> diff --git a/libavcodec/qsvenc_h264.c b/libavcodec/qsvenc_h264.c
> >>>> index
> >>>> 09e4c0e..5f60675 100644
> >>>> --- a/libavcodec/qsvenc_h264.c
> >>>> +++ b/libavcodec/qsvenc_h264.c
> >>>> @@ -34,6 +34,7 @@
> >>>>  #include "qsv.h"
> >>>>  #include "qsv_internal.h"
> >>>>  #include "qsvenc.h"
> >>>> +#include "hwaccel.h"
> >>>>
> >>>>  typedef struct QSVH264EncContext {
> >>>>      AVClass *class;
> >>>> @@ -169,6 +170,16 @@ static const AVCodecDefault
> qsv_enc_defaults[]
> >> = {
> >>>>      { NULL },
> >>>>  };
> >>>>
> >>>> +#define HWACCEL_QSV(codec) \
> >>>> +    HW_CONFIG_HWACCEL(1, 1, 1, QSV,          QSV,
> >> ff_
> >>>> ## codec ##
> >>>> _qsv_hwaccel)
> >>>> +
> >>>> +const AVHWAccel ff_h264_qsv_hwaccel = {
> >>>> +    .name                 = "h264_qsv",
> >>>> +    .type                 = AVMEDIA_TYPE_VIDEO,
> >>>> +    .id                   = AV_CODEC_ID_H264,
> >>>> +    .pix_fmt              = AV_PIX_FMT_QSV
> >>>> +};
> >>>> +
> >>>>  AVCodec ff_h264_qsv_encoder = {
> >>>>      .name           = "h264_qsv",
> >>>>      .long_name      = NULL_IF_CONFIG_SMALL("H.264 / AVC /
> >> MPEG-4
> >>>> AVC /
> >>>> MPEG-4 part 10 (Intel Quick Sync Video acceleration)"), @@ -186,5
> >>>> +197,9 @@ AVCodec ff_h264_qsv_encoder = {
> >>>>      .priv_class     = &class,
> >>>>      .defaults       = qsv_enc_defaults,
> >>>>      .caps_internal  = FF_CODEC_CAP_INIT_CLEANUP,
> >>>> +    .hw_configs            = (const
> AVCodecHWConfigInternal*[]) {
> >>>> +                       HWACCEL_QSV(h264),
> >>>> +    NULL
> >>>> +    },
> >>>>      .wrapper_name   = "qsv",
> >>>>  };
> >>>> --
> >>>> 1.8.3.1
> >>>
> >>> LGTM. Verified this fix # ticket 7030.
> >>
> >> The fact that we removed the fake HWAccels for a reason still stands.
> >> Please talk to Mark Thompson before commiting anything like this.
> >>
> >> - Hendrik
> >
> > Sure. Won't push such a patch without clear commit message and also
> > need to fix other qsv encoders broken issue. (this patch is only available for
> h264.) Waiting Mark's comment/confirmation.
> 
> So I think the problem behind this is that in some configurations the encoder
> and decoder somehow can't use the same device at the same time, and it
> isn't carried between them.  I think that's something to do with using
> Redhat/Centos and therefore an ancient (pre-2014?) Linux version which
> lacks support for rendernodes, but I never managed to confirm that.  These
> cases work fine for me on both Windows and non-ancient Linux, 

Could you specify which Linux version/MSDK version/Linux kernel version you are using?
I can reproduce it with MSS2017 on Centos7.3 (Linux kernel 3.10), and also lastest open source MSDK on Ubuntu16.04 (Linux Kernel 4.12).
There are other users with MSS2017 on Centos7.4 reported this issue.

>and they will also work if the device gets passed from the decoder to encoder via
> hardware frames.

Yes, "hardware frame" path on my side is working well too. 

> 
> The hack in the patch above is adding decoder metadata to the encoder
> which allows the -init_hw_device option to end up with the device
> automatically applied to both the encoder and the decoder.  However, we
> never actually defined that metadata for encoders so this is definitely not
> appropriate.  There was some intent to define it for encoders too (indeed,
> there is some vestigial code in ffmpeg_hw.c which this is hooking to get the
> effect it wants), but it turned out that the only interesting metadata anyone
> actually wanted was to know whether an encoder can hardware or not, and
> that was added as AV_CODEC_CAP_HARDWARE / AV_CODEC_CAP_HYBRID.
> The HWConfig parts for encoders were then left undefined because noone
> actually wanted to look at that information at runtime.
> 
> If you want to fix this, I suggest one or both of:
> * Discuss what the HWConfig information for encoders should contain and
> how you would like people to use it, precisely define and document it, then
> modify ffmpeg_hw.c suitably to pass devices automatically based on it.
> * Add a new option of some kind to work like -hwaccel_device for decoders
> and -filter_hw_device for filters.
> 
> Another possibility would be to add a qsv-specific hack in ffmpeg.c to make
> this particular case work (with no change to the libraries).  Not sure exactly
> what that would entail, but maybe something like the strstr() hack that was
> present in 3.4 because we hadn't sorted out what codec metadata to
> provide at that point.  (I'd prefer to avoid this given the effort that went
> into removing all of the specific code from ffmpeg.c, but it is a possibility.)

Thanks for your valuable suggestion. I've not checked much detailed of this bug (just tried to reproduce it and verified this patch). 
Will have a further look but need to check why it works well on some specified Linux version just like yours.

> 
> Thanks,
> 
> - Mark
> 
> 
> PS: When thinking about this, note that at the AMF and NVENC encoders can
> also make use of an AVCodecContext.hw_device_ctx on input, with multiple
> types accepted in both cases.