[FFmpeg-devel] [PATCH v2 3/5] avutil/hwcontext_d3d11va: add a format check for staging texture

[Wu, Tong1]

> > -----Original Message-----
> > From: ffmpeg-devel <ffmpeg-devel-bounces at ffmpeg.org> On Behalf Of
> Tong
> > Wu
> > Sent: Friday, April 29, 2022 12:45 PM
> > To: ffmpeg-devel at ffmpeg.org
> > Cc: Tong Wu <tong1.wu at intel.com>
> > Subject: [FFmpeg-devel] [PATCH v2 3/5] avutil/hwcontext_d3d11va: add a
> > format check for staging texture
> >
> > The texDesc.Format needs to be filled in with a corresponding format.
> > I
> > add a format check to initialize the format in case sometimes the
> > ctx->internal->priv is not initialized, such as during the hwmap
> > process.
> 
> ctx->internal->priv is D3D11VAFramesContext. When it wouldn't be
> initialized, then hwmap couldn't work.
> 
> D3D11VAFramesContext.format is set during d3d11va_frames_init.
> You would need to find out whether init is not called or
> whether AVHWFramesContext.sw_format is not (yet) set during
> init.
> 

For the hwmap process, the new FramesContext is created in av_hwframe_ctx_create_derived and the init function is never called.

> 
> If that doesn't work out for some reason, I think the next best
> solution would be to add a 'format parameter' to
> d3d11va_create_staging_texture() and in d3d11va_transfer_data()
> (the only caller) do ID3D11Texture2D_GetDesc() on the frame
> texture ('texture' variable) and pass the returned format to
> d3d11va_create_staging_texture()
> 

I think this solution makes sense. I will resubmit the patch. Thanks for the review.

Regards,
Tong


> Kind regards,
> softworkz
> 
> 
> >
> > $ ffmpeg.exe -y -hwaccel qsv -init_hw_device d3d11va=d3d11 \
> > -init_hw_device qsv=qsv at d3d11 -c:v h264_qsv \
> > -i input.h264 -vf
> >
> "hwmap=derive_device=d3d11va,format=d3d11,hwdownload,format=nv12"
> \
> > -f null -
> >
> > Signed-off-by: Tong Wu <tong1.wu at intel.com>
> > ---
> >  libavutil/hwcontext_d3d11va.c | 16 ++++++++++++++++
> >  1 file changed, 16 insertions(+)
> >
> > diff --git a/libavutil/hwcontext_d3d11va.c
> > b/libavutil/hwcontext_d3d11va.c
> > index db529acbb4..0ec0e07d3a 100644
> > --- a/libavutil/hwcontext_d3d11va.c
> > +++ b/libavutil/hwcontext_d3d11va.c
> > @@ -349,6 +349,8 @@ static int
> > d3d11va_create_staging_texture(AVHWFramesContext *ctx)
> >      AVD3D11VADeviceContext *device_hwctx = ctx->device_ctx->hwctx;
> >      D3D11VAFramesContext              *s = ctx->internal->priv;
> >      HRESULT hr;
> > +    int i;
> > +
> >      D3D11_TEXTURE2D_DESC texDesc = {
> >          .Width          = ctx->width,
> >          .Height         = ctx->height,
> > @@ -360,6 +362,20 @@ static int
> > d3d11va_create_staging_texture(AVHWFramesContext *ctx)
> >          .CPUAccessFlags = D3D11_CPU_ACCESS_READ |
> > D3D11_CPU_ACCESS_WRITE,
> >      };
> >
> > +    if (!texDesc.Format) {
> > +        for (i = 0; i < FF_ARRAY_ELEMS(supported_formats); i++) {
> > +            if (ctx->sw_format == supported_formats[i].pix_fmt) {
> > +                texDesc.Format = supported_formats[i].d3d_format;
> > +                break;
> > +            }
> > +        }
> > +        if (i == FF_ARRAY_ELEMS(supported_formats)) {
> > +            av_log(ctx, AV_LOG_ERROR, "Unsupported pixel format:
> > %s\n",
> > +                av_get_pix_fmt_name(ctx->sw_format));
> > +            return AVERROR(EINVAL);
> > +        }
> > +    }
> > +
> >      hr = ID3D11Device_CreateTexture2D(device_hwctx->device, &texDesc,
> > NULL, &s->staging_texture);
> >      if (FAILED(hr)) {
> >          av_log(ctx, AV_LOG_ERROR, "Could not create the staging
> > texture (%lx)\n", (long)hr);
>