[FFmpeg-devel] [PATCH] avutil/hwcontext_dxva2: No longer improperly freeing IDirect3DSurface9 objects

[Aaron Levinson]

Please disregard--I will submit a new patch with a better commit message.

Aaron

On 5/16/2017 4:09 AM, Aaron Levinson wrote:
> Purpose: No longer improperly freeing IDirect3DSurface9 objects in
> hwcontext_dxva2.c.  Added dxva2_pool_release_dummy() and using it in
> call to av_buffer_create() in dxva2_pool_alloc().  Prior to this
> change, av_buffer_create() was called with NULL for the third
> argument, which indicates that av_buffer_default_free() should be used
> to free the buffer's data.  Eventually, it gets to buffer_pool_free()
> and calls buf->free() on a surface object (which is
> av_buffer_default_free()).  This can result in a crash when the debug
> version of the C-runtime is used on Windows.  While it doesn't appear
> to result in a crash when the release version of the C-runtime is used
> on Windows, it likely results in memory corruption, since av_free() is
> being called on memory that was allocated using
> IDirectXVideoAccelerationService::CreateSurface().
>
> Signed-off-by: Aaron Levinson <alevinsn at aracnet.com>
> ---
>  libavutil/hwcontext_dxva2.c | 9 ++++++++-
>  1 file changed, 8 insertions(+), 1 deletion(-)
>
> diff --git a/libavutil/hwcontext_dxva2.c b/libavutil/hwcontext_dxva2.c
> index 4ed0d56..6c41788 100644
> --- a/libavutil/hwcontext_dxva2.c
> +++ b/libavutil/hwcontext_dxva2.c
> @@ -121,6 +121,13 @@ static void dxva2_frames_uninit(AVHWFramesContext *ctx)
>      }
>  }
>
> +static void dxva2_pool_release_dummy(void *opaque, uint8_t *data)
> +{
> +    // important not to free anything here--data is a surface object
> +    // associated with the call to CreateSurface(), and these surfaces are
> +    // released in dxva2_frames_uninit()
> +}
> +
>  static AVBufferRef *dxva2_pool_alloc(void *opaque, int size)
>  {
>      AVHWFramesContext      *ctx = (AVHWFramesContext*)opaque;
> @@ -130,7 +137,7 @@ static AVBufferRef *dxva2_pool_alloc(void *opaque, int size)
>      if (s->nb_surfaces_used < hwctx->nb_surfaces) {
>          s->nb_surfaces_used++;
>          return av_buffer_create((uint8_t*)s->surfaces_internal[s->nb_surfaces_used - 1],
> -                                sizeof(*hwctx->surfaces), NULL, 0, 0);
> +                                sizeof(*hwctx->surfaces), dxva2_pool_release_dummy, 0, 0);
>      }
>
>      return NULL;
>