[FFmpeg-devel] [PATCH] avcodec/dxva2: d3d11va: don't try to get surface description for nullptr

[Anton Fedchin]

>2019-01-05 11:44 GMT+01:00, Anton Fedchin <afedchin at weezlabs.com>:
>> From: Anton Fedchin <afedchin at ruswizards.com>
>>
>> after 153b36f there is a possibility to crash when trying to get index of
>> a surface which points to nirvana. it may occurs when a stream starts with
>> non i-frame.
>> ---
>>  libavcodec/dxva2.c | 10 ++++++----
>>  1 file changed, 6 insertions(+), 4 deletions(-)
>>
>> diff --git a/libavcodec/dxva2.c b/libavcodec/dxva2.c
>> index 32416112bf..dfae500444 100644
>> --- a/libavcodec/dxva2.c
>> +++ b/libavcodec/dxva2.c
>> @@ -771,16 +771,18 @@ unsigned ff_dxva2_get_surface_index(const
>> AVCodecContext *avctx,
>>  #if CONFIG_D3D11VA
>>      if (avctx->pix_fmt == AV_PIX_FMT_D3D11)
>>          return (intptr_t)frame->data[1];
>> -    if (avctx->pix_fmt == AV_PIX_FMT_D3D11VA_VLD) {
>> +    if (avctx->pix_fmt == AV_PIX_FMT_D3D11VA_VLD && surface) {
>>          D3D11_VIDEO_DECODER_OUTPUT_VIEW_DESC viewDesc;
>>
>> ID3D11VideoDecoderOutputView_GetDesc((ID3D11VideoDecoderOutputView*)
>> surface, &viewDesc);
>>          return viewDesc.Texture2D.ArraySlice;
>>      }
>>  #endif
>
>>  #if CONFIG_DXVA2
>> -    for (i = 0; i < DXVA_CONTEXT_COUNT(avctx, ctx); i++) {
>> -        if (avctx->pix_fmt == AV_PIX_FMT_DXVA2_VLD && ctx->dxva2.surface[i]
>> == surface)
>> -            return i;
>> +    if (avctx->pix_fmt == AV_PIX_FMT_DXVA2_VLD) {
>> +        for (i = 0; i < DXVA_CONTEXT_COUNT(avctx, ctx); i++) {
>> +            if (ctx->dxva2.surface[i] == surface)
>> +                return i;
>> +        }
>
>How is this change related?
>
>Carl Eugen

Hi Carl,

If previous condition is failed (i.e. pix_fmt is AV_PIX_FMT_D3D11VA_VLD  and surface is null) it will iterate through all surfaces again with no sense.
Yes, this is not related directly to the fix, but the fix changes method's behavior if surface is null and I found it necessary to change this block also