[FFmpeg-devel] [PATCH] avcodec/libaomdec: decode extradata in aom_init

[James Almer]

On 3/8/2024 6:49 AM, Steven Liu wrote:
> before patch:
> test failed when using libaomdec.
> TEST    enhanced-flv-av1
> --- src/tests/ref/fate/enhanced-flv-av1	2024-03-08 11:27:13.577935587 +0800
> +++ tests/data/fate/enhanced-flv-av1	2024-03-08 17:38:25.810152322 +0800
> @@ -3,7 +3,7 @@
>   #media_type 0: video
>   #codec_id 0: av1
>   #dimensions 0: 320x176
> -#sar 0: 0/1
> +#sar 0: 1/1
>   0,          0,          0,       41,     4718, 0xc4d912ec
>   0,         42,         42,       41,     5109, 0xc065dd7d, F=0x0
>   0,         83,         83,       41,      192, 0x160c5335, F=0x0
> 
> after patch:
> test passed
> 
> Signed-off-by: Steven Liu <lq at chinaffmpeg.org>
> ---
>   libavcodec/libaomdec.c | 12 ++++++++++++
>   1 file changed, 12 insertions(+)
> 
> diff --git a/libavcodec/libaomdec.c b/libavcodec/libaomdec.c
> index 69eec8b089..2df2fa6fd7 100644
> --- a/libavcodec/libaomdec.c
> +++ b/libavcodec/libaomdec.c
> @@ -58,6 +58,18 @@ static av_cold int aom_init(AVCodecContext *avctx,
>           return AVERROR(EINVAL);
>       }
>   
> +    if (aom_codec_decode(&ctx->decoder, avctx->extradata, avctx->extradata_size,
> +                         NULL) != AOM_CODEC_OK) {

Extradata may have the ISOBMFF prefix bytes, so you need to skip them 
unless aom_codec_decode() can handle them, which i assume it doesn't.
And of course, if there's nothing after the four prefix bytes, don't 
call anything.

> +        const char *error  = aom_codec_error(&ctx->decoder);
> +        const char *detail = aom_codec_error_detail(&ctx->decoder);
> +
> +        av_log(avctx, AV_LOG_ERROR, "Failed to decode extradata: %s\n", error);
> +        if (detail)
> +            av_log(avctx, AV_LOG_ERROR, "  Additional information: %s\n", detail);
> +
> +        return AVERROR(EINVAL);
> +    }
> +
>       return 0;
>   }
>