[FFmpeg-devel] [PATCH] Check codec_whitelist before reinitializing AVCtx.priv_data.

[Dale Curtis]

On Sat, Aug 17, 2024 at 1:42 AM Anton Khirnov <anton at khirnov.net> wrote:

> I don't follow, why would any code outside of libavcodec care about
> anything in private data?
>

Sorry I was imprecise. The issue is that ff_codec_close() is called during
"free_and_end", which releases a bunch of fields on the AVCodecContext
which may already have been populated -- in Chromium's case, the problem is
freeing the side data values which contain rotation metadata:
https://github.com/FFmpeg/FFmpeg/blob/master/libavcodec/avcodec.c#L464

Checking the allow list before allocating anying allows us to skip the
free_and_end path.

- dale