[FFmpeg-devel] [PATCH] lavc: reset codec on receiving packet after EOF in compat_decode

[Hendrik Leppkes]

On Thu, Nov 9, 2017 at 1:21 PM, James Cowgill <jcowgill at debian.org> wrote:
> In commit 061a0c14bb57 ("decode: restructure the core decoding code"), the
> deprecated avcodec_decode_* APIs were reworked so that they called into the
> new avcodec_send_packet / avcodec_receive_frame API. This had the side effect
> of prohibiting sending new packets containing data after a drain
> packet, but in previous versions of FFmpeg this "worked" and some
> applications relied on it.
>
> To restore some compatibility, reset the codec if we receive a new non-drain
> packet using the old API after draining has completed. While this does
> not give the same behaviour as the old API did, in the majority of cases
> it works and it does not require changes to any other part of the decoding
> code.
>
> Fixes ticket #6775
> Signed-off-by: James Cowgill <jcowgill at debian.org>
> ---
>  libavcodec/decode.c | 5 +++++
>  1 file changed, 5 insertions(+)
>
> diff --git a/libavcodec/decode.c b/libavcodec/decode.c
> index 86fe5aef52..2f1932fa85 100644
> --- a/libavcodec/decode.c
> +++ b/libavcodec/decode.c
> @@ -726,6 +726,11 @@ static int compat_decode(AVCodecContext *avctx, AVFrame *frame,
>
>      av_assert0(avci->compat_decode_consumed == 0);
>
> +    if (avci->draining_done && pkt && pkt->size != 0) {
> +        av_log(avctx, AV_LOG_WARNING, "Got unexpected packet after EOF\n");
> +        avcodec_flush_buffers(avctx);
> +    }
> +

I don't think this is a good idea. Draining and not flushing
afterwards is a bug in the calling code, and even before recent
changes it would result in inconsistent behavior and even crashes
(with select decoders).

- Hendrik