[FFmpeg-devel] [PATCH v2] avcodec/h264, videotoolbox: fix crash after VT decoder fails

[Ronald S. Bultje]

Hi,

On Tue, Feb 21, 2017 at 1:48 PM, Aman Gupta <ffmpeg at tmm1.net> wrote:

> diff --git a/libavcodec/h264dec.c b/libavcodec/h264dec.c
> index 41c0964392..a0ae632fed 100644
> --- a/libavcodec/h264dec.c
> +++ b/libavcodec/h264dec.c
> @@ -850,7 +850,12 @@ static int output_frame(H264Context *h, AVFrame *dst,
> H264Picture *srcp)
>      AVFrame *src = srcp->f;
>      const AVPixFmtDescriptor *desc = av_pix_fmt_desc_get(src->format);
>      int i;
> -    int ret = av_frame_ref(dst, src);
> +    int ret;
> +
> +    if (src->format == AV_PIX_FMT_VIDEOTOOLBOX && src->buf[0]->size == 1)
> +        return AVERROR_INVALIDDATA;
> +
> +    ret = av_frame_ref(dst, src);
>      if (ret < 0)
>          return ret;


This is a total hack :) Is there a way to hide this into VT-specific code
outside h264*.[ch]?

Ronald