[FFmpeg-devel] [PATCH] avcodec/hevc_refs: don't use the frame's AVBufferRef sizes when initializing planes

[Anton Khirnov]

Quoting James Almer (2022-03-06 16:39:39)
> Use the actual plane sizes instead.
> 
> Signed-off-by: James Almer <jamrial at gmail.com>
> ---
>  libavcodec/hevc_refs.c | 14 +++++++++++---
>  1 file changed, 11 insertions(+), 3 deletions(-)
> 
> diff --git a/libavcodec/hevc_refs.c b/libavcodec/hevc_refs.c
> index 35b8e5e696..b0035356ac 100644
> --- a/libavcodec/hevc_refs.c
> +++ b/libavcodec/hevc_refs.c
> @@ -22,6 +22,7 @@
>   */
>  
>  #include "libavutil/avassert.h"
> +#include "libavutil/imgutils.h"
>  
>  #include "thread.h"
>  #include "hevc.h"
> @@ -401,9 +402,16 @@ static HEVCFrame *generate_missing_ref(HEVCContext *s, int poc)
>  
>      if (!s->avctx->hwaccel) {
>          if (!s->ps.sps->pixel_shift) {
> -            for (i = 0; frame->frame->buf[i]; i++)
> -                memset(frame->frame->buf[i]->data, 1 << (s->ps.sps->bit_depth - 1),
> -                       frame->frame->buf[i]->size);
> +            size_t sizes[4];
> +            ptrdiff_t linesizes[4];
> +            for (i = 0; i < 4; i++)
> +                linesizes[i] = frame->frame->linesize[i];
> +            if (av_image_fill_plane_sizes(sizes, frame->frame->format, frame->frame->height, linesizes) < 0) {
> +                ff_hevc_unref_frame(s, frame, ~0);
> +                return NULL;
> +            }
> +            for (i = 0; frame->frame->data[i]; i++)
> +                memset(frame->frame->data[i], 1 << (s->ps.sps->bit_depth - 1), sizes[i]);

wouldn't it be simpler and shorter to use linesize[i] * (height >> sps->vshift)?

-- 
Anton Khirnov