[FFmpeg-devel] [PATCH] lavc/vp9: set update_map to 0 when segmentation.enabled is 0

[Hendrik Leppkes]

On Thu, Feb 29, 2024 at 7:19 AM llyyr <llyyr at yukari.in> wrote:
>
> segmentation.update_map is never reset to 0 on a new frame, and retains
> the value from the previous frame. This bugs out a bunch of hwaccel
> drivers when segmentation.enabled is 0 but update_map isn't because
> they don't ignore values behind switches. We also do this for vp8* so
> this commit is just mirroring the vp8 logic.
>
> This fixes an issue with certain samples** that causes blocky
> artifacts with vaapi and d3d11va (as far as known hwaccel drivers go).
> Mesa worked around*** this by ignoring this field if
> segmentation.enabled is 0, but d3d11va still doesn't work.
>
> * https://git.ffmpeg.org/gitweb/ffmpeg.git/blob/HEAD:/libavcodec/vp8.c#l811
> ** https://github.com/mpv-player/mpv/issues/13533
> *** https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27816
>
> Signed-off-by: llyyr <llyyr at yukari.in>
> ---
>  libavcodec/vp9.c | 2 ++
>  1 file changed, 2 insertions(+)
>
> diff --git a/libavcodec/vp9.c b/libavcodec/vp9.c
> index 855936cdc1c7..4a628625131e 100644
> --- a/libavcodec/vp9.c
> +++ b/libavcodec/vp9.c
> @@ -717,6 +717,8 @@ static int decode_frame_header(AVCodecContext *avctx,
>                  s->s.h.segmentation.feat[i].skip_enabled = get_bits1(&s->gb);
>              }
>          }
> +    } else {
> +        s->s.h.segmentation.update_map = 0;
>      }
>
>      // set qmul[] based on Y/UV, AC/DC and segmentation Q idx deltas
>
> base-commit: d263fce2b209e86a5a1e8f1b6aa33430ecc2c187
> --

Change LGTM.
I was debugging the same issue today, and found the same problem with
some hwaccels not properly ignoring update_map when segmentation is
disabled.

Will apply soon if there are no further comments.

- Hendrik