[FFmpeg-devel] [PATCH] lavc/vvc: Fix assertion bound on qPy_{a, b}

[Nuo Mi]

On Sun, Aug 25, 2024 at 10:37 PM Nicolas George <george at nsup.org> wrote:

> Nuo Mi (12024-08-25):
> > Yes, I mean the v1. sorry for misleading you.
>
> Ah, ok. Then… maybe.
>
> Maybe v1 is the proper fix. Maybe the proper fix is to find where qPy_a
> or qPy_b are set to 63 if they can only be <63 and fix that part of the
> code. Only somebody who knows the codec can say which is the proper fix,
> but it is highly unlikely any other change is.
>
Hi Nicolas,
>From the spec
QpY = ( ( qPY_PRED + CuQpDeltaVal + 64 + 2 * QpBdOffset ) % ( 64 +
QpBdOffset ) ) - QpBdOffset
So qPY_A and qPY_B <= 63. Not < 63.
Frank's patch will fix the assert.

qPY_A and qPY_B calculations have no obvious issues.
We read  qPY_A and qPY_B  from a table. We add an assertion here to ensure
the table hasn't been polluted.

Thank you


> Regards,
>
> --
>   Nicolas George
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".
>