[FFmpeg-devel] [PATCH v2 1/3] avcodec/x86/vvc/vvc_alf: fix integer overflow

[Nuo Mi]

On Fri, May 31, 2024 at 4:49 AM Ronald S. Bultje <rsbultje at gmail.com> wrote:

> Hi Andreas,
>
> On Thu, May 30, 2024 at 2:33 PM Andreas Rheinhardt <
> andreas.rheinhardt at outlook.com> wrote:
>
> > toqsxw at outlook.com:
> > > From: Wu Jianhua <toqsxw at outlook.com>
> > >
> > > Some tests fails with certain seeds
> > >
> > > tests/checkasm/checkasm 2325607578 --test=vvc_alf
> > > checkasm: using random seed 2325607578
> > > AVX2:
> > >     vvc_alf_filter_luma_120x20_12_avx2 (vvc_alf.c:104)
> > >     vvc_alf_filter_luma_120x24_12_avx2 (vvc_alf.c:104)
> > >     vvc_alf_filter_luma_120x28_12_avx2 (vvc_alf.c:104)
> > >     vvc_alf_filter_luma_120x32_12_avx2 (vvc_alf.c:104)
> > >     vvc_alf_filter_luma_120x36_12_avx2 (vvc_alf.c:104)
> > >     vvc_alf_filter_luma_120x40_12_avx2 (vvc_alf.c:104)
> > >     vvc_alf_filter_luma_120x44_12_avx2 (vvc_alf.c:104)
> > >     vvc_alf_filter_luma_120x48_12_avx2 (vvc_alf.c:104)
> > >     vvc_alf_filter_luma_120x52_12_avx2 (vvc_alf.c:104)
> > >     vvc_alf_filter_luma_120x56_12_avx2 (vvc_alf.c:104)
> > >     vvc_alf_filter_luma_120x60_12_avx2 (vvc_alf.c:104)
> > >     vvc_alf_filter_luma_120x64_12_avx2 (vvc_alf.c:104)
> > >     vvc_alf_filter_luma_120x68_12_avx2 (vvc_alf.c:104)
> > >     vvc_alf_filter_luma_120x72_12_avx2 (vvc_alf.c:104)
> > >     vvc_alf_filter_luma_120x76_12_avx2 (vvc_alf.c:104)
> > >     vvc_alf_filter_luma_120x80_12_avx2 (vvc_alf.c:104)
> > >     vvc_alf_filter_luma_120x84_12_avx2 (vvc_alf.c:104)
> > >     vvc_alf_filter_luma_120x88_12_avx2 (vvc_alf.c:104)
> > >     vvc_alf_filter_luma_120x92_12_avx2 (vvc_alf.c:104)
> > >     vvc_alf_filter_luma_120x96_12_avx2 (vvc_alf.c:104)
> > >     vvc_alf_filter_luma_120x100_12_avx2 (vvc_alf.c:104)
> > >     vvc_alf_filter_luma_120x104_12_avx2 (vvc_alf.c:104)
> > >     vvc_alf_filter_luma_120x108_12_avx2 (vvc_alf.c:104)
> > >     vvc_alf_filter_luma_120x112_12_avx2 (vvc_alf.c:104)
> > >     vvc_alf_filter_luma_120x116_12_avx2 (vvc_alf.c:104)
> > >     vvc_alf_filter_luma_120x120_12_avx2 (vvc_alf.c:104)
> > >     vvc_alf_filter_luma_120x124_12_avx2 (vvc_alf.c:104)
> > >     vvc_alf_filter_luma_120x128_12_avx2 (vvc_alf.c:104)
> > >   - vvc_alf.alf_filter   [FAILED]
> > >   - vvc_alf.alf_classify [OK]
> > > checkasm: 28 of 9216 tests have failed
> > >
> > > Reported-by: James Almer <jamrial at gmail.com>
> > > Signed-off-by: Wu Jianhua <toqsxw at outlook.com>
> > > ---
> > >  libavcodec/x86/vvc/vvc_alf.asm | 3 ++-
> > >  1 file changed, 2 insertions(+), 1 deletion(-)
> > >
> > > diff --git a/libavcodec/x86/vvc/vvc_alf.asm
> > b/libavcodec/x86/vvc/vvc_alf.asm
> > > index 71e821c27b..f7b3e2a6cc 100644
> > > --- a/libavcodec/x86/vvc/vvc_alf.asm
> > > +++ b/libavcodec/x86/vvc/vvc_alf.asm
> > > @@ -356,7 +356,8 @@ SECTION .text
> > >
> > >      FILTER_VB         xq
> > >
> > > -    paddw             m0, m2
> > > +    ; sum += curr
> > > +    paddsw             m0, m2
> > >
> > >      ; clip to pixel
> > >      CLIPW             m0, m14, m15
> >
> > And can I get an answer to the question of whether the issue is present
> > when used by the actual decoder and not only the checkasm test?
> >
>
> From my reading of the source code, this could happen in a crafted (e.g.
> fuzzed) stream.
>
Yes, the test case is valid. So even if the normal clips don't have this
issue, some clips might.

>
> Ronald
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".
>