[FFmpeg-devel] [PATCH] videodsp: don't overread edges in vfix3 emu_edge.

[Ronald S. Bultje]

Hi Andreas,

On Sat, Oct 24, 2015 at 11:41 AM, Andreas Cadhalpun <
andreas.cadhalpun at googlemail.com> wrote:

> On 24.10.2015 14:47, Ronald S. Bultje wrote:
> > ---
> >  libavcodec/x86/videodsp.asm | 15 +++++++++++----
> >  1 file changed, 11 insertions(+), 4 deletions(-)
> >
> > diff --git a/libavcodec/x86/videodsp.asm b/libavcodec/x86/videodsp.asm
> > index 25d4364..48f5ac0 100644
> > --- a/libavcodec/x86/videodsp.asm
> > +++ b/libavcodec/x86/videodsp.asm
> > @@ -194,8 +194,12 @@ hvar_fn
> >  %elif (%2-%%off) == 2
> >      mov            valw, [srcq+%2-2]
> >  %elifidn %1, body
> > -    mov            vald, [srcq+%2-3]
> > -%else
> > +    mov            valb, [srcq+%2-1]
> > +    sal            vald, 16
> > +    mov            valw, [srcq+%2-3]
> > +%elifidn %1, bottom
> > +    movd mm %+ %%mmx_idx, [srcq+%2-4]
> > +%else ; top
> >      movd mm %+ %%mmx_idx, [srcq+%2-3]
> >  %endif
> >  %endif ; (%2-%%off) >= 1
> > @@ -251,12 +255,15 @@ hvar_fn
> >      mov     [dstq+%2-2], valw
> >  %elifidn %1, body
> >      mov     [dstq+%2-3], valw
> > -    shr            vald, 16
> > +    sar            vald, 16
> >      mov     [dstq+%2-1], valb
> >  %else
> >      movd           vald, mm %+ %%mmx_idx
> > +%ifidn %1, bottom
> > +    sar            vald, 8
> > +%endif
> >      mov     [dstq+%2-3], valw
> > -    shr            vald, 16
> > +    sar            vald, 16
> >      mov     [dstq+%2-1], valb
> >  %endif
> >  %endif ; (%2-%%off) >= 1
> >
>
> This fixes the crash and passes FATE, so looks good to me.
> Maybe mention the bug report in the commit message, e.g.:
> Bug-Debian: https://bugs.debian.org/801745


Thanks for testing, so this does actually fix the bug?

Ronald