[FFmpeg-devel] [PATCH] videodsp: don't overread edges in vfix3 emu_edge.

[Ronald S. Bultje]

Hi Andreas,

On Sat, Oct 24, 2015 at 11:58 AM, Andreas Cadhalpun <
andreas.cadhalpun at googlemail.com> wrote:

> Hi Ronald,
>
> On 24.10.2015 17:55, Ronald S. Bultje wrote:
> > On Sat, Oct 24, 2015 at 11:41 AM, Andreas Cadhalpun <
> > andreas.cadhalpun at googlemail.com> wrote:
> >
> >> On 24.10.2015 14:47, Ronald S. Bultje wrote:
> >>> ---
> >>>  libavcodec/x86/videodsp.asm | 15 +++++++++++----
> >>>  1 file changed, 11 insertions(+), 4 deletions(-)
> >>>
> >>> diff --git a/libavcodec/x86/videodsp.asm b/libavcodec/x86/videodsp.asm
> >>> index 25d4364..48f5ac0 100644
> >>> --- a/libavcodec/x86/videodsp.asm
> >>> +++ b/libavcodec/x86/videodsp.asm
> >>> @@ -194,8 +194,12 @@ hvar_fn
> >>>  %elif (%2-%%off) == 2
> >>>      mov            valw, [srcq+%2-2]
> >>>  %elifidn %1, body
> >>> -    mov            vald, [srcq+%2-3]
> >>> -%else
> >>> +    mov            valb, [srcq+%2-1]
> >>> +    sal            vald, 16
> >>> +    mov            valw, [srcq+%2-3]
> >>> +%elifidn %1, bottom
> >>> +    movd mm %+ %%mmx_idx, [srcq+%2-4]
> >>> +%else ; top
> >>>      movd mm %+ %%mmx_idx, [srcq+%2-3]
> >>>  %endif
> >>>  %endif ; (%2-%%off) >= 1
> >>> @@ -251,12 +255,15 @@ hvar_fn
> >>>      mov     [dstq+%2-2], valw
> >>>  %elifidn %1, body
> >>>      mov     [dstq+%2-3], valw
> >>> -    shr            vald, 16
> >>> +    sar            vald, 16
> >>>      mov     [dstq+%2-1], valb
> >>>  %else
> >>>      movd           vald, mm %+ %%mmx_idx
> >>> +%ifidn %1, bottom
> >>> +    sar            vald, 8
> >>> +%endif
> >>>      mov     [dstq+%2-3], valw
> >>> -    shr            vald, 16
> >>> +    sar            vald, 16
> >>>      mov     [dstq+%2-1], valb
> >>>  %endif
> >>>  %endif ; (%2-%%off) >= 1
> >>>
> >>
> >> This fixes the crash and passes FATE, so looks good to me.
> >> Maybe mention the bug report in the commit message, e.g.:
> >> Bug-Debian: https://bugs.debian.org/801745
> >
> >
> > Thanks for testing, so this does actually fix the bug?
>
> Yes, at least I couldn't reproduce the crash anymore with this patch
> applied.


Great, thanks for verifying that. I'll apply with your suggested addition
to the commit message.

Ronald