[FFmpeg-devel] [PATCH] videodsp: don't overread edges in vfix3 emu_edge.

[Andreas Cadhalpun]

On 24.10.2015 14:47, Ronald S. Bultje wrote:
> ---
>  libavcodec/x86/videodsp.asm | 15 +++++++++++----
>  1 file changed, 11 insertions(+), 4 deletions(-)
> 
> diff --git a/libavcodec/x86/videodsp.asm b/libavcodec/x86/videodsp.asm
> index 25d4364..48f5ac0 100644
> --- a/libavcodec/x86/videodsp.asm
> +++ b/libavcodec/x86/videodsp.asm
> @@ -194,8 +194,12 @@ hvar_fn
>  %elif (%2-%%off) == 2
>      mov            valw, [srcq+%2-2]
>  %elifidn %1, body
> -    mov            vald, [srcq+%2-3]
> -%else
> +    mov            valb, [srcq+%2-1]
> +    sal            vald, 16
> +    mov            valw, [srcq+%2-3]
> +%elifidn %1, bottom
> +    movd mm %+ %%mmx_idx, [srcq+%2-4]
> +%else ; top
>      movd mm %+ %%mmx_idx, [srcq+%2-3]
>  %endif
>  %endif ; (%2-%%off) >= 1
> @@ -251,12 +255,15 @@ hvar_fn
>      mov     [dstq+%2-2], valw
>  %elifidn %1, body
>      mov     [dstq+%2-3], valw
> -    shr            vald, 16
> +    sar            vald, 16
>      mov     [dstq+%2-1], valb
>  %else
>      movd           vald, mm %+ %%mmx_idx
> +%ifidn %1, bottom
> +    sar            vald, 8
> +%endif
>      mov     [dstq+%2-3], valw
> -    shr            vald, 16
> +    sar            vald, 16
>      mov     [dstq+%2-1], valb
>  %endif
>  %endif ; (%2-%%off) >= 1
> 

This fixes the crash and passes FATE, so looks good to me.
Maybe mention the bug report in the commit message, e.g.:
Bug-Debian: https://bugs.debian.org/801745

Best regards,
Andreas