[FFmpeg-devel] [PATCH] swscale: avoid overread in planar2x
Sun Sep 12 15:43:45 CEST 2010
On Sun, Sep 12, 2010 at 10:00:07AM -0300, Ramiro Polla wrote:
> On Sun, Sep 12, 2010 at 7:44 AM, Michael Niedermayer <michaelni at gmx.at> wrote:
> > On Sun, Sep 12, 2010 at 12:11:20PM +0200, Michael Niedermayer wrote:
> >> On Sun, Sep 12, 2010 at 12:11:58AM -0300, Ramiro Polla wrote:
> >> > The MMX2/3dnow code in planar2x in rgb2rgb currently reads one byte
> >> > prior to the source buffer. This leads to a crash on Windows and Mac
> >> > OS X when the buffer is allocated on the beginning of a page. This
> >> > should also be a problem on Linux but I haven't seen many crashes
> >> > because of overreading...
> >> the code looks buggy
> >> the first pixel should be  not [-1] that can be achived by
> >> moving the reading code of it to the end of the loop and constructing
> >> the contents of the first mm4 outside the loop
> > after a second look i see that the C code overrides the first pixel and thus
> > the output should not be affected, still my suggestion though seems nicer than
> > requireing all buffers to have a byte allocated before them
> Thanks for the suggestion. Patch attached. No speed difference measured.
> rgb2rgb_template.c | 4 +++-
> 1 file changed, 3 insertions(+), 1 deletion(-)
> 9c5b0f0bc55c99bcfbda486bff43d26c34cb96f5 planar2x_avoid_overread_2.diff
> Index: rgb2rgb_template.c
> --- rgb2rgb_template.c (revision 32166)
> +++ rgb2rgb_template.c (working copy)
> @@ -1773,12 +1773,13 @@ static inline void RENAME(planar2x)(const uint8_t
> const x86_reg mmxSize= srcWidth&~15;
> __asm__ volatile(
> "mov %4, %%"REG_a" \n\t"
> + "movq (%0, %%"REG_a"), %%mm4 \n\t"
> + "psllq $8, %%mm4 \n\t"
the first value is then 0 which would lead to a darker border pixel
Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB
No snowflake in an avalanche ever feels responsible. -- Voltaire
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Size: 198 bytes
Desc: Digital signature
More information about the ffmpeg-devel