[Ffmpeg-devel] clever 8-bit MMX loop filter ABS test
Tue May 3 09:23:33 CEST 2005
I saw this piece of code in the list last thursday. While it is a rather
clever way to do the alpha/beta test, it won't produce results that are
identical to the 16-bit code.
> Here's an example for the test ABS(P0-Q0)<Alpha, using 8b only:
> input: mm7 = Alpha value, 8bits, replicated 8 times
> movd mm0, [Q0] ; four pixels 'Q0' in lower 32bits
> punpckldq mm0, [P0] ; four pixels 'P0' in higher 32bits
> pshufw mm1, mm0, 01001110b ; P0 | Q0 (Swap P0 and Q0)
> paddusb mm0, mm7 ; Q0+Alpha | P0+Alpha
> psubusb mm0, mm1 ; Q0+Alpha-P0 | P0+Alpha-Q0
> At this point: mm0 contains zeros in the lower 32bits if P0>=Q0+Alpha,
> and zeros in the higher 32bits if Q0>=P0+Alpha.
The statement directly above is not always true, because of the 8-bit
saturation. Consider the case when P0 and Q0 are both 255 and Alpha > 0.
In this case, all of the bits of mm0 will be 0 even though
"ABS(P0-Q0)<Alpha" is true. This will lead to a false negative and the
decoder will fail to filter an edge that it should have filtered.
> Note: you can repeat/pair the above 3 instructions for the other tests
> (ABS(P1-P0)<Beta, etc...), and accumulate the results in mm0 using
> a 'por' instruction...
> In the end, when one wants the final result:
> pminub mm0, [One] ; mask is now made of '0' or '1'.
> ; [One] is 1, replicated 8 times
> pshufw mm1, mm0, 01001110b ; Swap the hi/lo 32 bits
> pxor mm0, [One] ; flip the bits
> pand mm1, mm0 ; => the higher 4 bytes of mm1 tell
> ; whether the pixels should be filtered or
> ; not.
Also the "pxor mm0, [One]" instruction should not be there. It
effectively neutralizes the absolute-value-ness of doing both the
Q0-P0 and P0-Q0 cases at once.
More information about the ffmpeg-devel