[FFmpeg-devel] [PATCH] swscale alpha channel support

[Cédric Schieli]

2009/3/5 Michael Niedermayer <michaelni at gmx.at>:
> On Thu, Mar 05, 2009 at 03:09:26PM +0100, C?dric Schieli wrote:
>> 2009/3/2 Michael Niedermayer <michaelni at gmx.at>:
>> > On Fri, Feb 27, 2009 at 11:30:25PM +0100, C?dric Schieli wrote:

[..]

>> Here is the updated patch set :
>>
>
>> #1 : sws_yuva420p_isPlanarYUV.patch
>> YUVA420P is a planar YUV format, isn't it ?
>
> yes, patch ok

applied

>>
>> #2 : sws_scale_use_4_planes.patch
>> don't ignore 4th plane
>
> ok

applied

[...]

>> #9 : swscale-example_use_alpha.patch
>> updated
>> the second hunk in this one needs some explanations :
>> without it, swscale-example will segfault
>> even without any of my patches, simply adding a uint16_t** field to
>> struct SwsContext and av_malloc'ing between 13 and 236 bytes (or 26
>> and 408 on x86_64) will make swscale-example to segfault (see
>> bug.diff)
>> after some debugging, I found that my alpPixBuf data get corrupted somehow
>> disabling MMX code with --disable-mmx prevent this from happening
>> running swscale-example under valgrind (even without bug.diff) will segfault
>> can someone can help on this ?
>
> in swscale_internal.h there are #defines that define the offsets of
> various fields in the struct so the asm can access them, you dont maybe
> mess that up by adding a field?
>
>
> and if valgrind itself segfaults, that should be reported to the valgrind
> devs ...

I finally found out what's going on here.
In initMMX2HScaler, when chrDstW is not divisible by 4, the last
filterPos element is initialized on the wrong index (not evenly
aligned)
Attached patch fixes it, and gives a nice decrease of ssdU/ssdV in
swscale-example (see attached swscale-example.log.diff.gz)


Regards,
C?dric Schieli
-------------- next part --------------
A non-text attachment was scrubbed...
Name: sws_fix_initMMX2HScaler.patch
Type: text/x-patch
Size: 520 bytes
Desc: not available
URL: <http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/attachments/20090307/ea7bdb81/attachment.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: swscale-example.log.diff.gz
Type: application/x-gzip
Size: 29024 bytes
Desc: not available
URL: <http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/attachments/20090307/ea7bdb81/attachment-0001.bin>