[FFmpeg-devel] [PATCH] swscale: Check whether slice height is multiple of chroma subsample.

[Michael Niedermayer]

On Tue, Feb 23, 2010 at 12:20:37AM -0300, Ramiro Polla wrote:
> Hi,
> 
> Check whether slice height is multiple of chroma subsample.
> 
> Currently if we take a 410p image and rescale with sliceH = 1, the
> output is black&white. I did this using the test tool at [1] (renamed
> to swscale-test2) and the command:
> ./swscale-test2 -i input.bmp -sws_flags bicubic -pix_fmt yuv410p -s
> 320x320 -scale -sliceH 1 -pix_fmt bgr24 -scale -o output.bmp
> 
> I found it weird that this code in utils.c expects the slices to start
> on subsample boundaries (by shifting nextSlice back and forth), but
> sws_scale() doesn't:
>     // calculate buffer sizes so that they won't run out while
> handling these damn slices
>     c->vLumBufSize= c->vLumFilterSize;
>     c->vChrBufSize= c->vChrFilterSize;
>     for (i=0; i<dstH; i++) {
>         int chrI= i*c->chrDstH / dstH;
>         int nextSlice= FFMAX(c->vLumFilterPos[i   ] + c->vLumFilterSize - 1,
>                            ((c->vChrFilterPos[chrI] +
> c->vChrFilterSize - 1)<<c->chrSrcVSubSample));
> 
>         nextSlice>>= c->chrSrcVSubSample;
>         nextSlice<<= c->chrSrcVSubSample;
>         if (c->vLumFilterPos[i   ] + c->vLumBufSize < nextSlice)
>             c->vLumBufSize= nextSlice - c->vLumFilterPos[i];
>         if (c->vChrFilterPos[chrI] + c->vChrBufSize <
> (nextSlice>>c->chrSrcVSubSample))
>             c->vChrBufSize= (nextSlice>>c->chrSrcVSubSample) -
> c->vChrFilterPos[chrI];
>     }
> 
> Ramiro Polla
> [1] http://article.gmane.org/gmane.comp.video.ffmpeg.devel/102702

>  swscale.c |    5 +++++
>  1 file changed, 5 insertions(+)
> cd845850f322996173e636f2655b7b4d5880feb4  0004-Check-whether-slice-height-is-multiple-of-chroma-sub.patch
> From 7b2354c9cf66eaf8d51b527239bc1403354b999c Mon Sep 17 00:00:00 2001
> From: Ramiro Polla <ramiro.polla at gmail.com>
> Date: Tue, 23 Feb 2010 00:12:51 -0300
> Subject: [PATCH] Check whether slice height is multiple of chroma subsample.

problem is that swscale can return such chroma != luma slices and
chaining slices scalers should work

[...]

-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

He who knows, does not speak. He who speaks, does not know. -- Lao Tsu
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
URL: <http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/attachments/20100223/11bbfff3/attachment.pgp>