[FFmpeg-devel] libavfilter/vsrc_testsrc : fix hang with testsrc2 and small size

[Martin Vignali]

2017-10-28 18:57 GMT+02:00 Nicolas George <george at nsup.org>:

> Le septidi 7 brumaire, an CCXXVI, Martin Vignali a écrit :
> > Don't know if there is a better way to fix it
> > but if the variable stay unsigned
> > this for loop create the problem
> > for (y = ymin; y < ymax - 15; y += 16) {
>
> Looks like a job for FFMAX or FFMIN to me.
>

Not sure i understand, can you explain a little bit more ?
In my test, using FFMAX, and FFMIN, with "negative" unsigned doesn't fix
the problem

for example with ymin = 1 and ymax = 1
FFMIN(ymin, ymax-15) return 1


And after tests, both dimensions need to be test

the previous patch, doesn't fix this :
./ffmpeg -lavfi testsrc2=s=2x600 -vframes 3 -f null -

so i propose something like this (but feel free to post a better approach)

    /* bottom right: checker with random noise */
    {
        int xmin = av_rescale(5, s->w, 8);//<== unsigned -> int
        int xmax = av_rescale(7, s->w, 8);//<== unsigned -> int
        int ymin = av_rescale(5, s->h, 8);//<== unsigned -> int
        int ymax = av_rescale(7, s->h, 8);//<== unsigned -> int
        unsigned i, r;
        int x, y;//<== unsigned -> int
        uint8_t alpha[256];

        r = s->pts;
        for (y = ymin; y < ymax - 15; y += 16) {//can hang
            for (x = xmin; x < xmax - 15; x += 16) {//can hang
                if ((x ^ y) & 16)
                    continue;
                for (i = 0; i < 256; i++) {
                    r = r * 1664525 + 1013904223;
                    alpha[i] = r >> 24;
                }
                set_color(s, &color, 0xFF00FF80);
                ff_blend_mask(&s->draw, &color, frame->data,
frame->linesize,
                                   frame->width, frame->height,
                                   alpha, 16, 16, 16, 3, 0, x, y);
            }
        }
    }

Martin