[FFmpeg-devel] [PATCH] Odd, random-appearing crashes

[Ray Simard]

While working on the MMX/emms_c() issue above, I ran into more crashes
in my test versions, though with much different circumstances, and
posted about it above (by accident, anonymously).  That turns out to be
the use of an uninitialized variable.

In vf_dshake.c, end_frame:

...
    Transform t;
...
    Transform orig;
...
    if (deshake->cx < 0 || deshake->cy < 0 || deshake->cw < 0       \
	 || deshake->ch < 0) {                                      \
        // Find the most likely global motion for the current frame \
        find_motion(deshake, (deshake->ref == NULL) ? in->data[0]   \
	: deshake->ref->data[0], in->data[0], link->w,ink->h,       \
        in->linesize[0], &t);
    } else {
   ...
        find_motion(deshake, src1, src2, deshake->cw, deshake->ch, \
        in->linesize[0], &t);
    }

    // Copy transform so we can output it later to compare to the
smoothed value
    orig.vector.x = t.vector.x;
    orig.vector.y = t.vector.y;
    orig.angle = t.angle;
    orig.zoom = t.zoom;

The last line uses the value of t.zoom in Transform struct t, but the
calls to find_motion that pass the pointer to t don't initialize it. The
result is unpredictable behavior; in my case, things worked right if I
compiled without optimization, but crashed if I compiled with it, and
only in Cygwin.  The attached patch clears both Transform
structs (t and orig) to zero.

There was also a variable in that turns out to be vestigial:

    for (x = 0; x < pos; x++) {
        totalangles += angles[x];
    }

    t->angle = totalangles / (pos - 1);

    t->angle = clean_mean(angles, pos);
    if (t->angle < 0.001)
        t->angle = 0;

It is assigned to t->angle, but t->angle is immediately assigned to the
return from clean_mean, and totalangles does not show up anywhere else.


-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: init-transforms-t-and-orig.patch
URL: <http://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20120111/08916072/attachment.ksh>