[Ffmpeg-devel] swscale and 64 bit problem (mmx code)

Karl H. Beckers karl.h.beckers
Thu Mar 1 13:25:40 CET 2007


> Message: 4
> Date: Thu, 1 Mar 2007 13:18:12 +0000
> From: Giancarlo Formicuccia <giancarlo.formicuccia at gmail.com>
> Subject: [Ffmpeg-devel] swscale and 64 bit problem (mmx code)
> To: ffmpeg-devel at mplayerhq.hu
> Message-ID: <200703011318.12860.giancarlo.formicuccia at gmail.com>
> Content-Type: text/plain;  charset="us-ascii"
>
> Hi *,
>
> libswscale seems to be unsafe on 64 bit platforms. The problem is
> the 32-bit chrMmxFilter array of SwsContext, because it stores
> pointers to allocated memory (lumMmxFilter has probably the same
> issue).
>
> For example this testcase crashes on my 64 bit P4:
>
> #include <stdio.h>
> #include <stdlib.h>
> #include <assert.h>
>
> #include <ffmpeg/avcodec.h>
> #include <ffmpeg/avutil.h>
> #include <ffmpeg/swscale.h>
>
> #define W1 640
> #define H1 480
> #define W2 352
> #define H2 288
>
> static void wait_for_alloc64()
> {
>   while(1) {
>     void *addr = av_malloc(1024*32);
>     assert(addr!=NULL);
>     if (addr>=(void *) 0x100000000ULL) {
>       fprintf(stderr, "Heap reached %p\n", addr);
>       break;
>     }
>   }
> }
>
> static AVPicture *alloc_picture(int w, int h)
> {
>   AVPicture *ret = av_malloc(sizeof(*ret));
>   int sz = avpicture_get_size(PIX_FMT_YUV420P, w, h);
>   uint8_t *buf = av_malloc(sz);
>   avpicture_fill(ret, buf, PIX_FMT_YUV420P, w, h);
>   return ret;
> }
>
> int main()
> {
>   AVPicture *p1 = alloc_picture(W1, H1);
>   AVPicture *p2 = alloc_picture(W2, H2);
>   struct SwsContext *ss;
>
>   /* Wait for the allocator to return >32 bit addressess */
>   wait_for_alloc64();
>
>   ss = sws_getContext(W1, H1, PIX_FMT_YUV420P, W2, H2, PIX_FMT_YUV420P,
>     SWS_FAST_BILINEAR, NULL, NULL, NULL);
>
>   sws_scale(ss, p1->data, p1->linesize, 0,
>     H1, p2->data, p2->linesize);
>
>   return 0;
> }
>
> The crash occurs at
> 0x00002b00be3e2914 in swScale_MMX (c=0x1000082a0, src=0x501010,
> srcStride=0x7fffeccfb4f0, srcSliceY=0, srcSliceH=480, dst=0x501050,
> dstStride=0x7fffeccfb500)
>     at swscale_template.c:944
> 944                             YSCALEYUV2YV12X(   0,
> CHR_MMX_FILTER_OFFSET, uDest,
> chrDstW)
>
> RIP is at swscale_template.c:20:
> "movq " #x "(%%"REG_S", %%"REG_a", 2), %%mm2\n\t" /* srcData */
>
> The heap is at 00501000-100021000, but c->chrMmxFilter[0] is 0x14c70 at
> this point (the low dword of 100014c70).
>
> Fixing the code is not trivial, due to the many hardcoded values in the
> asm code...
> Ideas?
>
> Giancarlo

++

this looks much like the reason for what people reported as xvidcap bug #
1656899.
https://sourceforge.net/tracker/index.php?func=detail&aid=1656899&group_id=81535&atid=563254

That may not add much to the description of the issue other than that
people seem to be getting 5-6 crashes out of 10 attempts in real life.

Karl.





More information about the ffmpeg-devel mailing list