[FFmpeg-devel] yuv420p_to_rgb24 core dump

[Stevens, Andrew J PNNL]

I'm writing some video analysis software and I need each frame converted to RGB24 in memory.
I encountered a core dump when trying to use a .mov as input.
I tried to reproduce it from the command line, but that worked fine.

Command line attempt:
$ ffmpeg -i questionable.mov -pix_fmt rgb24 abc%08d.ppm
FFmpeg version 0.6, Copyright (c) 2000-2010 the FFmpeg developers
  built on Sep 17 2010 09:26:42 with gcc 4.1.2 20080704 (Red Hat 4.1.2-48)
...
Seems stream 0 codec frame rate differs from container frame rate: 100000.00 (100000/1) -> 15.00 (50000/3333)
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'questionable.mov':
  Metadata:
    major_brand     : qt
    minor_version   : 0
    compatible_brands: qt
  Duration: 00:00:19.33, start: 0.-733260, bitrate: 1236 kb/s
    Stream #0.0(und): Video: h264, yuv420p, 1280x800 [PAR 1:1 DAR 8:5], 1231 kb/s, 12.42 fps, 15 tbr, 50k tbn, 100k tbc
Output #0, image2, to 'abc%08d.ppm':
  Metadata:
    encoder         : Lavf52.64.2
    Stream #0.0(und): Video: ppm, rgb24, 1280x800 [PAR 1:1 DAR 8:5], q=2-31, 200 kb/s, 90k tbn, 15 tbc
...

Code:
SwsContext *img_convert_ctx =  sws_getContext(width,height, codecContext->pix_fmt, width, height, PIX_FMT_RGB24 , SWS_BICUBIC, NULL, NULL, NULL);
sws_scale(img_convert_ctx, rawFrame->data, rawFrame->linesize, 0, height, scaledFrame->data, scaledFrame->linesize);


Backtrace:
 (gdb) r
Starting program: foo
[Thread debugging using libthread_db enabled]
Probed with size=2048 and score=100
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x1180630]ISO: File Type Major Brand: qt
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x1180630]All info found
[h264 @ 0x1181a80]no picture

Program received signal SIGSEGV, Segmentation fault.
0x0000000000899cc8 in yuv420_rgb24_MMX2 (c=0x1994c80, src=0x7fffffffd5a0, srcStride=0x7fffffffd570, srcSliceY=0, srcSliceH=800,
    dst=0x7fffffffd580, dstStride=0x7fffffffd560) at libswscale/x86/yuv2rgb_template2.c:327
327             YUV2RGB_INITIAL_LOAD
(gdb) bt
#0  0x0000000000899cc8 in yuv420_rgb24_MMX2 (c=0x1994c80, src=0x7fffffffd5a0, srcStride=0x7fffffffd570, srcSliceY=0, srcSliceH=800,
    dst=0x7fffffffd580, dstStride=0x7fffffffd560) at libswscale/x86/yuv2rgb_template2.c:327
#1  0x0000000000878ae7 in sws_scale (c=0x1994c80, src=0x11820c0, srcStride=0x11820e0, srcSliceY=0, srcSliceH=800, dst=0x11882f0,
    dstStride=0x1188310) at libswscale/swscale.c:1931


Andrew Stevens