[FFmpeg-trac] #3226(avcodec:new): SEGFAULT in libavcodec in ff_emu_edge_vfix3_mmx.body_loop when playing video in vlc.

[FFmpeg]

#3226: SEGFAULT in libavcodec in ff_emu_edge_vfix3_mmx.body_loop when playing
video in vlc.
-----------------------------------+----------------------------------
             Reporter:  michalsrb  |                     Type:  defect
               Status:  new        |                 Priority:  normal
            Component:  avcodec    |                  Version:  2.1.1
             Keywords:             |               Blocked By:
             Blocking:             |  Reproduced by developer:  0
Analyzed by developer:  0          |
-----------------------------------+----------------------------------
 Summary: Playing a video in vlc crashes with SEGFAULT in libavcodec. Crash
 happens at random time (each time different). System is opensuse 12.3.

 Video information:
 {{{
 Input #0, avi, from 'Videos/Boy Meets World Season 1 - 7 DVDRip/Season
 4/Boy.Meets.World.S04E06.DVDRip.XviD-NODLABS.avi':
   Metadata:
     encoder         : Nandub v1.0rc2
   Duration: 00:21:57.70, start: 0.000000, bitrate: 1108 kb/s
     Stream #0:0: Video: mpeg4 (Advanced Simple Profile) (XVID /
 0x44495658), yuv420p, 512x384 [SAR 1:1 DAR 4:3], 29.97 tbr, 29.97 tbn,
 29.97 tbc
     Stream #0:1: Audio: mp3 (U[0][0][0] / 0x0055), 48000 Hz, stereo, s16p,
 135 kb/s
 }}}

 Backtrace:
 {{{
 #0  ff_emu_edge_vfix3_mmx.body_loop () at libavcodec/x86/videodsp.asm:333
 #1  0x00007fffcadc4f5c in emulated_edge_mc (h_extend_var=<optimized out>,
 hfix_tbl=0x7fffcb1db220 <hfixtbl_sse2>, v_extend_var=<optimized out>,
     vfix_tbl=0x7fffcb1db160 <vfixtbl_sse>, h=192, w=3, src_y=<optimized
 out>, src_x=<optimized out>, block_h=9, block_w=<optimized out>,
 src_stride=<optimized out>,
     src=<optimized out>, dst_stride=256, dst=<optimized out>) at
 libavcodec/x86/videodsp_init.c:175
 #2  emulated_edge_mc_sse2 (buf=0x7fffcc04b410
 "\200\200\200\202\201\201\202\202\203", buf_stride=256, src=<optimized
 out>, src_stride=<optimized out>,
     block_w=<optimized out>, block_h=9, src_x=253, src_y=184, w=256,
 h=192) at libavcodec/x86/videodsp_init.c:232
 #3  0x00007fffcab8bf03 in mpeg_motion_internal (mb_y=23, is_mpeg12=0,
 h=16, motion_y=0, motion_x=117, pix_op=0x7fffb4c16ec8,
 ref_picture=<optimized out>,
     field_select=0, bottom_field=0, field_based=0, dest_cr=0x7fffcc05a840
 "\202\202\202\202\201\201\201\200", dest_cb=0x7fffcc058840 "vvwwwxxx",
     dest_y=0x7fffcc056840
 "\026\026\026\026\026\026\025\025\031\032\033\035\037\"\" ",
 s=0x7fffb4c148c0) at libavcodec/mpegvideo_motion.c:333
 #4  mpeg_motion (s=0x7fffb4c148c0, dest_y=0x7fffcc056840
 "\026\026\026\026\026\026\025\025\031\032\033\035\037\"\" ",
 dest_cb=0x7fffcc058840 "vvwwwxxx",
     dest_cr=0x7fffcc05a840 "\202\202\202\202\201\201\201\200",
 field_select=0, ref_picture=<optimized out>, pix_op=0x7fffb4c16ec8,
 motion_x=117, motion_y=0, h=16,
     mb_y=23) at libavcodec/mpegvideo_motion.c:384
 #5  0x00007fffcab8c9e7 in MPV_motion_internal (is_mpeg12=<optimized out>,
 qpix_op=<optimized out>, pix_op=<optimized out>, ref_picture=<optimized
 out>,
     dir=<optimized out>, dest_cr=<optimized out>, dest_cb=<optimized out>,
 dest_y=<optimized out>, s=<optimized out>) at
 libavcodec/mpegvideo_motion.c:958
 #6  ff_MPV_motion (s=s at entry=0x7fffb4c148c0,
 dest_y=dest_y at entry=0x7fffcc056840
 "\026\026\026\026\026\026\025\025\031\032\033\035\037\"\" ",
     dest_cb=dest_cb at entry=0x7fffcc058840 "vvwwwxxx",
 dest_cr=dest_cr at entry=0x7fffcc05a840 "\202\202\202\202\201\201\201\200",
 dir=dir at entry=1,
     ref_picture=ref_picture at entry=0x7fffb4c152d8, pix_op=0x7fffb4c16ec8,
 qpix_op=0x7fffb4c169d8) at libavcodec/mpegvideo_motion.c:992
 #7  0x00007fffcab74973 in MPV_decode_mb_internal (is_mpeg12=0,
 lowres_flag=0, block=<optimized out>, s=0x7fffb4c148c0) at
 libavcodec/mpegvideo.c:2796
 #8  ff_MPV_decode_mb (s=s at entry=0x7fffb4c148c0, block=<optimized out>) at
 libavcodec/mpegvideo.c:2928
 #9  0x00007fffca905870 in decode_slice (s=s at entry=0x7fffb4c148c0) at
 libavcodec/h263dec.c:274
 #10 0x00007fffca906893 in ff_h263_decode_frame (avctx=0x7fffb4c18500,
 data=0x7fffb4c18020, got_frame=0x7fffd01cacdc, avpkt=<optimized out>) at
 libavcodec/h263dec.c:701
 #11 0x00007fffcac789a2 in avcodec_decode_video2 (avctx=0x7fffb4c18500,
 picture=0x7fffb4c18020, got_picture_ptr=0x7fffd01cacdc,
 avpkt=0x7fffd01cace0)
     at libavcodec/utils.c:2062
 #12 0x00007fffcb7e705e in ?? () from
 /usr/lib64/vlc/plugins/codec/libavcodec_plugin.so
 #13 0x00007ffff7141380 in ?? () from /usr/lib64/libvlccore.so.7
 #14 0x00007ffff71427a0 in ?? () from /usr/lib64/libvlccore.so.7
 #15 0x00007ffff79aae0f in start_thread () from /lib64/libpthread.so.0
 #16 0x00007ffff74da44d in clone () from /lib64/libc.so.6
 }}}

 Sample file:
 http://gdmt.cz/sample-file.avi
 (I was able to reproduce it on this sample, happened after ~5 tries,
 crashed in first few seconds.)

 I am not sure how (if possible) to reproduce it with ffmpeg command only.

--
Ticket URL: <https://trac.ffmpeg.org/ticket/3226>
FFmpeg <http://ffmpeg.org>
FFmpeg issue tracker