[FFmpeg-trac] #5133(avcodec:new): left shift of negative value in mc_luma_scaled()

[FFmpeg]

#5133: left shift of negative value in mc_luma_scaled()
---------------------------------+--------------------------------------
             Reporter:  tsmith   |                     Type:  defect
               Status:  new      |                 Priority:  normal
            Component:  avcodec  |                  Version:  git-master
             Keywords:           |               Blocked By:
             Blocking:           |  Reproduced by developer:  0
Analyzed by developer:  0        |
---------------------------------+--------------------------------------
 Summary of the bug:
 UBSan: libavcodec/vp9.c:2838:49: runtime error: left shift of negative
 value -12

 How to reproduce:
 {{{
 % ffmpeg -f ivf -i <test_case> -f null -
 ffmpeg version N-77718-g72673ad Copyright (c) 2000-2016 the FFmpeg
 developers
   built with Ubuntu clang version 3.7.1-svn253742-1~exp1
 (branches/release_37) (based on LLVM 3.7.1)
   configuration: --cc=clang --cxx=clang++ --disable-libxcb --disable-xlib
 --disable-logging --disable-ffprobe --disable-ffplay --disable-sdl
 --disable-ffserver --disable-doc --disable-pthreads --disable-network
 --disable-d3d11va --disable-dxva2 --disable-vaapi --disable-vda --disable-
 vdpau --disable-stripping --disable-runtime-cpudetect --disable-
 securetransport --disable-iconv
   libavutil      55. 12.100 / 55. 12.100
   libavcodec     57. 22.100 / 57. 22.100
   libavformat    57. 21.101 / 57. 21.101
   libavdevice    57.  0.100 / 57.  0.100
   libavfilter     6. 23.100 /  6. 23.100
   libswscale      4.  0.100 /  4.  0.100
   libswresample   2.  0.101 /  2.  0.101
 [NULL @ 0x619000005a80] [IMGUTILS @ 0x7f82ba2f61a0] Picture size 0x0 is
 invalid
 [NULL @ 0x619000005a80] Ignoring invalid width/height values
 [NULL @ 0x619000005a80] [IMGUTILS @ 0x7f82ba2f65a0] Picture size 0x0 is
 invalid
 [vp9 @ 0x619000005a80] Warning: not compiled with thread support, using
 thread emulation
 Truncating packet of size 536879170 to 124
 Input #0, ivf, from
 '/home/user/Desktop/ffmpeg/ivf_corpus/00ab3dc20d1e87034e06d4de4f4ba944dea4b906':
   Duration: 00:00:00.19, start: 1179551295.937500, bitrate: 8 kb/s
     Stream #0:0: Video: vp9 (Profile 1) (VP90 / 0x30395056), yuv444p(tv),
 1x1916, 16 tbr, 16 tbn, 16 tbc
 [wrapped_avframe @ 0x619000002380] Warning: not compiled with thread
 support, using thread emulation
 [vp9 @ 0x619000002d80] Warning: not compiled with thread support, using
 thread emulation
 Output #0, null, to 'pipe:':
   Metadata:
     encoder         : Lavf57.21.101
     Stream #0:0: Video: wrapped_avframe, yuv444p, 1x1916, q=2-31, 200
 kb/s, 16 fps, 16 tbn, 16 tbc
     Metadata:
       encoder         : Lavc57.22.100 wrapped_avframe
 Stream mapping:
   Stream #0:0 -> #0:0 (vp9 (native) -> wrapped_avframe (native))
 Press [q] to stop, [?] for help
 libavcodec/vp9.c:2838:49: runtime error: left shift of negative value -12
     #0 0x1dc1f12 in mc_luma_scaled
 /home/user/code/ffmpeg/libavcodec/vp9.c:2838:49
     #1 0x1dc1f12 in inter_pred_scaled_8bpp
 /home/user/code/ffmpeg/libavcodec/vp9_mc_template.c:212
     #2 0x1ca196f in inter_recon
 /home/user/code/ffmpeg/libavcodec/vp9.c:2980:13
     #3 0x1ca196f in inter_recon_8bpp
 /home/user/code/ffmpeg/libavcodec/vp9.c:3040
     #4 0x1ca196f in decode_b /home/user/code/ffmpeg/libavcodec/vp9.c:3294
     #5 0x1c8fbbd in decode_sb
 /home/user/code/ffmpeg/libavcodec/vp9.c:3386:9
     #6 0x1c903d1 in decode_sb
 /home/user/code/ffmpeg/libavcodec/vp9.c:3434:13
     #7 0x1c903d1 in decode_sb
 /home/user/code/ffmpeg/libavcodec/vp9.c:3434:13
     #8 0x1c903d1 in decode_sb
 /home/user/code/ffmpeg/libavcodec/vp9.c:3434:13
     #9 0x1c75f55 in vp9_decode_frame
 /home/user/code/ffmpeg/libavcodec/vp9.c:4164:29
     #10 0x1a3fc96 in avcodec_decode_video2
 /home/user/code/ffmpeg/libavcodec/utils.c:2107:19
     #11 0x589546 in decode_video /home/user/code/ffmpeg/ffmpeg.c:2069:11
     #12 0x589546 in process_input_packet
 /home/user/code/ffmpeg/ffmpeg.c:2318
     #13 0x5990bc in process_input /home/user/code/ffmpeg/ffmpeg.c:3980:5
     #14 0x5726c6 in transcode_step /home/user/code/ffmpeg/ffmpeg.c:4068:11
     #15 0x5726c6 in transcode /home/user/code/ffmpeg/ffmpeg.c:4122
     #16 0x56f73c in main /home/user/code/ffmpeg/ffmpeg.c:4314:9
     #17 0x7f82bd066ec4 in __libc_start_main /build/buildd/eglibc-2.19/csu
 /libc-start.c:287
     #18 0x466445 in _start
 (/home/user/Desktop/ffmpeg/ffmpeg_full+0x466445)
 }}}

--
Ticket URL: <https://trac.ffmpeg.org/ticket/5133>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker