[FFmpeg-cvslog] avcodec/motion_est_template: Fix undefined shifts in CHECK_MV_DIR()

Michael Niedermayer git at videolan.org
Wed Jul 1 14:38:15 CEST 2015


ffmpeg | branch: master | Michael Niedermayer <michaelni at gmx.at> | Wed Jul  1 12:16:37 2015 +0200| [56fd4705c0496bc108ac84c03465d33351d366a5] | committer: Michael Niedermayer

avcodec/motion_est_template: Fix undefined shifts in CHECK_MV_DIR()

Fixes: signal_sigsegv_2e64ee0_2762_cov_4170502227_missing_frames.avi

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni at gmx.at>

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=56fd4705c0496bc108ac84c03465d33351d366a5
---

 libavcodec/motion_est_template.c |    6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/libavcodec/motion_est_template.c b/libavcodec/motion_est_template.c
index ae2cbde..cb87801 100644
--- a/libavcodec/motion_est_template.c
+++ b/libavcodec/motion_est_template.c
@@ -384,13 +384,13 @@ static int qpel_motion_search(MpegEncContext * s,
 
 #define CHECK_MV_DIR(x,y,new_dir)\
 {\
-    const unsigned key = ((y)<<ME_MAP_MV_BITS) + (x) + map_generation;\
-    const int index= (((y)<<ME_MAP_SHIFT) + (x))&(ME_MAP_SIZE-1);\
+    const unsigned key = ((unsigned)(y)<<ME_MAP_MV_BITS) + (x) + map_generation;\
+    const int index= (((unsigned)(y)<<ME_MAP_SHIFT) + (x))&(ME_MAP_SIZE-1);\
     if(map[index]!=key){\
         d= cmp(s, x, y, 0, 0, size, h, ref_index, src_index, cmpf, chroma_cmpf, flags);\
         map[index]= key;\
         score_map[index]= d;\
-        d += (mv_penalty[((x)<<shift)-pred_x] + mv_penalty[((y)<<shift)-pred_y])*penalty_factor;\
+        d += (mv_penalty[(int)((unsigned)(x)<<shift)-pred_x] + mv_penalty[(int)((unsigned)(y)<<shift)-pred_y])*penalty_factor;\
         if(d<dmin){\
             best[0]=x;\
             best[1]=y;\



More information about the ffmpeg-cvslog mailing list