[FFmpeg-trac] #11686(swscale:new): [Security] signed integer overflow on libswscale/output.c

[FFmpeg]

#11686: [Security] signed integer overflow on libswscale/output.c
-------------------------------------+-------------------------------------
             Reporter:  flyfish101   |                     Type:  defect
               Status:  new          |                 Priority:  critical
            Component:  swscale      |                  Version:  git-
             Keywords:  swscale ,    |  master
  overflow                           |               Blocked By:
             Blocking:               |  Reproduced by developer:  0
Analyzed by developer:  0            |
-------------------------------------+-------------------------------------
 Summary of the bug:
 signed integer overflow

 poc:
 [https://drive.google.com/file/d/1afws3WCzvRBc213jnIMfz_96MFNglGzd/view?usp=sharing]

 fuzz at Fuzz2:~/Desktop/projects_oss/FFmpeg/tools/fuzzout$
 ./target_sws_fuzzer /home/fuzz/Desktop/langgraph/testpro/AFL-
 Agent/utils_c_389
 Reading 339 bytes from /home/fuzz/Desktop/langgraph/testpro/AFL-
 Agent/utils_c_389
 2 x 3 yuva420p10le -> 26 x 3 bgra64le
 libswscale/output.c:1325:33: runtime error: signed integer overflow:
 -3421696 * 2048 cannot be represented in type 'int'
 SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior
 libswscale/output.c:1325:33 in
 libswscale/output.c:1325:55: runtime error: signed integer overflow:
 -3421696 * 2048 cannot be represented in type 'int'
 SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior
 libswscale/output.c:1325:55 in
 libswscale/output.c:1325:44: runtime error: signed integer overflow:
 1582301184 + 1582301184 cannot be represented in type 'int'
 SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior
 libswscale/output.c:1325:44 in
 libswscale/output.c:1325:65: runtime error: signed integer overflow:
 -1130364928 - 1073741824 cannot be represented in type 'int'
 SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior
 libswscale/output.c:1325:65 in
 libswscale/output.c:1326:55: runtime error: signed integer overflow:
 -3487744 * 2048 cannot be represented in type 'int'
 SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior
 libswscale/output.c:1326:55 in
 libswscale/output.c:1326:44: runtime error: signed integer overflow:
 1073739776 + 1447034880 cannot be represented in type 'int'
 SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior
 libswscale/output.c:1326:44 in
 libswscale/output.c:1326:65: runtime error: signed integer overflow:
 -1774192640 - 1073741824 cannot be represented in type 'int'
 SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior
 libswscale/output.c:1326:65 in
 Execution successful.
-- 
Ticket URL: <https://trac.ffmpeg.org/ticket/11686>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker