[FFmpeg-cvslog] avutil/mathematics: Fix 2 overflows in av_add_stable()

[Michael Niedermayer]

ffmpeg | branch: master | Michael Niedermayer <michael at niedermayer.cc> | Sat Aug 10 21:09:03 2019 +0200| [679f3408906081036afacbc94f325793f303c039] | committer: Michael Niedermayer

avutil/mathematics: Fix 2 overflows in av_add_stable()

Fixes: signed integer overflow: 9223372036854775807 + 1 cannot be represented in type 'long'
Fixes: 16022/clusterfuzz-testcase-minimized-ffmpeg_DEMUXER_fuzzer-5759796759756800

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael at niedermayer.cc>

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=679f3408906081036afacbc94f325793f303c039
---

 libavutil/mathematics.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/libavutil/mathematics.c b/libavutil/mathematics.c
index 1bf044cdf1..0485db7222 100644
--- a/libavutil/mathematics.c
+++ b/libavutil/mathematics.c
@@ -198,7 +198,7 @@ int64_t av_add_stable(AVRational ts_tb, int64_t ts, AVRational inc_tb, int64_t i
     m = inc_tb.num * (int64_t)ts_tb.den;
     d = inc_tb.den * (int64_t)ts_tb.num;
 
-    if (m % d == 0)
+    if (m % d == 0 && ts <= INT64_MAX - m / d)
         return ts + m / d;
     if (m < d)
         return ts;
@@ -206,6 +206,10 @@ int64_t av_add_stable(AVRational ts_tb, int64_t ts, AVRational inc_tb, int64_t i
     {
         int64_t old = av_rescale_q(ts, ts_tb, inc_tb);
         int64_t old_ts = av_rescale_q(old, inc_tb, ts_tb);
+
+        if (old == INT64_MAX)
+            return ts;
+
         return av_rescale_q(old + 1, inc_tb, ts_tb) + (ts - old_ts);
     }
 }