[FFmpeg-cvslog] avformat/hls: Limit start_seq_no to one bit less

[Michael Niedermayer]

ffmpeg | branch: release/5.0 | Michael Niedermayer <michael at niedermayer.cc> | Thu Jun 16 23:02:11 2022 +0200| [40ed3f6e8434bff12835133a14475eb3ed2fd140] | committer: Michael Niedermayer

avformat/hls: Limit start_seq_no to one bit less

This avoids overflow checks on additions with 32bit numbers

Fixes: signed integer overflow: 9223372036854775806 + 2 cannot be represented in type 'long'
Fixes: 44012/clusterfuzz-testcase-minimized-ffmpeg_dem_HLS_fuzzer-4747770734444544
Fixes: 48065/clusterfuzz-testcase-minimized-ffmpeg_dem_HLS_fuzzer-5372410355908608

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael at niedermayer.cc>
(cherry picked from commit d8ee01425459aaafe36acc7743b3f9f28a01821b)
Signed-off-by: Michael Niedermayer <michael at niedermayer.cc>

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=40ed3f6e8434bff12835133a14475eb3ed2fd140
---

 libavformat/hls.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/libavformat/hls.c b/libavformat/hls.c
index 53be0f591c..53d30d97ad 100644
--- a/libavformat/hls.c
+++ b/libavformat/hls.c
@@ -833,10 +833,10 @@ static int parse_playlist(HLSContext *c, const char *url,
             if (ret < 0)
                 goto fail;
             seq_no = strtoull(ptr, NULL, 10);
-            if (seq_no > INT64_MAX) {
+            if (seq_no > INT64_MAX/2) {
                 av_log(c->ctx, AV_LOG_DEBUG, "MEDIA-SEQUENCE higher than "
-                        "INT64_MAX, mask out the highest bit\n");
-                seq_no &= INT64_MAX;
+                        "INT64_MAX/2, mask out the highest bit\n");
+                seq_no &= INT64_MAX/2;
             }
             pls->start_seq_no = seq_no;
         } else if (av_strstart(line, "#EXT-X-PLAYLIST-TYPE:", &ptr)) {