[FFmpeg-devel] [PATCH] cafdec: fix overflow checking in read_header()

[Xi Wang]

Several compilers such as clang/icc/pathscale will optimize the check
pos + size < pos (assuming size > 0) into false, since signed integer
overflow is undefined behavior in C.  This breaks overflow checking.
Use a safe precondition check instead.

Signed-off-by: Xi Wang <xi.wang at gmail.com>
---
Below is the simplified code. 

#include <stdint.h>
void bar(void);
void foo(int64_t pos, int64_t size)
{
	if (size > 0) {
		if (pos + size < pos)
			bar();
	}
}

$ clang -S -o - t.c -O2
foo:                                    # @foo
	.cfi_startproc
# BB#0:                                 # %entry
	ret

icc and pathscale produce the same output.
---
 libavformat/cafdec.c |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/libavformat/cafdec.c b/libavformat/cafdec.c
index f12226a..337758e 100644
--- a/libavformat/cafdec.c
+++ b/libavformat/cafdec.c
@@ -300,7 +300,7 @@ static int read_header(AVFormatContext *s)
         }
 
         if (size > 0) {
-            if (pos + size < pos)
+            if (pos > INT64_MAX - size)
                 return AVERROR_INVALIDDATA;
             avio_skip(pb, FFMAX(0, pos + size - avio_tell(pb)));
         }
-- 
1.7.10.4