[FFmpeg-devel] [PATCH] avformat/cache: Avoid int-overflow in cache compare function

[Bryan Huh]

cache protocol indexes its cache using AVTreeNodes which require a cmp
function for inserting and searching new cache-entries. This cmp
function expects a 32-bit int return value (negative, zero, or positive)
but the cache cmp function returns an int64_t which can overflow the
int, giving negative numbers for when it should be positive, vice versa.
This manifests itself only for very large files (e.g. 4GB+)
---
 libavformat/cache.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/libavformat/cache.c b/libavformat/cache.c
index 31f63e6..d41161d 100644
--- a/libavformat/cache.c
+++ b/libavformat/cache.c
@@ -67,7 +67,7 @@ typedef struct Context {
 
 static int cmp(const void *key, const void *node)
 {
-    return (*(const int64_t *) key) - ((const CacheEntry *) node)->logical_pos;
+    return FFDIFFSIGN(*(const int64_t *)key, ((const CacheEntry *) node)->logical_pos);
 }
 
 static int cache_open(URLContext *h, const char *arg, int flags, AVDictionary **options)
-- 
1.7.1