[FFmpeg-devel] [PATCH] avformat/cache: Avoid int-overflow in cache compare function

[Michael Niedermayer]

On Sat, Nov 07, 2015 at 08:53:54PM -0800, Bryan Huh wrote:
> cache protocol indexes its cache using AVTreeNodes which require a cmp
> function for inserting and searching new cache-entries. This cmp
> function expects a 32-bit int return value (negative, zero, or positive)
> but the cache cmp function returns an int64_t which can overflow the
> int, giving negative numbers for when it should be positive, vice versa.
> This manifests itself only for very large files (e.g. 4GB+)
> ---
>  libavformat/cache.c |    3 ++-
>  1 files changed, 2 insertions(+), 1 deletions(-)
> 
> diff --git a/libavformat/cache.c b/libavformat/cache.c
> index 31f63e6..5631586 100644
> --- a/libavformat/cache.c
> +++ b/libavformat/cache.c
> @@ -67,7 +67,8 @@ typedef struct Context {
>  
>  static int cmp(const void *key, const void *node)
>  {
> -    return (*(const int64_t *) key) - ((const CacheEntry *) node)->logical_pos;
> +    int64_t diff = (*(const int64_t *) key) - ((const CacheEntry *) node)->logical_pos;
> +    return diff > 0 ? 1 : diff < 0 ? -1 : 0;
>  }

see FFDIFFSIGN()


[...]
-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

Into a blind darkness they enter who follow after the Ignorance,
they as if into a greater darkness enter who devote themselves
to the Knowledge alone. -- Isha Upanishad
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: Digital signature
URL: <http://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20151108/96b30ad8/attachment.sig>