[FFmpeg-devel] stsz overflow

Frank Barchard fbarchard
Tue Aug 25 01:20:42 CEST 2009

On Mon, Aug 24, 2009 at 4:08 PM, Alex Converse <alex.converse at gmail.com>wrote:

> >     num_bytes = (entries*field_size+4)>>3;
> The intermediate product here is the part that overflows. A final
> num_bytes calculated with appropriate intermediate precision should
> fit in in an unsigned 32-bit integer. Why not just fix that rather
> than reduce the number of entries supported?

Ya, you're right.  I see other code that uses uint64_t to calculate sizes
for malloc, and then it would have a chance of success.
The code that follows the failed malloc returns an error code, which is

    num_bytes = (unsigned int)(((uint64_t)entries*field_size+7)>>3);
    buf = av_malloc(num_bytes+FF_INPUT_BUFFER_PADDING_SIZE);    if
(!buf) {        av_freep(&sc->sample_sizes);        return

updated patch here:

Index: libavformat/mov.c
--- libavformat/mov.c   (revision 19695)
+++ libavformat/mov.c   (working copy)
@@ -1262,7 +1262,7 @@
     if (!sc->sample_sizes)
         return AVERROR(ENOMEM);

-    num_bytes = (entries*field_size+4)>>3;
+    num_bytes = (unsigned int)(((uint64_t)entries*field_size+4)>>3);

     buf = av_malloc(num_bytes+FF_INPUT_BUFFER_PADDING_SIZE);
     if (!buf) {

More information about the ffmpeg-devel mailing list