[FFmpeg-devel] [PATCH 1/3] exr: limit expected_len to tmp buffer size

[Martin Vignali]

>
> td->xsize = 800
> td->ysize = 16
> s->current_channel_offset = 5
> td->channel_line_size = s->current_channel_offset * td->xsize = 4000
> uncompressed_size = td->channel_line_size * td->ysize = 64000
>
> s->nb_channels = 3
>
> p s->channels[0].pixel_type = EXR_HALF
> expected_len = td->xsize * td->ysize * 2 = 25600
>
> p s->channels[1].pixel_type = EXR_UINT
> expected_len = expected_len + td->xsize * td->ysize * 4 = 76800
>
> p s->channels[2].pixel_type = EXR_HALF
> expected_len = expected_len + td->xsize * td->ysize * 2 = 102400
>
> => expected_len > uncompressed_size
>
> > Did you have a test sample ?
>
> Yes.
>
>
Thanks for your explanation.

I think the trouble come from another place
The current channel offset is not correct for uint32 (because it use 1<<0 =
1 instead of 4 for uint32)

Some months ago i propose a patch who was not accepted for uint32 support
in exr.
But the current channel offset is correctly calculate in this patch.

Conversation can be found here :
http://ffmpeg.org/pipermail/ffmpeg-devel/2016-April/192527.html

The interested part is that :
-                s->current_channel_offset += 1 << current_pixel_type;
+                if (current_pixel_type == EXR_HALF) {
+                    s->current_channel_offset += 2;
+                } else {/* Float or UINT32 */
+                    s->current_channel_offset += 4;
+                }


I think it's better to fix the uncompress size calculation.

Martin