[FFmpeg-devel] [PATCH] lavu: check for overflow in av_clip_intp2_c

[Andreas Cadhalpun]

On 13.01.2016 20:06, Michael Niedermayer wrote:
> On Wed, Jan 13, 2016 at 12:52:21AM +0100, Andreas Cadhalpun wrote:
>> ---
>>  libavutil/common.h | 2 +-
>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/libavutil/common.h b/libavutil/common.h
>> index f3276a2..5ae2847 100644
>> --- a/libavutil/common.h
>> +++ b/libavutil/common.h
>> @@ -211,7 +211,7 @@ static av_always_inline av_const int32_t av_clipl_int32_c(int64_t a)
>>   */
>>  static av_always_inline av_const int av_clip_intp2_c(int a, int p)
>>  {
>> -    if ((a + (1 << p)) & ~((2 << p) - 1))
>> +    if (a >= INT_MAX - (1 << p) || ((a + (1 << p)) & ~((2 << p) - 1)))
> 
> does this differ in any case from using unsigned for the addition?

I don't think so.

> if not then using unsigned avoids the additional operations

OK, patch doing that attached.

Best regards,
Andreas

-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0001-lavu-prevent-overflow-in-av_clip_intp2_c.patch
Type: text/x-diff
Size: 969 bytes
Desc: not available
URL: <http://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20160114/4f1edb62/attachment.patch>