[FFmpeg-devel] [PATCH 5/5] avcodec/wavpack: Avoid undefined shift in get_tail()

[Andreas Rheinhardt]

James Almer:
> On 12/18/2022 2:08 PM, Michael Niedermayer wrote:
>> Fixes: left shift of 1208485947 by 1 places cannot be represented in
>> type 'int'
>> Fixes:
>> 54058/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_WAVPACK_fuzzer-5827521084260352
>>
>> Found-by: continuous fuzzing process
>> https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
>> Signed-off-by: Michael Niedermayer <michael at niedermayer.cc>
>> ---
>>   libavcodec/wavpack.c | 2 +-
>>   1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/libavcodec/wavpack.c b/libavcodec/wavpack.c
>> index 3cb4077550..42859ab0a1 100644
>> --- a/libavcodec/wavpack.c
>> +++ b/libavcodec/wavpack.c
>> @@ -129,7 +129,7 @@ static av_always_inline unsigned
>> get_tail(GetBitContext *gb, unsigned k)
>>       e   = (1LL << (p + 1)) - k - 1;
>>       res = get_bits_long(gb, p);
>>       if (res >= e)
>> -        res = (res << 1) - e + get_bits1(gb);
>> +        res = ((unsigned)res << 1) - e + get_bits1(gb);
> 
> Don't we usually do << 1U for this?
> 

Definitely not. The type of a shift is given by the left operand, not
the right operand, so using << 1U doesn't help at all here.
(We often use "* 2U" in such cases; "* (1U << 1)" would also be possible.)

- Andreas