[FFmpeg-devel] [PATCH 2/2] avcodec/hevc_cabac: Check prefix so as to avoid invalid shifts in coeff_abs_level_remaining_decode()

[James Almer]

On 1/15/2018 9:33 PM, Ronald S. Bultje wrote:
> Hi,
> 
> On Mon, Jan 15, 2018 at 6:37 PM, Michael Niedermayer <michael at niedermayer.cc
>> wrote:
> 
>> I suspect that this can be limited tighter, but i failed to find anything
>> in the spec that would confirm that.
>>
>> Fixes: 4833/clusterfuzz-testcase-minimized-5302840101699584
>> Fixes: runtime error: left shift of 134217730 by 4 places cannot be
>> represented in type 'int'
>>
>> Found-by: continuous fuzzing process https://github.com/google/oss-
>> fuzz/tree/master/projects/ffmpeg
>> Signed-off-by: Michael Niedermayer <michael at niedermayer.cc>
>> ---
>>  libavcodec/hevc_cabac.c | 2 +-
>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/libavcodec/hevc_cabac.c b/libavcodec/hevc_cabac.c
>> index 743168500c..faa36d5459 100644
>> --- a/libavcodec/hevc_cabac.c
>> +++ b/libavcodec/hevc_cabac.c
>> @@ -998,7 +998,7 @@ static av_always_inline int coeff_abs_level_remaining_decode(HEVCContext
>> *s, int
>>      } else {
>>          int prefix_minus3 = prefix - 3;
>>
>> -        if (prefix == CABAC_MAX_BIN) {
>> +        if (prefix == CABAC_MAX_BIN || prefix_minus3 + rc_rice_param >=
>> 31) {
>>              av_log(s->avctx, AV_LOG_ERROR, "CABAC_MAX_BIN : %d\n",
>> prefix);
>>              return 0;
>>          }
> 
> 
> I understand this is unrelated to the patch, but I once again want to point
> out how utterly useless this error message is for end users :-(.
> 
> Ronald

This one is particularly bizarre as well. It prints "CABAC_MAX_BIN"
followed by it's constant value, as if it could change.
That apparently will not be the case anymore after this patch, though,
with the new check.