[FFmpeg-devel] [PATCH 1/2] lavc/vaapi_encode: fix the caculation overflow

Qu, Pengfei pengfei.qu at intel.com
Tue Mar 13 03:56:50 EET 2018



-----Original Message-----
From: ffmpeg-devel [mailto:ffmpeg-devel-bounces at ffmpeg.org] On Behalf Of Mark Thompson
Sent: Tuesday, March 13, 2018 6:54 AM
To: ffmpeg-devel at ffmpeg.org
Subject: Re: [FFmpeg-devel] [PATCH 1/2] lavc/vaapi_encode: fix the caculation overflow

On 12/03/18 05:38, Pengfei Qu wrote:
> this fix the overflow during the caculation before value assignment.
> 
> Signed-off-by: Pengfei Qu <Pengfei.Qu at intel.com>
> ---
>  libavcodec/vaapi_encode.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/libavcodec/vaapi_encode.c b/libavcodec/vaapi_encode.c 
> index 36c85a3..78347d4 100644
> --- a/libavcodec/vaapi_encode.c
> +++ b/libavcodec/vaapi_encode.c
> @@ -1168,9 +1168,9 @@ static av_cold int vaapi_encode_init_rate_control(AVCodecContext *avctx)
>              rc_target_percentage = 100;
>          } else {
>              rc_bits_per_second   = avctx->rc_max_rate;
> -            rc_target_percentage = (avctx->bit_rate * 100) / rc_bits_per_second;
> +            rc_target_percentage = (unsigned long)(avctx->bit_rate * 
> + 100) / rc_bits_per_second;

What is the problem here?  All this will do is truncate the 64-bit value on platforms with 32-bit long.
[Pengfei] same as the below.
>          }
> -        rc_window_size = (hrd_buffer_size * 1000) / avctx->bit_rate;
> +        rc_window_size = (unsigned long)(hrd_buffer_size * 1000) / 
> + avctx->bit_rate;

Perhaps you want to ensure that the multiplication doesn't overflow rather than truncating the result after it has overflowed.
[Pengfei] Yes.
>      }
>  
>      ctx->rc_params.misc.type = VAEncMiscParameterTypeRateControl;
> 
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel at ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel


More information about the ffmpeg-devel mailing list