[FFmpeg-devel] [PATCH v2] vaapi_encode_h264: Only set pic_order_cnt_type to 0 with B-frames

[Xiang, Haihao]

On Di, 2023-01-10 at 09:17 +0100, David Rosca wrote:
> v2: frame_num steps by 2
> 
> ---
>  libavcodec/vaapi_encode_h264.c | 7 +++++--
>  1 file changed, 5 insertions(+), 2 deletions(-)
> 
> diff --git a/libavcodec/vaapi_encode_h264.c b/libavcodec/vaapi_encode_h264.c
> index 7a6b54ab6f..8093c47179 100644
> --- a/libavcodec/vaapi_encode_h264.c
> +++ b/libavcodec/vaapi_encode_h264.c
> @@ -349,7 +349,7 @@ static int
> vaapi_encode_h264_init_sequence_params(AVCodecContext *avctx)
>      sps->chroma_format_idc    = 1;
>  
>      sps->log2_max_frame_num_minus4 = 4;
> -    sps->pic_order_cnt_type        = 0;
> +    sps->pic_order_cnt_type        = ctx->max_b_depth ? 0 : 2;
>      sps->log2_max_pic_order_cnt_lsb_minus4 = 4;
>  
>      sps->max_num_ref_frames = priv->dpb_frames;
> @@ -621,7 +621,10 @@ static int
> vaapi_encode_h264_init_picture_params(AVCodecContext *avctx,
>      } else {
>          av_assert0(prev);
>  
> -        hpic->frame_num = hprev->frame_num + prev->is_reference;
> +        hpic->frame_num = hprev->frame_num;
> +        if (prev->is_reference) {
> +            hpic->frame_num += ctx->max_b_depth ? 1 : 2;
> +        }
>  
>          hpic->last_idr_frame = hprev->last_idr_frame;
>          hpic->idr_pic_id     = hprev->idr_pic_id;

The coded stream is probably corrupted on Intel's Tiger Lake after applying this
change.

$ ffmpeg -y -init_hw_device vaapi=va:/dev/dri/renderD128 -f lavfi -i testsrc -vf
"format=nv12,hwupload" -bf 0 -c:v h264_vaapi -vframes 100 out.h264

$ ffprobe out.h264
......
[h264 @ 0x55cc3d7dc200] mmco: unref short failure
[h264 @ 0x55cc3d7dc200] number of reference frames (0+2) exceeds max (1;
probably corrupt input), discarding one
[h264 @ 0x55cc3d7dc200] mmco: unref short failure
[h264 @ 0x55cc3d7dc200] number of reference frames (0+2) exceeds max (1;
probably corrupt input), discarding one
[h264 @ 0x55cc3d7dc200] mmco: unref short failure
[h264 @ 0x55cc3d7dc200] number of reference frames (0+2) exceeds max (1;
probably corrupt input), discarding one
[h264 @ 0x55cc3d7dc200] mmco: unref short failure
[h264 @ 0x55cc3d7dc200] number of reference frames (0+2) exceeds max (1;
probably corrupt input), discarding one
[h264 @ 0x55cc3d7dc200] mmco: unref short failure
[h264 @ 0x55cc3d7dc200] number of reference frames (0+2) exceeds max (1;
probably corrupt input), discarding one
[h264 @ 0x55cc3d7dc200] mmco: unref short failure
[h264 @ 0x55cc3d7dc200] number of reference frames (0+2) exceeds max (1;
probably corrupt input), discarding one

Thanks
Haihao