[FFmpeg-devel] [PATCH V3 3/4] lavc/vaapi_encode_h264: respect "slices" option in h264 vaapi encoder

[Mark Thompson]

On 18/08/17 03:14, Jun Zhao wrote:
> From 64c53df3134c0b478d212f141e6dbe4c3743b3c6 Mon Sep 17 00:00:00 2001
> From: Jun Zhao <jun.zhao at intel.com>
> Date: Tue, 1 Aug 2017 23:05:44 -0400
> Subject: [PATCH V3 3/4] lavc/vaapi_encode_h264: respect "slices" option in
>  h264 vaapi encoder
> 
> Enable multi-slice support in AVC/H.264 vaapi encoder.
> 
> Signed-off-by: Wang, Yi A <yi.a.wang at intel.com>
> Signed-off-by: Jun Zhao <jun.zhao at intel.com>
> ---
>  libavcodec/vaapi_encode_h264.c | 32 +++++++++++++++++++++++++++-----
>  1 file changed, 27 insertions(+), 5 deletions(-)
> 
> diff --git a/libavcodec/vaapi_encode_h264.c b/libavcodec/vaapi_encode_h264.c
> index f9fcd805a4..74d7480321 100644
> --- a/libavcodec/vaapi_encode_h264.c
> +++ b/libavcodec/vaapi_encode_h264.c
> @@ -141,6 +141,8 @@ typedef struct VAAPIEncodeH264Context {
>      int mb_width;
>      int mb_height;
>  
> +    int last_mb_index;
> +
>      int fixed_qp_idr;
>      int fixed_qp_p;
>      int fixed_qp_b;
> @@ -957,6 +959,7 @@ static int vaapi_encode_h264_init_picture_params(AVCodecContext *avctx,
>      VAEncPictureParameterBufferH264  *vpic = pic->codec_picture_params;
>      VAAPIEncodeH264Context           *priv = ctx->priv_data;
>      int i;
> +    int slices;
>  
>      if (pic->type == PICTURE_TYPE_IDR) {
>          av_assert0(pic->display_order == pic->encode_order);
> @@ -1002,7 +1005,19 @@ static int vaapi_encode_h264_init_picture_params(AVCodecContext *avctx,
>      vpic->pic_fields.bits.idr_pic_flag = (pic->type == PICTURE_TYPE_IDR);
>      vpic->pic_fields.bits.reference_pic_flag = (pic->type != PICTURE_TYPE_B);
>  
> -    pic->nb_slices = 1;
> +    slices = 1;
> +    if (ctx->max_slices) {
> +        if (avctx->slices <= ctx->max_slices) {
> +            slices = avctx->slices;
> +        } else {
> +            av_log(avctx, AV_LOG_ERROR, "The max slices number per frame "
> +                   "cannot more than %d.\n", ctx->max_slices);
> +            return AVERROR_INVALIDDATA;
> +        }
> +    }
> +    pic->nb_slices = slices;
> +
> +    priv->last_mb_index = 0;
>  
>      return 0;
>  }
> @@ -1052,14 +1067,17 @@ static int vaapi_encode_h264_init_slice_params(AVCodecContext *avctx,
>          av_assert0(0 && "invalid picture type");
>      }
>  
> -    // Only one slice per frame.
> -    vslice->macroblock_address = 0;
> -    vslice->num_macroblocks = priv->mb_width * priv->mb_height;
> +    vslice->macroblock_address = priv->last_mb_index;
> +    vslice->num_macroblocks =
> +        ((slice->index + 1) * priv->mb_width * priv->mb_height) / pic->nb_slices - priv->last_mb_index;
> +    priv->last_mb_index += vslice->num_macroblocks;
>  
>      vslice->macroblock_info = VA_INVALID_ID;
>  
>      vslice->pic_parameter_set_id = vpic->pic_parameter_set_id;
> -    vslice->idr_pic_id = priv->idr_pic_count++;
> +    vslice->idr_pic_id = priv->idr_pic_count;
> +    if (priv->last_mb_index == priv->mb_width * priv->mb_height)
> +        priv->idr_pic_count++;
>  
>      vslice->pic_order_cnt_lsb = (pic->display_order - priv->last_idr_frame) &
>          ((1 << (4 + vseq->seq_fields.bits.log2_max_pic_order_cnt_lsb_minus4)) - 1);
> @@ -1157,6 +1175,10 @@ static av_cold int vaapi_encode_h264_configure(AVCodecContext *avctx)
>  #endif
>      }
>  
> +    if (!ctx->max_slices && avctx->slices > 0)
> +        av_log(avctx, AV_LOG_WARNING, "The encode slice option is not "
> +               "supported with the driver.\n");
> +
>      return 0;
>  }
>  
> -- 
> 2.11.0
> 

I guess this works, but I still think these patches need more thought about / explanation of what the useful result we want actually is.

As such, what is your intended use-case for this?

Some uses for multiple slices that I can think of:
1) Optimising encode-time parallelism.
2) Optimising decode-time parallelism.
3) Imposing slice size limits for some transmission constraint (e.g. RFC 3984 video in non-non-interleaved mode).
4) Conforming to some standard which requires them (e.g. bluray at higher bitrates).
(More?)

For VAAPI encode I think both (1) and (3) are irrelevant - (1) doesn't matter because the hardware encoders beign targetted don't use this sort of technique, while (3) is just impossible because of API constraints.  Hence we can consider only (2) and (4).

The AVCodecContext.slices option maps directly to the x264 slice_count option to support (2), and I suppose also (4) - this generates slices which are always whole rows of macroblocks.  It is also possible to generate equally-sized slices (which need not be rectangular) via the slice_max_mbs option, but that's much less obvious.

For (4), based on what I've seen of bluray-type encodes they always make their slices with rectangular regions, with all but the final slice being of identical size.  (I haven't looked at many, though, so this is purely anecdotal and there may be a lot of counterexamples.)

Thoughts?

- Mark