[FFmpeg-trac] #7650(avcodec:new): FFmpeg MPEG-2 encode fails to "invalid RC mode" when iHD driver is used through VA-API instead of QSV

[FFmpeg]

#7650: FFmpeg MPEG-2 encode fails to "invalid RC mode" when iHD driver is used
through VA-API instead of QSV
-------------------------------------+-----------------------------------
             Reporter:  eero-t       |                    Owner:
                 Type:  defect       |                   Status:  new
             Priority:  normal       |                Component:  avcodec
              Version:  unspecified  |               Resolution:
             Keywords:               |               Blocked By:
             Blocking:               |  Reproduced by developer:  0
Analyzed by developer:  0            |
-------------------------------------+-----------------------------------

Comment (by fulinjie):

 VA_RC_VBR mode is set in vaapi_encode_init_rate_control:
 https://github.com/FFmpeg/FFmpeg/blob/eb81fd792fa88f9015b5e99e4940a464de9182a5/libavcodec/vaapi_encode.c#L1362

 And calls ctx->codec->configure() in ff_vaapi_encode_init to set
 vaapi_encode_mpeg2_configure.

 You may try to use default CQP mode by remove -b:v 2M, or add support for
 VBR and CBR:

 {{{
 --- a/libavcodec/vaapi_encode_mpeg2.c
 +++ b/libavcodec/vaapi_encode_mpeg2.c
 @@ -541,6 +541,13 @@ static av_cold int
 vaapi_encode_mpeg2_configure(AVCodecContext *avctx)
                 "%d / %d / %d for I- / P- / B-frames.\n",
                 priv->quant_i, priv->quant_p, priv->quant_b);

 +    } else if (ctx->va_rc_mode == VA_RC_CBR ||
 +               ctx->va_rc_mode == VA_RC_VBR) {
 +        // These still need to be set for quantiser_scale_code.
 +        priv->quant_i = 10;
 +        priv->quant_p = 10;
 +        priv->quant_b = 10;
 +
      } else {
          av_assert0(0 && "Invalid RC mode.");
      }


 }}}

--
Ticket URL: <https://trac.ffmpeg.org/ticket/7650#comment:1>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker