[FFmpeg-trac] #7650(avcodec:new): FFmpeg MPEG-2 encode fails to "invalid RC mode" when iHD driver is used through VA-API instead of QSV

[FFmpeg]

#7650: FFmpeg MPEG-2 encode fails to "invalid RC mode" when iHD driver is used
through VA-API instead of QSV
-------------------------------------+-----------------------------------
             Reporter:  eero-t       |                    Owner:
                 Type:  defect       |                   Status:  new
             Priority:  normal       |                Component:  avcodec
              Version:  unspecified  |               Resolution:
             Keywords:  vaapi        |               Blocked By:
             Blocking:               |  Reproduced by developer:  0
Analyzed by developer:  0            |
-------------------------------------+-----------------------------------

Comment (by fulinjie):

 I think the psnr issue was caused by the mismatch between FFmpeg and iHD
 driver.

 In FFmpeg VAAPI level, hrd_buffer_size and hrd_initial_buffer_fullness
 will be set in vaapi_encode_init_rate_control:
 https://github.com/FFmpeg/FFmpeg/blob/282a4718576d6928b6c5900db89b45d83556407a/libavcodec/vaapi_encode.c#L1305
 with a higher priority (vbvBuffer_size set in
 vaapi_encode_mpeg2_init_sequence_params will be covered in driver).


 The mismatch exists in the unit of the m_vbvBufferSize:
 ffmpeg vaapi: in bits;
 mpeg2 in driver: in 16kbits;

 Another problem is the m_rateControlMethod was set to RATECONTROL_CBR if
 calling DdiEncodeMpeg2::ParseMiscParamVBV in driver.

 One way to solve this PSNR issue in FFmpeg level is set the bufsize
 specially for mpeg2 as the unit of 16kb, and discard the default CBR mode
 in driver.


 Another way is to fix this all in driver level:
 https://github.com/intel/media-driver/pull/495

--
Ticket URL: <https://trac.ffmpeg.org/ticket/7650#comment:4>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker