[FFmpeg-trac] #8713(ffmpeg:new): NVenc encodes broken? Now treble file size for exactly same commandline

[FFmpeg]

#8713: NVenc encodes broken? Now treble file size for exactly same commandline
--------------------------------------+----------------------------------
             Reporter:  hydra3333     |                    Owner:
                 Type:  defect        |                   Status:  new
             Priority:  critical      |                Component:  ffmpeg
              Version:  git-master    |               Resolution:
             Keywords:  nvenc nvidia  |               Blocked By:
             Blocking:                |  Reproduced by developer:  0
Analyzed by developer:  0             |
--------------------------------------+----------------------------------

Comment (by hydra3333):

 1. 37140ebd87f549eae86a5b548d717a1e97203dd6 11 days ago Andreas Rheinhardt
 avformat/id3v2: Remove unnecessary indirection
 gives lower file sizes

 2. c0f01eaf12479b5d0160e1a8d10a73744fac5ab2 8 days ago  Andreas Rheinhardt
 avformat/oggdec: Avoid duplicating buffer when adding...
 under nvidia driver 446.14 the result is:
 gives lower file sizes

 3. 7f76f20fa0a8aad6de50634a5cb60dc53117d6ab 3 days ago  Jun Zhao
 checkasm: sw_rgb: Fix mixed declaration and code
 under nvidia driver 446.14 the result is:
 gives lower file sizes

 4. 0842fd2c25c050e595083ba1e8c5176da3d55c9b 3 days ago  Roman Arzumanyan
 avcodec/nvenc: zero avg and max bitrate in CQ mode
 under nvidia driver 446.14 the result is:


 5. 470bbf60d4cd34abd2cabea40910af6c0fc2366c 24 hours ago        Roman
 Arzumanyan        avcodec/nvenc: honor max bitrate in CQ mode
 under nvidia driver 446.14 the result is:
 HIGHER file size

 6. b6c8444e2387d9a75b24d5ec1d8308fe6e37645b (MASTER) 46 min ago James
 Almer     avutil/buffer: separate public and internal flags insid...
 master
 under nvidia driver 446.14 the result is:
 HIGHER file size

 So, it broke between
 7f76f20fa0a8aad6de50634a5cb60dc53117d6ab 3 days ago     Jun Zhao
 checkasm: sw_rgb: Fix mixed declaration and code
 and
 470bbf60d4cd34abd2cabea40910af6c0fc2366c 24 hours ago   Roman Arzumanyan
 avcodec/nvenc: honor max bitrate in CQ mode

 So ... I suspect '''0842fd2c25c050e595083ba1e8c5176da3d55c9b''' broke it
 first, as it did this
 {{{
 diff --git a/libavcodec/nvenc.c b/libavcodec/nvenc.c
 index cbdddd3..a9f9600 100644 (file)
 --- a/libavcodec/nvenc.c
 +++ b/libavcodec/nvenc.c
 @@ -947,12 +947,17 @@ static av_cold void
 nvenc_setup_rate_control(AVCodecContext *avctx)
      if (ctx->zerolatency)
          ctx->encode_config.rcParams.zeroReorderDelay = 1;

 -    if (ctx->quality)
 -    {
 +    if (ctx->quality) {
          //convert from float to fixed point 8.8
          int tmp_quality = (int)(ctx->quality * 256.0f);
          ctx->encode_config.rcParams.targetQuality = (uint8_t)(tmp_quality
 >> 8);
          ctx->encode_config.rcParams.targetQualityLSB =
 (uint8_t)(tmp_quality & 0xff);
 +
 +        av_log(avctx, AV_LOG_VERBOSE, "CQ(%d) mode enabled.\n",
 tmp_quality);
 +
 +        //CQ mode shall discard max & avg bitrate;
 +        avctx->bit_rate    = ctx->encode_config.rcParams.averageBitRate =
 0;
 +        avctx->rc_max_rate = ctx->encode_config.rcParams.maxBitRate     =
 0;
      }
  }
 }}}

 Then '''470bbf60d4cd34abd2cabea40910af6c0fc2366c''' did not fix the break,
 as it did this
 {{{
 diff --git a/libavcodec/nvenc.c b/libavcodec/nvenc.c
 index 880454c..e269c71 100644 (file)
 --- a/libavcodec/nvenc.c
 +++ b/libavcodec/nvenc.c
 @@ -956,9 +956,9 @@ static av_cold void
 nvenc_setup_rate_control(AVCodecContext *avctx)

          av_log(avctx, AV_LOG_VERBOSE, "CQ(%d) mode enabled.\n",
 tmp_quality);

 -        //CQ mode shall discard max & avg bitrate;
 -        avctx->bit_rate    = ctx->encode_config.rcParams.averageBitRate =
 0;
 -        avctx->rc_max_rate = ctx->encode_config.rcParams.maxBitRate     =
 0;
 +        //CQ mode shall discard avg bitrate & honor max bitrate;
 +        ctx->encode_config.rcParams.averageBitRate = avctx->bit_rate = 0;
 +        ctx->encode_config.rcParams.maxBitRate = avctx->rc_max_rate;
      }
  }
 }}}

 File sizes produced with the various commits under nvidia driver 446.14
 (in ascending order of commit) :
 {{{
 05/06/2020  11:08 PM     1,476,054,817
 01-ffmpeg_nvenc_MY_HOME_ENCODE_ffmpeg_37140ebd87f549eae86a5b548d717a1e97203dd6.mp4
 06/06/2020  12:33 AM     1,476,054,817
 01-ffmpeg_nvenc_MY_HOME_ENCODE_ffmpeg_c0f01eaf12479b5d0160e1a8d10a73744fac5ab2.mp4
 06/06/2020  01:13 AM     1,476,054,817
 01-ffmpeg_nvenc_MY_HOME_ENCODE_ffmpeg_7f76f20fa0a8aad6de50634a5cb60dc53117d6ab.mp4
 06/06/2020  01:58 AM     5,109,921,859
 01-ffmpeg_nvenc_MY_HOME_ENCODE_ffmpeg_0842fd2c25c050e595083ba1e8c5176da3d55c9b.mp4
 06/06/2020  01:25 AM     5,109,921,859
 01-ffmpeg_nvenc_MY_HOME_ENCODE_ffmpeg_470bbf60d4cd34abd2cabea40910af6c0fc2366c.mp4
 06/06/2020  01:35 AM     5,109,921,859
 01-ffmpeg_nvenc_MY_HOME_ENCODE_ffmpeg_b6c8444e2387d9a75b24d5ec1d8308fe6e37645b.mp4
 }}}

 Where to next ?

 Separately to the above, tomorrow I hope to downgrade the nvidia driver
 and do a couple of commit comparisons.

--
Ticket URL: <https://trac.ffmpeg.org/ticket/8713#comment:1>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker