[FFmpeg-trac] #11541(avcodec:new): All NVENC encoders change bt709 color space to bt470bg when pixel format is not specified

[FFmpeg]

#11541: All NVENC encoders change bt709 color space to bt470bg when pixel format is
not specified
-------------------------------------+-------------------------------------
             Reporter:  fmnijk       |                     Type:  defect
               Status:  new          |                 Priority:  normal
            Component:  avcodec      |                  Version:  git-
             Keywords:  colorspace   |  master
  NVENC scale                        |               Blocked By:
             Blocking:               |  Reproduced by developer:  0
Analyzed by developer:  0            |
-------------------------------------+-------------------------------------
 When using any NVENC encoder (av1_nvenc, hevc_nvenc, h264_nvenc) with
 zscale or scale filter to set color space parameters, the color space gets
 incorrectly set to bt470bg instead of bt709 when not explicitly specifying
 a pixel format.

 ## Steps to reproduce:
 1. Use an EXR file with linear transfer characteristics
 2. Use zscale or scale to explicitly set matrix/color space to bt709
 3. Encode with any NVENC encoder without specifying pixel format

 ## Expected behavior:
 Output video should have color_space=bt709 as specified in the filter.

 ## Actual behavior:
 Output video has color_space=bt470bg despite explicitly setting
 matrix/color space to bt709.

 ## Problem examples:With zscale:
 ffmpeg -i input_%04d.exr -vf "zscale=matrix=bt709:transfer=iec61966-2-1"
 -c:v av1_nvenc output.mp4

 With scale:
 ffmpeg -i input_%04d.exr -vf
 "scale=out_color_matrix=bt709:out_transfer=iec61966-2-1" -c:v av1_nvenc
 output.mp4

 Output properties for both cases:
 "color_space": "bt470bg",  # Should be bt709
 "color_transfer": "iec61966-2-1"

 ## Working example (without NVENC):
 ffmpeg -i input_%04d.exr -vf "zscale=matrix=bt709:transfer=iec61966-2-1"
 output.mp4

 Output properties:
 "color_space": "bt709",
 "color_transfer": "iec61966-2-1"

 ## Workarounds:
 1. Adding any format filter to the chain fixes the issue:
 ffmpeg -i input_%04d.exr -vf
 "zscale=matrix=bt709:transfer=iec61966-2-1,format=yuv420p" -c:v av1_nvenc
 output.mp4

 2. Explicitly specifying pixel format as a parameter also works:
 ffmpeg -i input_%04d.exr -vf "zscale=matrix=bt709:transfer=iec61966-2-1"
 -pix_fmt yuv420p -c:v av1_nvenc output.mp4

 Either way correctly outputs:
 "color_space": "bt709",
 "color_transfer": "iec61966-2-1"

 ## System Information:
 ffmpeg version N-119165-gf3f1a48a07-20250407
 OS: Windows 10 22H2
 GPU: RTX 4090
 NVIDIA Driver: NVIDIA Studio Driver 572.83
-- 
Ticket URL: <https://trac.ffmpeg.org/ticket/11541>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker