[FFmpeg-trac] #9573(avcodec:new): libx264 ignores color range flag for gray10 input

[FFmpeg]

#9573: libx264 ignores color range flag for gray10 input
-------------------------------------+-------------------------------------
             Reporter:  Diederick    |                    Owner:  (none)
  Niehorster                         |
                 Type:  defect       |                   Status:  new
             Priority:  normal       |                Component:  avcodec
              Version:  git-master   |               Resolution:
             Keywords:  libx264      |               Blocked By:
  gray10                             |
             Blocking:               |  Reproduced by developer:  0
Analyzed by developer:  0            |
-------------------------------------+-------------------------------------
Comment (by pdr0):

 Replying to [comment:4 Diederick Niehorster]:

 > 1. libx264 with full range gray10 input seems to ignore the full range
 flag (also when adding a filter -vf scale=in_range=pc:out_range=pc to the
 command, i just tried)
 > 2. libx264 with crf 0 and pix_fmt yuv420p10le is not lossless, but shows
 many pixels that are off by up to 5 intensity values (should this be
 reported separately?)


 test2.mp4 (and test3.mp4) has full range flag - ffmpeg yuv420p10le(pc) .
 Mediainfo also specifies full range flag.

 It looks like something changed with libx264; -qp 0 is lossless, but -crf
 0 is not

 Even the filesize is almost 3x larger with -qp 0 vs. -crf 0

 test3.mp4 is lossless, psnr inf

 {{{
 ffmpeg -y -i test.mkv -vf
 "scale=in_range=pc:out_range=pc,format=yuv420p10" -c:v libx264 -preset
 veryfast -qp 0 test3.mp4 -y
 }}}

 {{{
 ffmpeg -i test3.mp4 -i test.mkv -lavfi
 "[0:v]scale=in_range=pc:out_range=pc,format=gray10le,settb=1/AVTB,setpts
 =PTS-STARTPTS[main];[1:v]settb=1/AVTB,setpts=PTS-
 STARTPTS[ref];[main][ref]psnr" -f null -
 }}}

 [Parsed_psnr_6 @ 000000da162e95c0] PSNR y:inf average:inf min:inf max:inf

 If you are using ffplay, you need to force full range rgb or gbrp playback
 with swscale or zscale to be consistent. It doesn't necessarily
 automatically acknowlege the full range flag for the RGB playback
 conversion

 {{{
 ffplay -vf "scale=in_range=pc, format=gbrp" -i test3.mp4
 }}}
-- 
Ticket URL: <https://trac.ffmpeg.org/ticket/9573#comment:7>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker