[FFmpeg-trac] #9576(avfilter:new): scale ignores AVFrame's input range

[FFmpeg]

#9576: scale ignores AVFrame's input range
-------------------------------------+-------------------------------------
             Reporter:  Diederick    |                     Type:  defect
  Niehorster                         |
               Status:  new          |                 Priority:  normal
            Component:  avfilter     |                  Version:  4.4.1
             Keywords:  scale        |               Blocked By:
  color_range                        |
             Blocking:               |  Reproduced by developer:  0
Analyzed by developer:  0            |
-------------------------------------+-------------------------------------
 Discovered when investigating #9573.

 When reading a file with yuv420p10 grayscale data in full range
 (color_range=2) and requesting conversion to gray10, conversion does not
 use full range. You need to tell the scale filter to assume the input is
 full range with in_range=pc, despite the AVFrames being delivered to the
 scale_frame() function of the filter having color_range=2 (full/pc/JPEG
 range set).

 Transcode command:
 {{{
 ffmpeg -y -i test3.mp4 -vf format=gray10 -c:v ffv1 -level 3 test2.mkv
 }}}
 If using
 {{{
 ffmpeg -y -i test3.mp4 -vf scale=in_range=pc,format=gray10 -c:v ffv1
 -level 3 test2.mkv
 }}}
 everything is fine.

 Command to see if video encoded fine:
 {{{
 ffmpeg -i test3.mp4 -i test2.mkv -lavfi
 "[0:v]extractplanes=y,settb=1/AVTB,setpts=PTS-
 STARTPTS[main];[1:v]settb=1/AVTB,setpts=PTS-STARTPTS[ref];[main][ref]psnr"
 -f null -
 }}}
 which yields PSNR y:26.424073 for the first command, and PSNR y:inf for
 the second.

 I don't quite understand why in_range=pc is needed. when looking at line
 802 of vf_scale.c, it does detect that the input is full range (i stepped
 through the code, it does, in_full is set to 1), and this seems to be set
 on the scale context in lines 809-816, but apparently it doesn't do
 anything.

 I'm not sure if this is a defect or an enhancement request, but i think
 since all the info is there and since sws_setColorspaceDetails() on lines
 809-816 doesn't seem to do what it should (set input range of the scaler
 to full), this is a defect.

 NB: out_range doesn't need to be set as its set to full range
 automatically for gray10 input due to handle_jpeg() in utils.c of
 libswscale
-- 
Ticket URL: <https://trac.ffmpeg.org/ticket/9576>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker