[FFmpeg-trac] #9781(avfilter:closed): zscale adds distortion when resampling video data

[FFmpeg]

#9781: zscale adds distortion when resampling video data
-------------------------------------+-------------------------------------
             Reporter:  Llyw         |                    Owner:  (none)
                 Type:  defect       |                   Status:  closed
             Priority:  normal       |                Component:  avfilter
              Version:  git-master   |               Resolution:  fixed
             Keywords:  zscale       |               Blocked By:
  distortion                         |
             Blocking:               |  Reproduced by developer:  0
Analyzed by developer:  0            |
-------------------------------------+-------------------------------------

Old description:

> Agreed. (I can understand a desire to close new issues as fast as
> possible, but I don't think that it's the right way forward in this
> case.)
>
> I think the actual root-cause for the distortion is the fact that the
> coordinates for the active area in the input slice is approximated using
> integer arithmetic, whereas the z.lib/zimg API actually stores these
> coordinates as double precision floating-point values, see
> https://github.com/sekrit-
> twc/zimg/blob/release-3.0.4/src/zimg/api/zimg.h#L568.
>
> So it should be possible to compute and assign these coordinates (without
> any integer/rounding operations) in double precision floating-point
> arithmetic.

New description:

 I noticed that the zscale filter can introduce distortion (by either
 cutting or crushing of the image, I have not been able to pinpoint the
 exact type of distortion yet) when resampling video input using
 [https://github.com/FFmpeg/FFmpeg/commit/30e2bb0f64 FFmpeg at 30e2bb0f64],
 but not when using FFmpeg 5.0.1:

 Take, for example, the 720p VP9 WebM version of Blender Foundation's
 Elephants Dream at
 https://upload.wikimedia.org/wikipedia/commons/transcoded/2/28/Elephants_Dream_%282006%29_1080p24.webm/Elephants_Dream_%282006%29_1080p24.webm.720p.vp9.webm.

 At roughly 7 minutes, 2 seconds a stills image of this video looks like
 this in VLC 3.0.17.4:
 [[Image(https://trac.ffmpeg.org/attachment/ticket/9781/Original.png)]]


 Then, resample to 480x272 pixels with a display aspect ratio of 16:9 and a
 sample aspect ratio of 136:135:
 {{{
 ffmpeg -i 'Elephants_Dream_(2006)_1080p24.webm.720p.vp9.webm' -vf
 'zscale=w=480:h=272:f=lanczos,setdar=dar=16/9' -pix_fmt yuv420p
 Output_Distorted.mp4
 }}}
 The corresponding stills image will look like this when resizing the video
 port to show a 720p resolution:
 [[Image(https://trac.ffmpeg.org/attachment/ticket/9781/Output_Distorted.png)]]

 I understand that there is some movement in this scene but if you look at
 the background you should clearly see the distortion I was talking about.


 Compare this with the same operation done using FFmpeg's `scale` filter:
 {{{
 ffmpeg -i 'Elephants_Dream_(2006)_1080p24.webm.720p.vp9.webm' -vf
 'scale=w=480:h=272:flags=lanczos+accurate_rnd,setdar=dar=16/9' -pix_fmt
 yuv420p Output_Undistorted.mp4
 }}}
 Here, the corresponding stills image looks like this and does not show
 significant distortion when comparing with the stills image of the input
 video (some minor mismatch is obviously expected due to the strong
 resampling and movement in the scene):
 [[Image(https://trac.ffmpeg.org/attachment/ticket/9781/Output_Undistorted.png)]]

 I am assuming that this might have something to do with the recent-ish
 efforts to parallelize the `zscale` filter and would appreciate it, if
 this behavior is investigated.

--
Comment (by Llyw):

 Agreed. (I can understand a desire to close new issues as fast as
 possible, but I don't think that it's the right way forward in this case.
 The relevant code in the z.lib/zimg API showing that it is possible to
 compute the input slice's active area precisely is at ​https://github.com
 /sekrit-twc/zimg/blob/release-3.0.4/src/zimg/api/zimg.h#L568)
-- 
Ticket URL: <https://trac.ffmpeg.org/ticket/9781#comment:10>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker