[FFmpeg-trac] #9781(avfilter:reopened): zscale adds distortion when resampling video data

[FFmpeg]

#9781: zscale adds distortion when resampling video data
-------------------------------------+-------------------------------------
             Reporter:  Llyw         |                    Owner:  (none)
                 Type:  defect       |                   Status:  reopened
             Priority:  normal       |                Component:  avfilter
              Version:  git-master   |               Resolution:
             Keywords:  zscale       |               Blocked By:
  distortion                         |
             Blocking:               |  Reproduced by developer:  0
Analyzed by developer:  0            |
-------------------------------------+-------------------------------------
Changes (by Llyw):

 * status:  closed => reopened
 * resolution:  fixed =>


Old description:

> I noticed that the zscale filter can introduce distortion (by either
> cutting or crushing of the image, I have not been able to pinpoint the
> exact type of distortion yet) when resampling video input using
> [https://github.com/FFmpeg/FFmpeg/commit/30e2bb0f64 FFmpeg at 30e2bb0f64],
> but not when using FFmpeg 5.0.1:
>
> Take, for example, the 720p VP9 WebM version of Blender Foundation's
> Elephants Dream at
> https://upload.wikimedia.org/wikipedia/commons/transcoded/2/28/Elephants_Dream_%282006%29_1080p24.webm/Elephants_Dream_%282006%29_1080p24.webm.720p.vp9.webm.
>
> At roughly 7 minutes, 2 seconds a stills image of this video looks like
> this in VLC 3.0.17.4:
> [[Image(https://trac.ffmpeg.org/attachment/ticket/9781/Original.png)]]
>

> Then, resample to 480x272 pixels with a display aspect ratio of 16:9 and
> a sample aspect ratio of 136:135:
> {{{
> ffmpeg -i 'Elephants_Dream_(2006)_1080p24.webm.720p.vp9.webm' -vf
> 'zscale=w=480:h=272:f=lanczos,setdar=dar=16/9' -pix_fmt yuv420p
> Output_Distorted.mp4
> }}}
> The corresponding stills image will look like this when resizing the
> video port to show a 720p resolution:
> [[Image(https://trac.ffmpeg.org/attachment/ticket/9781/Output_Distorted.png)]]
>
> I understand that there is some movement in this scene but if you look at
> the background you should clearly see the distortion I was talking about.
>

> Compare this with the same operation done using FFmpeg's `scale` filter:
> {{{
> ffmpeg -i 'Elephants_Dream_(2006)_1080p24.webm.720p.vp9.webm' -vf
> 'scale=w=480:h=272:flags=lanczos+accurate_rnd,setdar=dar=16/9' -pix_fmt
> yuv420p Output_Undistorted.mp4
> }}}
> Here, the corresponding stills image looks like this and does not show
> significant distortion when comparing with the stills image of the input
> video (some minor mismatch is obviously expected due to the strong
> resampling and movement in the scene):
> [[Image(https://trac.ffmpeg.org/attachment/ticket/9781/Output_Undistorted.png)]]
>
> I am assuming that this might have something to do with the recent-ish
> efforts to parallelize the `zscale` filter and would appreciate it, if
> this behavior is investigated.

New description:

 Agreed. (I can understand a desire to close new issues as fast as
 possible, but I don't think that it's the right way forward in this case.)

 I think the actual root-cause for the distortion is the fact that the
 coordinates for the active area in the input slice is approximated using
 integer arithmetic, whereas the z.lib/zimg API actually stores these
 coordinates as double precision floating-point values, see
 https://github.com/sekrit-
 twc/zimg/blob/release-3.0.4/src/zimg/api/zimg.h#L568.

 So it should be possible to compute and assign these coordinates (without
 any integer/rounding operations) in double precision floating-point
 arithmetic.

--
-- 
Ticket URL: <https://trac.ffmpeg.org/ticket/9781#comment:8>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker