[FFmpeg-trac] #9781(avfilter:new): zscale adds distortion when resampling video data

[FFmpeg]

#9781: zscale adds distortion when resampling video data
-------------------------------------+-------------------------------------
             Reporter:  Llyw         |                    Owner:  (none)
                 Type:  defect       |                   Status:  new
             Priority:  normal       |                Component:  avfilter
              Version:  git-master   |               Resolution:
             Keywords:  zscale       |               Blocked By:
  distortion                         |
             Blocking:               |  Reproduced by developer:  0
Analyzed by developer:  0            |
-------------------------------------+-------------------------------------
Changes (by Llyw):

 * cc: Llyw (added)


Old description:

> I noticed that the zscale filter can introduce distortion (by either
> cutting or crushing of the image, I have not been able to pinpoint the
> exact type of distortion yet) when resampling video input using
> [https://github.com/FFmpeg/FFmpeg/commit/30e2bb0f64 FFmpeg at 30e2bb0f64],
> but not when using FFmpeg 5.0.1:
>
> Take, for example, the 720p VP9 WebM version of Blender Foundation's
> Elephants Dream at
> https://upload.wikimedia.org/wikipedia/commons/transcoded/2/28/Elephants_Dream_%282006%29_1080p24.webm/Elephants_Dream_%282006%29_1080p24.webm.720p.vp9.webm.
>
> At roughly 7 minutes, 2 seconds a stills image of this video looks like
> this in VLC 3.0.17.4: (See first attachment)
>

> Then, resample to 480x272 pixels with a display aspect ratio of 16:9 and
> a sample aspect ratio of 136:135:
> {{{
> ffmpeg -i 'Elephants_Dream_(2006)_1080p24.webm.720p.vp9.webm' -vf
> 'zscale=w=480:h=272:f=lanczos,setdar=dar=16/9' -pix_fmt yuv420p
> Output_Distorted.mp4
> }}}
> The corresponding stills image will look like this when resizing the
> video port to show a 720p resolution: (See second attachment)
>
> I understand that there is some movement in this scene but if you look at
> the background you should clearly see the distortion I was talking about.
>

> Compare this with the same operation done using FFmpeg's `scale` filter:
> {{{
> ffmpeg -i 'Elephants_Dream_(2006)_1080p24.webm.720p.vp9.webm' -vf
> 'scale=w=480:h=272:flags=lanczos+accurate_rnd,setdar=dar=16/9' -pix_fmt
> yuv420p Output_Undistorted.mp4
> }}}
> Here, the corresponding stills image looks like this and does not show
> significant distortion when comparing with the stills image of the input
> video (some minor mismatch is obviously expected due to the strong
> resampling and movement in the scene): (See third attachment)
>
> I am assuming that this might have something to do with the recent-ish
> efforts to parallelize the `zscale` filter and would appreciate it, if
> this behavior is investigated.

New description:

 I noticed that the zscale filter can introduce distortion (by either
 cutting or crushing of the image, I have not been able to pinpoint the
 exact type of distortion yet) when resampling video input using
 [https://github.com/FFmpeg/FFmpeg/commit/30e2bb0f64 FFmpeg at 30e2bb0f64],
 but not when using FFmpeg 5.0.1:

 Take, for example, the 720p VP9 WebM version of Blender Foundation's
 Elephants Dream at
 https://upload.wikimedia.org/wikipedia/commons/transcoded/2/28/Elephants_Dream_%282006%29_1080p24.webm/Elephants_Dream_%282006%29_1080p24.webm.720p.vp9.webm.

 At roughly 7 minutes, 2 seconds a stills image of this video looks like
 this in VLC 3.0.17.4:
 [[Image(https://trac.ffmpeg.org/attachment/ticket/9781/Original.png)]]


 Then, resample to 480x272 pixels with a display aspect ratio of 16:9 and a
 sample aspect ratio of 136:135:
 {{{
 ffmpeg -i 'Elephants_Dream_(2006)_1080p24.webm.720p.vp9.webm' -vf
 'zscale=w=480:h=272:f=lanczos,setdar=dar=16/9' -pix_fmt yuv420p
 Output_Distorted.mp4
 }}}
 The corresponding stills image will look like this when resizing the video
 port to show a 720p resolution:
 [[Image(https://trac.ffmpeg.org/attachment/ticket/9781/Output_Distorted.png)]]

 I understand that there is some movement in this scene but if you look at
 the background you should clearly see the distortion I was talking about.


 Compare this with the same operation done using FFmpeg's `scale` filter:
 {{{
 ffmpeg -i 'Elephants_Dream_(2006)_1080p24.webm.720p.vp9.webm' -vf
 'scale=w=480:h=272:flags=lanczos+accurate_rnd,setdar=dar=16/9' -pix_fmt
 yuv420p Output_Undistorted.mp4
 }}}
 Here, the corresponding stills image looks like this and does not show
 significant distortion when comparing with the stills image of the input
 video (some minor mismatch is obviously expected due to the strong
 resampling and movement in the scene):
 [[Image(https://trac.ffmpeg.org/attachment/ticket/9781/Output_Undistorted.png)]]

 I am assuming that this might have something to do with the recent-ish
 efforts to parallelize the `zscale` filter and would appreciate it, if
 this behavior is investigated.

--
-- 
Ticket URL: <https://trac.ffmpeg.org/ticket/9781#comment:1>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker