[FFmpeg-trac] #5996(avfilter:new): Overlay output changes greatly if yuva420 instead of yuv420

[FFmpeg]

#5996: Overlay output changes greatly if yuva420 instead of yuv420
----------------------------------+--------------------------------------
             Reporter:  Misaki    |                     Type:  defect
               Status:  new       |                 Priority:  normal
            Component:  avfilter  |                  Version:  git-master
             Keywords:  overlay   |               Blocked By:
             Blocking:            |  Reproduced by developer:  0
Analyzed by developer:  0         |
----------------------------------+--------------------------------------
 I am using the most recent static git build. This particular bug has
 existed for a long time.

 A white-on-black overlay that has an alpha channel or plane will be very
 bright even for very low alpha values, if the output from the overlay
 filter is yuva420. If the output is yuv420, then the output seems to be
 proportional to the degree of transparency.

 How to replicate:

  ffmpeg -hide_banner -filter_complex
 "color=black:d=1[b],color=white:d=1[w],color=white:2x2:d=1,geq=255/24*N:128,scale=20:20,scale=320:240,[w]alphamerge,[b]overlay,format=yuva420p,showinfo"
 -f null -

 The scale then alphamerge is intended to allow variable transparency in a
 frame in a way that's faster than using the blend filter with expressions,
 and much faster than using the geq filter on a full-sized frame (unless
 there's been optimization in the past year).

 In this case, yuva420p is the default output for the previous filters. If
 you change the output to an mp4 file, the default output will change to
 yuv420p since H.264 doesn't have an alpha channel. But putting yuva420p
 before the showinfo, and yuv420p after it, does result in the same
 problem.

 The first few lines of output for showinfo:

 [Parsed_showinfo_9 @ 0xa79dc0] n:   0 pts:      0 pts_time:0       pos:
 -1 fmt:yuva420p sar:1/1 s:320x240 i:P iskey:1 type:I checksum:02EDAAF8
 plane_checksum:[35B2C10E 2BAB822B 2BAB822B 77F0E576] mean:[16 128 128 255]
 stdev:[0.0 0.0 0.0 0.0]
 [null @ 0xa7f860] Encoder did not produce proper pts, making some up.
 [Parsed_showinfo_9 @ 0xa79dc0] n:   1 pts:      1 pts_time:0.04    pos:
 -1 fmt:yuva420p sar:1/1 s:320x240 i:P iskey:1 type:I checksum:02EDAAF8
 plane_checksum:[35B2C10E 2BAB822B 2BAB822B 77F0E576] mean:[16 128 128 255]
 stdev:[0.0 0.0 0.0 0.0]
 [Parsed_showinfo_9 @ 0xa79dc0] n:   2 pts:      2 pts_time:0.08    pos:
 -1 fmt:yuva420p sar:1/1 s:320x240 i:P iskey:1 type:I checksum:AA52FF12
 plane_checksum:[7F151537 2BAB822B 2BAB822B 77F0E576] mean:[76 128 128 255]
 stdev:[0.0 0.0 0.0 0.0]
 [Parsed_showinfo_9 @ 0xa79dc0] n:   3 pts:      3 pts_time:0.12    pos:
 -1 fmt:yuva420p sar:1/1 s:320x240 i:P iskey:1 type:I checksum:247DA2F0
 plane_checksum:[3B13B906 2BAB822B 2BAB822B 77F0E576] mean:[132 128 128
 255] stdev:[0.0 0.0 0.0 0.0]
 [Parsed_showinfo_9 @ 0xa79dc0] n:   4 pts:      4 pts_time:0.16    pos:
 -1 fmt:yuva420p sar:1/1 s:320x240 i:P iskey:1 type:I checksum:5A2A5139
 plane_checksum:[89D4674F 2BAB822B 2BAB822B 77F0E576] mean:[165 128 128
 255] stdev:[0.0 0.0 0.0 0.0]
 [Parsed_showinfo_9 @ 0xa79dc0] n:   5 pts:      5 pts_time:0.2     pos:
 -1 fmt:yuva420p sar:1/1 s:320x240 i:P iskey:1 type:I checksum:A5FA6A74
 plane_checksum:[8640808A 2BAB822B 2BAB822B 77F0E576] mean:[183 128 128
 255] stdev:[0.0 0.0 0.0 0.0]
 [Parsed_showinfo_9 @ 0xa79dc0] n:   6 pts:      6 pts_time:0.24    pos:
 -1 fmt:yuva420p sar:1/1 s:320x240 i:P iskey:1 type:I checksum:628E4F37
 plane_checksum:[4B30654D 2BAB822B 2BAB822B 77F0E576] mean:[194 128 128
 255] stdev:[0.0 0.0 0.0 0.0]

 Luminance goes 16, 16 (since the color filter had mpeg range of 16~235 for
 luminance, and it was converted to full-range gray after the second scale
 so the 10 went to 0), then 76, 132, 165, 183, 194.

 If you change the yuva420p to yuv420p, then . . . -v debug (or trace)
 doesn't actually show any changes because overlay filter doesn't report
 what its output is, but showinfo shows that the luminance has changed to
 16, 16, 21, 31, 42, 53, 63, 74, . . .

 If you switch the colours so white is on the bottom and black is on top
 with an alpha channel, then the outputs as yuva and yuv are almost, but in
 fact not exactly the same as each other. The last values for yuva420p
 output are . . . 99, 89, 78, 68, 57, 49, 38, 27, 18, 16, 16. For yuv420p
 output, it's . . . 103, 92, 81, 71, 60, 50, 39, 28, 18, 16, 16.

 Other pixel formats may be bugged in different ways. The most significant
 effect I have seen is that the fade filter, when set to affect
 transparency, has this rather abrupt drop-off or onset of brightness from
 bright-over-dark. It might all be due to how the overlay filter works. If
 the intention was for the layers to combine 'non-linearly' due to gamma,
 then 'black over white' overlay should also be mostly bright, instead of
 having a steady progression.

--
Ticket URL: <https://trac.ffmpeg.org/ticket/5996>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker