Unexpected Color Conversion (bgr->yuv vs rgb->yuv); first is bad, bmp affected

[natt]

Input file is bgr24. Output file is yuv420p. If I instruct ffmpeg to convert to rgb24 first (instead of bgr24->yuv420p directly), the colors are slightly different.

Test input, exact command lines, and console output attached.

[natt]

output of testcase executable

[Elon Musk]

Whatever issue you are reporting I can not reproduce it.
rgb24(->bgr24)->yuv is fine here

[natt]

Replying to richardpl:

Whatever issue you are reporting I can not reproduce it.
rgb24(->bgr24)->yuv is fine here

The two conversions should be identical, right? Since rgb24<->bgr24 is lossless, bgr->rgb->yuv should produce the same output as bgr->yuv. If it doesn't, it's not working right.

ffmpeg -f rawvideo -video_size 256x256 -pix_fmt bgr24 -i testin.raw -sws_flags lanczos -f rawvideo -pix_fmt yuv420p md5:
0ba2a7494930baaa64869462d54b85cc

ffmpeg -f rawvideo -video_size 256x256 -pix_fmt bgr24 -i testin.raw -filter format=rgb24 -sws_flags lanczos -f rawvideo -pix_fmt yuv420p md5:
d5bd2d94675b67d23eb2661ca462966d

[Carl Eugen Hoyos]

If there is an issue, is is reproducible with this command:

$ ./ffmpeg -i tests/lena.pnm -pix_fmt yuv420p -f crc - -vf format=bgr24 -pix_fmt yuv420p -f crc -
ffmpeg version N-37568-g8162c6f Copyright (c) 2000-2012 the FFmpeg developers
  built on Feb  5 2012 16:31:32 with gcc 4.5.3
  configuration: --cc='/usr/local/gcc-4.5.3/bin/gcc -m32'
  libavutil      51. 37.100 / 51. 37.100
  libavcodec     54.  0.102 / 54.  0.102
  libavformat    54.  0.100 / 54.  0.100
  libavdevice    53.  4.100 / 53.  4.100
  libavfilter     2. 61.100 /  2. 61.100
  libswscale      2.  1.100 /  2.  1.100
  libswresample   0.  6.100 /  0.  6.100
Input #0, image2, from 'tests/lena.pnm':
  Duration: 00:00:00.04, start: 0.000000, bitrate: N/A
    Stream #0:0: Video: ppm, rgb24, 256x256, 25 tbr, 25 tbn, 25 tbc
[buffer @ 0x8dd20a0] w:256 h:256 pixfmt:rgb24 tb:1/1000000 sar:0/1 sws_param:
[buffersink @ 0x8dd2360] auto-inserting filter 'auto-inserted scale 0' between the filter 'src' and the filter 'out'
[scale @ 0x8dd28e0] w:256 h:256 fmt:rgb24 -> w:256 h:256 fmt:yuv420p flags:0x4
[buffer @ 0x8de20e0] w:256 h:256 pixfmt:rgb24 tb:1/1000000 sar:0/1 sws_param:
[buffersink @ 0x8de1780] auto-inserting filter 'auto-inserted scale 1' between the filter 'Parsed_format_0' and the filter 'out'
[format @ 0x8de0a80] auto-inserting filter 'auto-inserted scale 2' between the filter 'src' and the filter 'Parsed_format_0'
[scale @ 0x8df3960] w:256 h:256 fmt:rgb24 -> w:256 h:256 fmt:bgr24 flags:0x4
[scale @ 0x8de0b40] w:256 h:256 fmt:bgr24 -> w:256 h:256 fmt:yuv420p flags:0x4
Output #0, crc, to 'pipe:':
  Metadata:
    encoder         : Lavf54.0.100
    Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 256x256, q=2-31, 200 kb/s, 90k tbn, 25 tbc
Output #1, crc, to 'pipe:':
  Metadata:
    encoder         : Lavf54.0.100
    Stream #1:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 256x256, q=2-31, 200 kb/s, 90k tbn, 25 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (ppm -> rawvideo)
  Stream #0:0 -> #1:0 (ppm -> rawvideo)
Press [q] to stop, [?] for help
CRC=0x7db3d981
CRC=0x74eb2924
frame=    1 fps=  0 q=0.0 Lq=0.0 size=       0kB time=00:00:00.04 bitrate=   3.0kbits/s
video:192kB audio:0kB global headers:0kB muxing overhead -99.992371%

[natt]

Replying to cehoyos:

If there is an issue, is is reproducible with this command:

Yes, I get exactly that.

>ffmpeg -i lena.pnm -pix_fmt yuv420p -f crc - -vf format=bgr24
-pix_fmt yuv420p -f crc -
ffmpeg version N-37541-g670229e Copyright (c) 2000-2012 the FFmpeg developers
  built on Feb  3 2012 20:07:47 with gcc 4.6.2
  configuration: --enable-gpl --enable-version3 --disable-w32threads --enable-ru
ntime-cpudetect --enable-avisynth --enable-bzlib --enable-frei0r --enable-libope
ncore-amrnb --enable-libopencore-amrwb --enable-libfreetype --enable-libgsm --en
able-libmp3lame --enable-libopenjpeg --enable-librtmp --enable-libschroedinger -
-enable-libspeex --enable-libtheora --enable-libvo-aacenc --enable-libvo-amrwben
c --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libxavs --enable-
libxvid --enable-zlib
  libavutil      51. 37.100 / 51. 37.100
  libavcodec     54.  0.102 / 54.  0.102
  libavformat    54.  0.100 / 54.  0.100
  libavdevice    53.  4.100 / 53.  4.100
  libavfilter     2. 61.100 /  2. 61.100
  libswscale      2.  1.100 /  2.  1.100
  libswresample   0.  6.100 /  0.  6.100
  libpostproc    52.  0.100 / 52.  0.100
Input #0, image2, from 'lena.pnm':
  Duration: 00:00:00.04, start: 0.000000, bitrate: N/A
    Stream #0:0: Video: ppm, rgb24, 256x256, 25 tbr, 25 tbn, 25 tbc
[buffer @ 003DA880] w:256 h:256 pixfmt:rgb24 tb:1/1000000 sar:0/1 sws_param:
[buffersink @ 01B9D2A0] auto-inserting filter 'auto-inserted scale 0' between th
e filter 'src' and the filter 'out'
[scale @ 01B9D680] w:256 h:256 fmt:rgb24 -> w:256 h:256 fmt:yuv420p flags:0x4
[buffer @ 01B9DB20] w:256 h:256 pixfmt:rgb24 tb:1/1000000 sar:0/1 sws_param:
[buffersink @ 01C1CCA0] auto-inserting filter 'auto-inserted scale 1' between th
e filter 'Parsed_format_0' and the filter 'out'
[format @ 01C1BE40] auto-inserting filter 'auto-inserted scale 2' between the fi
lter 'src' and the filter 'Parsed_format_0'
[scale @ 01C1AAA0] w:256 h:256 fmt:rgb24 -> w:256 h:256 fmt:bgr24 flags:0x4
[scale @ 01C1BDE0] w:256 h:256 fmt:bgr24 -> w:256 h:256 fmt:yuv420p flags:0x4
Output #0, crc, to 'pipe:':
  Metadata:
    encoder         : Lavf54.0.100
    Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 256x256, q=2-31,
200 kb/s, 90k tbn, 25 tbc
Output #1, crc, to 'pipe:':
  Metadata:
    encoder         : Lavf54.0.100
    Stream #1:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 256x256, q=2-31,
200 kb/s, 90k tbn, 25 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (ppm -> rawvideo)
  Stream #0:0 -> #1:0 (ppm -> rawvideo)
Press [q] to stop, [?] for help
CRC=0x7db3d981
CRC=0x74eb2924
frame=    1 fps=  0 q=0.0 Lq=0.0 size=       0kB time=00:00:00.04 bitrate=   3.0
kbits/s
video:192kB audio:0kB global headers:0kB muxing overhead -99.992371%

[Roger Pack]

anybody have any clues on this one?

[fireattack]

So I assume pix_fmt routine is part of swscale and the problem is only limited to that?

The same problem doesn't seem to happen if format conversion is done in colorspace.

Test:

ffmpeg -i reference.pnm -vf colorspace=iall=bt601-6-625:all=bt709:format=yuv420p -f crc - -vf format=bgr24,colorspace=iall=bt601-6-625:all=bt709:format=yuv420p -f crc -

[fireattack]

Just some additional info about the impact of this issue. It seems to affect quite a lot downstream software from my observation lately.

I've noticed this "color shifted to yellow" issue in many recording utilities and programs, a noble one being nVidia's ShadowPlay.

Hopefully someone could find the root cause and fix this long standing issue.

[gdgsdg123]

So... the problem can be concluded as -pix_fmt interfering with the format video filter?

Or will ​this be related?.. (one of the swscale hazards)

[fireattack]

They're not interfering anything, -pix_fmt, filter format all have the same discoloration bug as soon as you are converting bgr24->yuv (but NOT when rgb24->yuv, so you can work around by (losslessly) converting to rgb24 first).

format argument in colorspace filter (NOT format filter, two different things), however, does NOT have this issue.

Summary (I tested with additional bt601 to bt709 conversion, but the principle is the same):

-vf scale=out_color_matrix=bt709,format=yuv420p BAD
-vf scale=out_color_matrix=bt709 -pix_fmt yuv420p BAD (-pix_fmt is basically same as format filter from my testing. So you can replace one to the other below)

-vf colormatrix=bt601:bt709,format=yuv420p BAD

-vf colorspace=iall=bt601-6-625:all=bt709:format=yuv420p GOOD (this is using format option in colorspace filter)
-vf colorspace=iall=bt601-6-625:all=bt709,format=yuv420p BAD (this is to chain a format filter)
-vf zscale=matrix=709,format=yuv420p GOOD

[Balling]

This is fixed by -vf scale=flags=accurate_rnd (https://ffmpeg.org/ffmpeg-all.html#Scaler-Options), or more accurately accurate_rnd+whatever algorithm you want. WRONG documentation, BTW, you can use them together.

255, 255, 255 BGR value is converted to (as can be checked by ffplay -i file -vf extractplanes=y) YCbCr values 234, 127, 127 (instead of good 235, 128, 128). So it is just an off-by-one somewhere. And no, it is not out-of-gamut. I suppose the bug is indeed somewhere in second line about special "unscaled" converter, whatever that means (you can print those with +print_info):

[swscaler @ 00000275e7aa00c0] bicubic scaler, from bgr24 to yuv420p using MMXEXT
[swscaler @ 00000275e7aa00c0] using unscaled bgr24 -> yuv420p special converter

as there is no this second line with print_info+accurate_rnd.

What I am concerned with are JPEGs: they are also affected (?), need to check RGB one.

Also see duplicate #8056: bmp is affected too, since it uses BGR. (DPX can be affected too.)

I also clarified what is the bad one: bgr, not rgb.

P.S. -vf scale=out_range=pc is not affected.
P.P.S. If accurate_rnd is bit perfect, we just need to switch as default for only bgr formats. Simple as that.