FRAPS Video Color Space

[Dennis]

The color space that ffmpeg uses on FRAPS videos seems to be wrong or mishandled.

First, running ffplay on a video (color space detected as yuvj420p), the output looks washed out (too white).
Second, when transcoding a video to H264 (which keeps the color space as yuvj420p) the output looks too dark. Forcing the color space to yuv420p, the output looks fine.
Third, when transcoding a video to MPEG2 (which changes the colorspace to yuv420p), the output looks fine.

Here is how ffmpeg detects the original file:

ffmpeg -i "C:\Fraps\Movies\cod2sp_s 2012-08-12 13-39-04-15.avi"
ffmpeg version N-43418-g633b90c Copyright (c) 2000-2012 the FFmpeg developers
  built on Aug  9 2012 23:56:26 with gcc 4.7.1 (GCC)
  configuration: --enable-gpl --enable-version3 --disable-pthreads --enable-runt
ime-cpudetect --enable-avisynth --enable-bzlib --enable-frei0r --enable-libass -
-enable-libcelt --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-l
ibfreetype --enable-libgsm --enable-libmp3lame --enable-libnut --enable-libopenj
peg --enable-librtmp --enable-libschroedinger --enable-libspeex --enable-libtheo
ra --enable-libutvideo --enable-libvo-aacenc --enable-libvo-amrwbenc --enable-li
bvorbis --enable-libvpx --enable-libx264 --enable-libxavs --enable-libxvid --ena
ble-zlib
  libavutil      51. 67.100 / 51. 67.100
  libavcodec     54. 51.100 / 54. 51.100
  libavformat    54. 22.104 / 54. 22.104
  libavdevice    54.  2.100 / 54.  2.100
  libavfilter     3.  7.100 /  3.  7.100
  libswscale      2.  1.101 /  2.  1.101
  libswresample   0. 15.100 /  0. 15.100
  libpostproc    52.  0.100 / 52.  0.100
[avi @ 000000000217aa20] non-interleaved AVI
Guessed Channel Layout for  Input Stream #0.1 : stereo
Input #0, avi, from 'C:\Fraps\Movies\orig.avi':
  Duration: 00:07:26.85, start: 0.000000, bitrate: 530091 kb/s
    Stream #0:0: Video: fraps (FPS1 / 0x31535046), yuvj420p, 1920x1080, 60 tbr,
60 tbn, 60 tbc
    Stream #0:1: Audio: pcm_s16le ([1][0][0][0] / 0x0001), 48000 Hz, stereo, s16
, 1536 kb/s
At least one output file must be specified

Let me know what other info you need.
Thanks.

[Carl Eugen Hoyos]

A sample will be needed and is the problem also reproducible if you encode to jpeg?

[Dennis]

Here is a four (4) second sample (289MB):
​http://www.djxmmx.net/ffmpeg/1636/cod2-credits-4sec.avi
MD5: d1426a8809acd5d8ff9e115a1b213f4a
SHA1: d0f0eb3384549bcf66f3ac996a0ad3a431f4331d

The color space is not 100% correct when encoded to JPEG.
It's close but not 100%.

Reference these files for a visual comparison only (not correct colorspace):
This is the original (rendered by WMPlayer with official FRAPS codec):
​http://www.djxmmx.net/ffmpeg/1636/orig.bmp

This is a JPEG rendered by FFmpeg:
​http://www.djxmmx.net/ffmpeg/1636/jpeg.jpg

This is an H264 encoded by FFmpeg, rendered by WMPlayer:
​http://www.djxmmx.net/ffmpeg/1636/h264.bmp

This is an MPEG2 encoded by FFmpeg, rendered by WMPlayer:
​http://www.djxmmx.net/ffmpeg/1636/mpeg2.bmp

[Dennis]

Doing some research on the Internet, I've found that FRAPS can use one of two color spaces (not sure how it chooses) for recording. These are YV12 or RGB24, I believe, so maybe FFmpeg is not equipped to detect both kinds of FRAPS files.

[Carl Eugen Hoyos]

Replying to DJX:

Doing some research on the Internet, I've found that FRAPS can use one of two color spaces (not sure how it chooses) for recording. These are YV12 or RGB24, I believe, so maybe FFmpeg is not equipped to detect both kinds of FRAPS files.

FFmpeg supports both RGB and YV12 in FRAPS, the problem you see happens only with YV12 (which can be interpreted in - subtly - different ways).

[Elon Musk]

This is not fraps decoder bug. Instead this is ffplay issue (which video output is quite limited) and colorspace handling/detection bug which is reported multiply times in several other bug reports.

[Carl Eugen Hoyos]

Could you point me to one of those tickets so I can connect them?

[Elon Musk]

The only valid issue may be that decoder use YUVJ when it actually should use YUV. Only way to find out is inspecting pixel values in decoded output.

[Carl Eugen Hoyos]

Replying to richardpl:

The only valid issue may be that decoder use YUVJ when it actaully should use YUV. Only way to find out is inspecting pixel values in decoded output.

So this ticket should be reopened until further inspection?

[Dennis]

This issue is more then FFplay.
I found the issue while using FFmpeg.

The main issue is when converting these videos with FFmpeg the color space is wrong.
FFmpeg (Auto detect color space) --> Libx264 (Auto detect color space) = Dark video output

If you wish to inspect the correct decoding values, the FRAPS codec is free with the trial version and allows you to decode these on Windows.

[Elon Musk]

Please read what I write.

[Elon Musk]

Also you should note that your reference is bmp which is rgb and ffmpeg output it jpeg which is yuv.
Comparing rgb with yuv is invalid. Trial Fraps codecs converts yuv to rgb.

[Dennis]

Yes, you also mentioned the YUVJ issue but I want to highlight that this in the main issue and is valid.

My screen shots are for visual reference only.
I just used BMP b/c I know it's lossless and it is the default in Windows.
I do not know much about color spaces.
I did not know BMP is RGB.

I'm not an expert on these matters at all, I'm just trying to help the project.

[Elon Musk]

I was wrong in such sense that yuvj vs yuv should not matter. The fraps sample you uploaded appears to make use of all bytes 0-255, so color-range should be set and pixel format switched from YUVJ (which is deprecated) to YUV.

[Elon Musk]

When transcoding your sample to png/bmp output is similar to your orig.bmp one but not same. So I really see no obvious bug here.
It is just that windows app internally converts to rgb. Question which conversion to bmp is closer to original is pointless because both are just wrong. So just use rgb with fraps.

[Dennis]

I really don't know any better way to show the official FRAPS codec rendering other than taking a screen shot in WMP and saving it in MSPaint.
That is why I did that but once again, it was mainly for a visual compare, not a literal compare.

I cannot use RGB in FRAPS all the time b/c it is slower.
...I assume you are referring to the Force RGB capture option in FRAPS.

[Elon Musk]

Sample use JPEG color range and that is reason why you get wrong colors when you view them with ffplay.

Please do not use ffplay when comparing, It outputs only yuv420 (rgb will look shitty) and ignores color ranges, and etc...

If I use more relevant application to display rgb images which are transcoded from fraps sample they are similar to your orig.bmp.

[Dennis]

Could you please try one more test.

Play cod2-credits-4sec.avi in WMP with official FRAPS binary codec.
Observe picture.

Convert cod2-credits-4sec.avi to H264 with FFmpeg
ffmpeg -i "cod2-credits-4sec.avi" "Out.mp4"

Play Out.mp4 in WMP
Observe picture.

To me, they are not the same.
Tested with N-49352-gc46943e.

[Elon Musk]

Thing is that fraps use YUVJ (full range) and whatever is stored in mp4 is using YUV (or YUVJ but program that you use to view it is not supporting it properly) and there is missing step to change colorspace. (Could use colormatrix filter) This is could be duplicate ticked (already reported in another form).

[Carl Eugen Hoyos]

Is this still reproducible after today's change?

[Dennis]

Affirmative, issue persists: transcoded output is still darker then original in Windows Media Player.

ffmpeg -i cod2-credits-4sec.avi c:\temp\test.mp4
ffmpeg version N-56575-g21f6ff3 Copyright (c) 2000-2013 the FFmpeg developers
  built on Sep 22 2013 18:08:30 with gcc 4.7.3 (GCC)
  configuration: --enable-gpl --enable-version3 --disable-w32threads --enable-av
isynth --enable-bzlib --enable-fontconfig --enable-frei0r --enable-gnutls --enab
le-iconv --enable-libass --enable-libbluray --enable-libcaca --enable-libfreetyp
e --enable-libgsm --enable-libilbc --enable-libmodplug --enable-libmp3lame --ena
ble-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-l
ibopus --enable-librtmp --enable-libschroedinger --enable-libsoxr --enable-libsp
eex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvo-aa
cenc --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libwavp
ack --enable-libx264 --enable-libxavs --enable-libxvid --enable-zlib
  libavutil      52. 46.100 / 52. 46.100
  libavcodec     55. 33.100 / 55. 33.100
  libavformat    55. 18.102 / 55. 18.102
  libavdevice    55.  3.100 / 55.  3.100
  libavfilter     3. 87.100 /  3. 87.100
  libswscale      2.  5.100 /  2.  5.100
  libswresample   0. 17.103 /  0. 17.103
  libpostproc    52.  3.100 / 52.  3.100
[avi @ 00000000028b7100] Stream #1: not enough frames to estimate rate; consider
 increasing probesize
Guessed Channel Layout for  Input Stream #0.1 : stereo
Input #0, avi, from 'cod2-credits-4sec.avi':
  Duration: 00:00:04.10, start: 0.000000, bitrate: 591512 kb/s
    Stream #0:0: Video: fraps (FPS1 / 0x31535046), yuvj420p(pc, bt709), 1920x108
0, 60 fps, 60 tbr, 60 tbn, 60 tbc
    Stream #0:1: Audio: pcm_s16le ([1][0][0][0] / 0x0001), 48000 Hz, stereo, s16
, 1536 kb/s
No pixel format specified, yuvj420p for H.264 encoding chosen.
Use -pix_fmt yuv420p for compatibility with outdated media players.
[libx264 @ 000000000280aae0] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2
AVX
[libx264 @ 000000000280aae0] profile High, level 4.2
[libx264 @ 000000000280aae0] 264 - core 135 r2345 f0c1c53 - H.264/MPEG-4 AVC cod
ec - Copyleft 2003-2013 - http://www.videolan.org/x264.html - options: cabac=1 r
ef=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed
_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pski
p=1 chroma_qp_offset=-2 threads=18 lookahead_threads=3 sliced_threads=0 nr=0 dec
imate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b
_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=
25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.
60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, mp4, to 'c:\temp\test.mp4':
  Metadata:
    encoder         : Lavf55.18.102
    Stream #0:0: Video: h264 (libx264) ([33][0][0][0] / 0x0021), yuvj420p, 1920x
1080, q=-1--1, 15360 tbn, 60 tbc
    Stream #0:1: Audio: aac (libvo_aacenc) ([64][0][0][0] / 0x0040), 48000 Hz, s
tereo, s16, 128 kb/s
Stream mapping:
  Stream #0:0 -> #0:0 (fraps -> libx264)
  Stream #0:1 -> #0:1 (pcm_s16le -> libvo_aacenc)
Press [q] to stop, [?] for help
frame=   67 fps=0.0 q=31.0 size=       0kB time=00:00:00.05 bitrate=   7.7kbits/
frame=  109 fps=101 q=31.0 size=     344kB time=00:00:00.94 bitrate=2974.1kbits/
frame=  130 fps= 83 q=31.0 size=     368kB time=00:00:01.10 bitrate=2738.8kbits/
frame=  148 fps= 71 q=31.0 size=     431kB time=00:00:01.95 bitrate=1811.1kbits/
frame=  168 fps= 65 q=31.0 size=     468kB time=00:00:01.95 bitrate=1963.5kbits/
frame=  190 fps= 61 q=31.0 size=     486kB time=00:00:02.10 bitrate=1895.6kbits/
frame=  214 fps= 59 q=31.0 size=     531kB time=00:00:02.95 bitrate=1471.7kbits/
frame=  229 fps= 55 q=31.0 size=     550kB time=00:00:02.95 bitrate=1525.6kbits/
frame=  246 fps= 45 q=-1.0 Lsize=    1423kB time=00:00:04.10 bitrate=2838.9kbits
/s dup=1 drop=0
video:1350kB audio:65kB subtitle:0 global headers:0kB muxing overhead 0.540729%
[libx264 @ 000000000280aae0] frame I:1     Avg QP:19.48  size:231215
[libx264 @ 000000000280aae0] frame P:62    Avg QP:21.34  size: 11727
[libx264 @ 000000000280aae0] frame B:183   Avg QP:26.78  size:  2315
[libx264 @ 000000000280aae0] consecutive B-frames:  0.4%  0.0%  3.7% 95.9%
[libx264 @ 000000000280aae0] mb I  I16..4:  6.0% 66.2% 27.8%
[libx264 @ 000000000280aae0] mb P  I16..4:  0.1%  0.7%  0.1%  P16..4: 28.8%  5.3
%  3.6%  0.0%  0.0%    skip:61.5%
[libx264 @ 000000000280aae0] mb B  I16..4:  0.0%  0.0%  0.0%  B16..8: 20.9%  0.7
%  0.1%  direct: 0.1%  skip:78.2%  L0:50.1% L1:49.7% BI: 0.2%
[libx264 @ 000000000280aae0] 8x8 transform intra:69.8% inter:77.6%
[libx264 @ 000000000280aae0] coded y,uvDC,uvAC intra: 80.3% 55.4% 18.1% inter: 4
.2% 2.4% 0.0%
[libx264 @ 000000000280aae0] i16 v,h,dc,p: 15% 19% 18% 48%
[libx264 @ 000000000280aae0] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 14% 19% 23%  5%  7%
 5% 12%  6%  9%
[libx264 @ 000000000280aae0] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 17% 18% 17%  7%  8%
 7% 12%  6%  7%
[libx264 @ 000000000280aae0] i8c dc,h,v,p: 62% 21% 15%  2%
[libx264 @ 000000000280aae0] Weighted P-Frames: Y:1.6% UV:0.0%
[libx264 @ 000000000280aae0] ref P L0: 64.7% 21.8% 10.9%  2.6%  0.1%
[libx264 @ 000000000280aae0] ref B L0: 92.9%  5.8%  1.3%
[libx264 @ 000000000280aae0] ref B L1: 95.7%  4.3%
[libx264 @ 000000000280aae0] kb/s:2696.62

[YellowOnion]

Your output is yuvj420p, its most probably a problem with Windows Media player or your video card drivers assuming its yuv420p, that is why when specifying yuv420p it plays fine, the moral is ALWAYS USE yuv420p, for compatibility reasons.

I've had similar issues where Windows Media Player didn't play the file correctly, but VLC played it fine.

[Carl Eugen Hoyos]

It is correct and known that WMP (incorrectly) interprets H264 yuvj420p as yuv420p, see the warning that FFmpeg prints:

No pixel format specified, yuvj420p for H.264 encoding chosen.
Use -pix_fmt yuv420p for compatibility with outdated media players.