[FFmpeg-devel] 10bit DNxHD vs dct_quantize_c
joseph at mirriad.com
Fri Mar 18 18:24:57 CET 2011
Having heard someone is working on a 10-bit safe (I)DCT assembly implementation, I put 10bit DNxHD decoding on hold for now. Instead I've been working on encoding and today I finally got that working. To get there I had to write my own DCT and quantization code, as I couldn't get the existing code working for 10bit, even having forced DCT to faandct implementation. The existing code I am refering to is dct_quantize_c(). I've got a couple of questions regarding it.
First of all, what's the advantage of using that complex looking quantization code in favour of using a dead-simple quantization formula from the VC-3 spec:
quantized[u, v] = sign(X[u, v]) * floor((abs(X[u, v]) * p) / (qsf * weight_table[u, v]))
where p is 32 for 8-bit samples and 8 for 10-bit ones.
Second, what's up with faandct()? It's definitely not equivalent to a textbook DCT transform, as it produces values in the wrong range. I saw the FAAN_POSTSCALE definition, but its effect is not documented anywhere and I can't figure out what it does on my own.
More information about the ffmpeg-devel