[FFmpeg-devel] [PATCH]Fix pnm encoding with bpc set

[Christophe Gisquet]

Hi,

2014-08-24 12:48 GMT+02:00 Carl Eugen Hoyos <cehoyos at ag.or.at>:
>> I think "[PATCH 2/5] pnmenc: use bits_per_raw_sample"
>> is wrong then
>
> Definitely, I still wonder how you tested it.

ljpeg 36bits from #2966 and having the actual bits_per_raw_sample value

What this situation underlines is that bits_per_raw_sample is
unreliable. Compare ljpeg RGB with more than 8 bits and pnmdec:
- ljpeg sets bits_per_raw_sample to 12 (because that's what the sample
dynamics is), colorspace is 16bits
- pnmdec set bits_per_raw_sample to 12, but scale samples to 16bits to
match colorspace, hereby making bits_per_raw_sample less useful
(samples have actually 16 bits of dynamics)
We have 2 cases resulting in the same info but needing different processing.

So, at this point you can't know if bits_per_raw_sample!=color space
bitdepth means the values have the right dynamics indicated by
bits_per_raw_sample, if they have been rescaled to get to the
colorspace bitdepth, and so on.

A potential "fix" would be to remove bits_per_raw_sample setting from pnmdec.c

But I expect some other decoders to be potentially wrong/doing yet
another thing at this point, so bits_per_raw_sample is not completely
reliable.

My suggestion in "[PATCH 2/5] pnmenc: use bits_per_raw_sample" was "if
bits_per_raw_sample is correct". This looks to uncertain to me, so
that patch should be reverted.

-- 
Christophe