[FFmpeg-devel] [PATCH] VC-1 MMX DSP functions

[christophe.gisquet at free.fr]

Hi,

(I hope the webmail I'm using wrong break anything)

Selon Loren Merritt <lorenm at u.washington.edu>:
> (a) Bias it. From your explanation, the possible values should be in
> [-4080,36720] which is 16 bits of dynamic range, it's just not nicely
> centered on 0.

Very good idea. To be more precise however, the bias will depend on the pair of
filters. Let's note A=[-1 9 9 -1] and B=[-4 18 53 -3]. The different cases are:
- A/2^1 => output in [-255; 2295]
  Convolved with A/2^6 afterwards, so bias should be -1024
- A/2^3 => [-64; 574]
  Convolved with B/2^6 afterwards, so bias should be -256
- B/2^3 => [-223; 2263]
  Convolved with A/2^6 afterwards, so bias should be -1024
- B/2^5 => [-56; 566]
  Convolved with B/2^6 afterwards, so bias should be -256

I think it's possible to apply it on the first filtering, by modifying the
rounder used to our need. I haven't thought yet much about un-biasing, but that
will need to be done after scaling. Use of power of 2 bias superior to the scale
factor should help reduce it to an addition, hopefully.

> (b) Instead of (-(p1+p4)+(p2+p3)+((p2+p3)<<3))>>1, do
> (((p2+p3)-(p1+p4))>>1)+((p2+p3)<<2)

The same problem occurs with B filter when on the horizontal pass, so I'll have
to think more about how it applies to such case.

Thanks for the input,
-- 
Christophe GISQUET