[Ffmpeg-devel] rounding question
Loren Merritt
lorenm
Mon Mar 12 20:20:58 CET 2007
On Fri, 9 Mar 2007, Marc Hoffman wrote:
> Looking at the set of pixel functions which do xy corner averaging I'm
> confused by the reference math. There are two functions one with rounding
> and the other with out. The one with rounding adds 4 samples and then adds
> 1/2 and truncates. The one that is titled no_rnd adds 1/4 and then
> truncates. I'm a little confused by the semantics and am not sure why this
> is.
I don't know where the terminology comes from (maybe some mpeg standard?),
but I can explain why those are the two algorithms used.
When dealing with rounding of real numbers, the standard mathematical
convention is to round .5 up. But when dealing with finite-precision
numbers (such as this case with only 2 bits of fraction), that would add a
bias towards larger numbers. i.e. if you run motion compensation for
several frames without coding a residual, the last frame will be lighter
than the first.
IIRC the standard in banking is to round .5 to the nearest even integer,
but video codecs wouldn't want to do that since it adds computational
complexity.
So mpeg4's algorithm alternates. Half the frames round .5 up, half round
down.
Note that this only applies to how you treat .5, any other fraction is
definitely closer to one integer than to another, so they always get
rounded to nearest.
--Loren Merritt
More information about the ffmpeg-devel
mailing list