[FFmpeg-devel] [PATCH 2/5] lavfi/gradfun: fix rounding in MMX code.
Reimar.Doeffinger at gmx.de
Fri Dec 7 21:29:39 CET 2012
On 7 Dec 2012, at 16:28, Loren Merritt <lorenm at u.washington.edu> wrote:
> On Fri, 7 Dec 2012, Reimar Döffinger wrote:
>> "Clément Bÿÿsch" <ubitux at gmail.com> wrote:
>>> Current code divide before increasing precision.
>> Are you sure the shift can _never_ overflow? It is definitely very tight.
> I expect it can overflow, but the case where it does so also has m=0 for
> sane filter strengths. So the appropriate fix would be to limit strength,
> not change the asm.
Since the proposal is to change the asm, do you mean the patch is right, it just should be combined with limiting the strength?
I have no objections about that, I just kind of assumed there was a good reason for doing things in this order.
More information about the ffmpeg-devel