[FFmpeg-devel] [PATCH] vf_framestep: add blend parameter for motion blur effect

[Matthias Troffaes]

Many thanks for your feedback, Michael and Nicolas.

On Wed, Apr 5, 2017 at 7:02 PM, Michael Niedermayer
<michael at niedermayer.cc> wrote:
> fails on qemu mips

Thank you for flagging this failure. I'm currently trying to reproduce
the problem. May I ask what toolchain you typically use to compile to
mips? (I'm trying debian malta on qemu-system-mips, but it seems that
the configure script can't correctly set up config.h in that
environment.)

On Thu, Apr 6, 2017 at 8:42 AM, Nicolas George <george at nsup.org> wrote:
> There is already blending code in vf_framerate and vf_overlay,

... indeed, and also in vf_blend.

> I think
> we should not add new similar code without sharing it as much as
> possible.

Thanks for pointing out the existence of similar code. I agree.

There is however one problem: all blend codes mentioned work on 2
frames only. Whilst it is possible to express an n-frame blend in
terms of n-1 consecutive 2-frame blends with appropriate weights, the
issues with doing so are that it is much slower for large n (n-1
additions + 1 divide per pixel for n-frame blend, versus n-1 additions
+ n-1 multiplications + n-1 divides per pixel for n-1 consecutive
2-frame blends), and that the precision might be compromised as the
current 2-frame blend code round everything to the integer at each
step. Here's some python code to demonstrate the latter problem:

rnd = round

def blend_exact(xs):
    return rnd(sum(xs) / float(len(xs)))

def blend_recurse(xs):
    head = xs[0]
    if len(xs) == 1:
        return head
    else:
        tail = xs[1:]
        return rnd((head + len(tail) * blend_recurse(tail)) / float(len(xs)))

def test(xs):
    print(blend_exact(xs))
    print(blend_recurse(xs))

test([1,1,3])
test([1,1,1,3])
...
test([1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,3])
...

The exact version gives 1 for all of these test cases, whilst the
recursive version gives 2 - I'd say that's quite bad: you can add as
many 1's to the front as you want, the result for the recursive
version will always return 2, even if the exact result is arbitrarily
close to 1.

One way of fixing this is by increasing the precision of the 2-frame
blend. However, the current blending codes for 2 frames, as they are
designed now in ffmpeg, do not have increased precision in the target
frame. They are therefore unsuitable for consecutively blending a
large number of frames: they are both slower and less inaccurate for
that purpose when compared to the exact version as in the patch.

This all said, I could absolutely see good use in refactoring the code
to provide a common basis for the various frame blending codes that
are already out there. Given that there is no suitable n-frame blend
code currently elsewhere (unless someone can point me to it, but I
didn't find anything), do you think it is fair of me to argue that
such refactoring is beyond the scope of this specific patch?

Kind regards,
Matthias