[FFmpeg-devel] [PATCH 3/5] ffmpeg: flush and drain video filters.

Michael Niedermayer michaelni at gmx.at
Wed Mar 14 18:12:23 CET 2012

On Wed, Mar 14, 2012 at 02:12:30PM +0100, Nicolas George wrote:
> Le quartidi 24 ventôse, an CCXX, Michael Niedermayer a écrit :
> > This sounds much better indeed
> TGood, thanks.
> > many
> > 
> > 1. one could show the video a few frames ago in a smaller picture in
> > the big one as an effect like if someone pointed a camera onto a
> > monitor displaying the video from the camera
> > 
> > 2. one could with several simple filters construct a denoise or
> > temporal average (IIR) filter, with audio it could be a echo filter.
> > 
> > 3. if one made a movie one could for example refer in the movie to a
> > recording of some earlier part of the movie by having that displayed
> > via a picture in picture filter or as a flashback (this maybe isnt
> > the most sane way to implement that though)
> > 
> > 4. iam sure there are various effects one could do with it ...
> [ TL;DR: if filters are deterministic, my proposal with EAGAIN and no FIFOs
> can handle loops. OTOH, either with poll_frame or EAGAIN, loops will lead to
> infinite recursion, and that needs to be detected. ]
> I am not sure I can see exactly how it can be done with a loop and not just
> a cycle, but let us assume it works.
> I believe practical examples will always have an input in the loop: a filter
> with two inputs, one coming from the loop and one coming from outside.
> Something like that:
>     L1   +----+       ,........       +----+  L4
>     ---->|    |  L2  :         :  L3  |    |----->
>          | F1 |----->: filters :----->| F2 |
>       ,->|    |      '.........'      |    |-.
>       |  +----+                       +----+ |
>       |              L5                      |
>       `--------------------------------------'
> Then, unless I am mistaken, there is no need to add a FIFO inside the loop
> to prevent it from producing an infinite number of frames: the availability
> of frames on L1 will regulate the loop.
> To make sure of that, I need to state another principle, I hope you will
> find it acceptable:

> # When a filter has several inputs, its output must not depend on the order
> # of arrival of frames on different inputs.

Its a bit difficult to agree to axioms
without fully understanding the consequences these axioms have.
But taking this in isolation it sounds reasonable

Theres a small problem with the first axiom though:

As reference:
    a filter should
    output its frame as soon at it can, as a reaction from one of its input, and
    not wait for a request on its output.

A frame duplication filter should not output all frames at once as
this could overload the CPU and interrupt smooth playback.
this is especially problematic if the following filters are CPU
intensive. The problem could easily happen with varable fps input
from some frame skip using encoder that decided to skip 2 seconds
or more of black frames. While the filter duplicates 60 frames and
the next denoises them teh audio que might become empty in a single
threaded application ...

yadif in 2field->2frame mode would be another example. yadif currently
does not return the 2nd frame before it has to

> In other words, both these sequences:
> push(in[0], buf1); push(in[1], buf2); push(in[0], buf3); push(in[1], buf4);
> push(in[0], buf1); push(in[1], buf2); push(in[1], buf4); push(in[0], buf3);
> should give the exact same output.
> I can grant two exceptions to this principle: First, if a filter has an
> internal queue that threatens to overflow, it can decide to takes measures.
> Second, if a filter relies on a PRNG, the order of arrival may affect the
> exact sequence produced by the PRNG, that is not too big a problem.
> Rationale for this principle: the order of arrival on different inputs is an
> implementation detail that can change at any time. If we were to try to run
> filters in separate threads with message queues for links, the order of
> arrival on different inputs would become mostly random.

> This principle has a corollary: a filter with several inputs, under normal
> circumstances, will eventually stop producing output if one of its inputs
> stays empty.

this is not so good, a overlay filter might have one of its inputs
reach EOF and the other still should be passed through.
Or from a different POV a news video might have a commentator who
comments various news footage shown in a picture behind her. These
videos will not always run / be shown during the whole news but the
other input (here the commentators video & audio) still continue and
should pass through the filter that combines both.

Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

Observe your enemies, for they first find out your faults. -- Antisthenes
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature
URL: <http://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20120314/583043ab/attachment.asc>

More information about the ffmpeg-devel mailing list