[FFmpeg-devel] [PATCH] pthread_frame: attempt to get frame to reduce latency

[Devin Heitmueller]

On Wed, Mar 11, 2020 at 10:28 AM Derek Buitenhuis
<derek.buitenhuis at gmail.com> wrote:
> > E.g. in FF_THREAD_FRAME 4320x2160 30fps video streaming, 4 threads, the frame caching is 99ms (33ms x 3frames)
> > If the  cpu-decoding-execution-time is 80ms ~ 120ms (dependent on video frame content).
>
> Also aside: It is not useful to measure frame delay in time. It's also not, IMO, maningful to use
> in-flight (wallclock) time.

Regardless of the actual proposed patch, I think the author's use of
wallclock time to describe the problem is very reasonable.  I do a
large amount of work where I'm measuring "glass-to-glass" latency,
where I am interested in the total pipeline (encode/network/decode),
and I definitely went through the experience of trying to figure out
why ffmpeg was introducing nearly 500ms of extra latency during
decoding.  It turned out that it was because my particular platform
had 8-cores and thus 16 decoding threads and thus 15x33ms of delay,
regardless of the stream complexity.

You may not personally care about latency, but there are lots of
people operating in a world where actual latency matters (e.g. news
interviews) and they want to be able to use ffmpeg for decoding in
such environments.  The "problem" is not how many threads are used.
The problem is "there's way too much latency" and proposed solutions
include reducing the thread count or changing the heuristic for
dequeuing frames from the thread pool.

Devin

-- 
Devin J. Heitmueller - Kernel Labs
http://www.kernellabs.com