[FFmpeg-user] Decoding frames as fast as possible

Evan cerebro.alexiel at gmail.com
Tue Jul 3 11:23:47 EEST 2018

Hi everyone,

I'm developping a media player featuring advanced video/frame processing.
To be able to seek to any particular frame, I need to know, for each frame,
its presentation time and if available, if it's a key frame or not. The
latter is optional, really.

To do that I build an "index" once the media is valid and opened.
It's a basically a loop, fetching a packet, checking if it's the video
stream I'm interested in, feeding the codec until he can produces a frame
or reaching the end of the file.

This is perfectly working but it's way too long for me, like 20 seconds for
a ~800MB mp4/h264 file.
I'm considering the following options:
- building the index in the background while playing the media: I guess
I'll have to copy the decoder context to not mix both processes but I
expect having playback (stuttering) issues.
- use multiple threads: from what I read, I can only use one thread for a
stream. I may eventually use two threads, one for video and one for audio
but I want to speed up video decoding fisrt.
- use the GPU to decode frames faster: while it sounds like a good idea, I
have to add two constraints: I'm on Windows 7 (and soon, Windows 10) and I
want a generic approach so not H.264-only.
- decode only packets: it's tempting but I think I will either loose
timestamp information or precision (because of I, P, B frames.
- use an option (av_dict_set) to do a dummy frame decoding: is there
anything like that?
- use reference counting to avoid costly frame allocations: I already tried
that but didn't see any difference.

For now, I will measure time elapsed in each piece of code in order to
pinpoint lengthy operations.
If you guys have any hint, I'd be glad to hear them.


More information about the ffmpeg-user mailing list