[FFmpeg-devel] [RFC] Filtering of data packets
michaelni at gmx.at
Thu Jul 18 17:33:32 CEST 2013
On Thu, Jul 18, 2013 at 03:54:19PM +0200, Nicolas George wrote:
> Data structure
> I already checked, AVFrame has almost all the fields present in AVPacket.
> The only exception is convergence_duration, that nobody uses, and it would
> be trivial to add.
> The packet data and size can go in AVFrame.data and
smells like a hack
> Format negotiation and automatic conversion
> Data packets are too abstract, I believe format negotiation and automatic
> conversion are not possible/relevant. The pixel_/sample_format field can
> be used to store a CodecID, to avoid feeding MP3 to an AC3 decoder, but
> that is all.
> Generic pads
> We already have setpts and asetpts, adding dsetpts, and later ssetpts for
> subtitles will start getting old pretty fast. Therefore, we need generic
> pads, so that setpts can work with any type of packet.
> Set AVFilterPad.type to AVMEDIA_TYPE_UNKNOWN to allow lavfi to connect
> this pad to anything without checking the type, and let
> AVFilter.query_formats check that the formats are consistent. If several
> generic filters are connected together, use the complex format negotiation
> to stabilize the type.
> The demuxer source should be able to expose any requested stream as an
> output pad, but I believe it is more practical if all attachments are
> optionally combined into a single output, with one packet per attachment.
> Typical use: connect the attachments output of the demuxer to the ASS
> subtitles renderer for embedded fonts; no need to know how many fonts
> there are.
> Sparse streams
> This is the second difficulty for subtitles filtering, and I believe it
> may also apply to data filtering although I do not have a specific
> scenario in mind for that case. But I have a solution.
> Consider the following scenario:
> [0:v] [1:v] overlay [out]
> A frame arrives on 0:v, so overlay will request one on 1:v. That works
> well because frames can be read from 0:v and 1:v independently.
> [0:a:0] [0:a:1] amerge [out]
> The same will happen, but it is not possible to read frames independently
> from any stream: frames will come however they appear in the input file.
> Requesting a frame on 0:a:1 may cause a frame to arrive on 0:a:0 instead,
> and it will need to be queued.
> It will still work because audio streams are continuous and properly
> interleaved: an audio frame covering the 42.1 - 42.3 time interval in
> stream #0 will be almost immediately followed or preceded by a frame
Only if you are lucky
Real files are not guranteed to e strictly correctly interleaved
also streams can stop and start and have gaps if you are unlucky
> covering a similar interval in stream #1. The filter needs some buffering,
> but not a lot.
> [0:v] [0:s] hardsub [out]
> The same issue happens, but in this case the subtitles stream is not
> continuous: you can have an action scene with only sfx for five minutes,
> hardsub will request a frame on 0:s, that will trigger reading from the
> file, it will find and decode a video frame. Video frames will accumulate
> in the 0:v input until either the buffer queue overflows or OOM happens.
> The solution I suggest is to have regular dummy frames on sparse streams.
> A dummy frame is marked by data being NULL, and counts only for its
In a sane world subtitle streams would have sane keyframe intervals
and there would be subtitle packets every 1-2sec
Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB
Dictatorship naturally arises out of democracy, and the most aggravated
form of tyranny and slavery out of the most extreme liberty. -- Plato
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Size: 198 bytes
Desc: Digital signature
More information about the ffmpeg-devel