[FFmpeg-devel] [PATCH] filter_design: document ownership and permissions.
nicolas.george at normalesup.org
Sun Aug 12 14:00:03 CEST 2012
Signed-off-by: Nicolas George <nicolas.george at normalesup.org>
doc/filter_design.txt | 155 ++++++++++++++++++++++++++++++++++++++++++++++++-
1 file changed, 154 insertions(+), 1 deletion(-)
If what is stated below is considered correct, then the next task will be to
ensure that all filters obey the rules (and I am fairly sure a lot of them
diff --git a/doc/filter_design.txt b/doc/filter_design.txt
index 9e3eb3d..2754930 100644
@@ -33,7 +33,160 @@ Format negotiation
Buffer references ownership and permissions
+ Audio and video data are voluminous; the buffer and buffer reference
+ mechanism is intended to avoid, as much as possible, expensive copies of
+ that data while still allowing the filters to produce correct results.
+ The data is stored in buffers represented by AVFilterBuffer structures.
+ They must not be accessed directly, but through references stored in
+ AVFilterBufferRef structures. Several references can point to the
+ same buffer; the buffer is automatically deallocated once all
+ corresponding references have been destroyed.
+ The characteristics of the data (resolution, sample rate, etc.) are
+ stored in the reference; different references for the same buffer can
+ show different characteristics. In particular, a video reference can
+ point to only a part of a video buffer.
+ A reference is usually obtained as input to the start_frame or
+ filter_samples method or requested using the ff_get_video_buffer or
+ ff_get_audio_buffer functions. A new reference on an existing buffer can
+ be created with the avfilter_ref_buffer. A reference is destroyed using
+ the avfilter_unref_bufferp function.
+ Reference ownership
+ At any time, a reference “belongs” to a particular piece of code,
+ usually a filter. With a few caveats that will be explained below, only
+ that piece of code is allowed to access it. It is also responsible for
+ destroying it, although this is sometimes done automatically (see the
+ section on link reference fields).
+ Here are the (fairly obvious) rules for reference ownership:
+ * A reference received by the start_frame or filter_samples method
+ belong to the corresponding filter.
+ Special exception: for video references: it may be used internally for
+ automatic copying and must not be destroyed before end_frame; it can
+ be given away to ff_start_frame.
+ * A reference passed to ff_start_frame or ff_filter_samples is given
+ away and must no longer be used.
+ * A reference created with avfilter_ref_buffer belongs to the code that
+ created it.
+ * A reference obtained with ff_get_video_buffer of ff_get_audio_buffer
+ belongs to the code that requested it.
+ * A reference given as return value by the get_video_buffer or
+ get_audio_buffer method is given away and must no longer be used.
+ Link reference fields
+ The AVFilterLink structure has a few AVFilterBufferRef fields. Here are
+ the rules to handle them:
+ * cur_buf is set before the start_frame and filter_samples methods to
+ the same reference given as argument to the methods and belongs to the
+ destination filter of the link. If it has not been cleared after
+ end_frame or filter_samples, libavfilter will automatically destroy
+ the reference; therefore, any filter that needs to keep the reference
+ for longer must set cur_buf to NULL.
+ * out_buf belongs to the source filter of the link and can be used to
+ store a reference to the buffer that has been sent to the destination.
+ If it is not NULL after end_frame or filter_samples, libavfilter will
+ automatically destroy the reference.
+ If a video input pad does not have a start_frame method, the default
+ method will allocate a buffer on the first output of the filter, store
+ the reference in out_buf and push a second reference to the output.
+ * src_buf, cur_buf_copy and partial_buf are used by libavfilter
+ internally and must not be accessed by filters.
+ Reference permissions
+ The AVFilterBufferRef structure has a perms field that describes what
+ the code that owns the reference is allowed to do to the buffer data.
+ Different references for the same buffer can have different permissions.
+ For video filters, the permissions only apply to the parts of the buffer
+ that have already been covered by the draw_slice method.
+ The value is a binary OR of the following constants:
+ * AV_PERM_READ: the owner can read the buffer data; this is essentially
+ always true and is there for self-documentation.
+ * AV_PERM_WRITE: the owner can modify the buffer data.
+ * AV_PERM_PRESERVE: the owner can rely on the fact that the buffer data
+ will not be modified by previous filters.
+ * AV_PERM_REUSE: the owner can output the buffer several times, without
+ modifying the data in between.
+ * AV_PERM_REUSE2: the owner can output the buffer several times an
+ modify the data in between (useless without the WRITE permissions).
+ * AV_PERM_ALIGN: the owner can access the data using fast operations
+ that require data alignment.
+ The READ, WRITE and PRESERVE permissions are about sharing the same
+ buffer between several filters to avoid expensive copies without them
+ doing conflicting changes on the data.
+ The REUSE and REUSE2 permissions are about special memory for direct
+ rendering. For example a buffer directly allocated in video memory must
+ not modified once it is displayed on screen, or it will cause tearing;
+ it will therefore not have the REUSE2 permission.
+ The ALIGN permission is about extracting part of the buffer, for
+ copy-less padding or cropping for example.
+ References received on input pads are guaranteed to have all the
+ permissions stated in the min_perms field and none of the permissions
+ stated in the rej_perms.
+ References obtained by ff_get_video_buffer and ff_get_video_buffer are
+ guaranteed to have at least all the permissions requested as argument.
+ References created by avfilter_ref_buffer have the same permissions as
+ the original reference minus the ones explicitly masked; the mask is
+ usually ~0 to keep the same permissions.
+ Filters should remove permissions on reference they give to output
+ whenever necessary. It can be automatically done by setting the
+ rej_perms field on the output pad.
+ Here are a few guidelines corresponding to common situations:
+ * Filters that modify and forward their frame (like drawtext) need the
+ WRITE permission.
+ * Filters that read their input to produce a new frame on output (like
+ scale) need the WRITE permission on input and WRITE permission on the
+ new buffer.
+ * Filters that intend to keep a reference after the filtering process
+ is finished (after end_frame or filter_samples returns) must have the
+ PRESERVE permission on it and remove the WRITE permission if they
+ create a new reference to give it away.
+ * Filters that intend to modify a reference they have kept after the end
+ of the filtering process need the REUSE2 permission and must remove
+ the PRESERVE permission if they create a new reference to give it
More information about the ffmpeg-devel