The avcodec_send_packet()/avcodec_receive_frame()/avcodec_send_frame()/ avcodec_receive_packet() functions provide an encode/decode API, which decouples input and output. More...

The avcodec_send_packet()/avcodec_receive_frame()/avcodec_send_frame()/ avcodec_receive_packet() functions provide an encode/decode API, which decouples input and output.

The API is very similar for encoding/decoding and audio/video, and works as follows:

Set up and open the AVCodecContext as usual.
Send valid input:
- For decoding, call avcodec_send_packet() to give the decoder raw compressed data in an AVPacket.
- For encoding, call avcodec_send_frame() to give the encoder an AVFrame containing uncompressed audio or video. In both cases, it is recommended that AVPackets and AVFrames are refcounted, or libavcodec might have to copy the input data. (libavformat always returns refcounted AVPackets, and av_frame_get_buffer() allocates refcounted AVFrames.)
Receive output in a loop. Periodically call one of the avcodec_receive_*() functions and process their output:
- For decoding, call avcodec_receive_frame(). On success, it will return an AVFrame containing uncompressed audio or video data.
- For encoding, call avcodec_receive_packet(). On success, it will return an AVPacket with a compressed frame. Repeat this call until it returns AVERROR(EAGAIN) or an error. The AVERROR(EAGAIN) return value means that new input data is required to return new output. In this case, continue with sending input. For each input frame/packet, the codec will typically return 1 output frame/packet, but it can also be 0 or more than 1.

At the beginning of decoding or encoding, the codec might accept multiple input frames/packets without returning a frame, until its internal buffers are filled. This situation is handled transparently if you follow the steps outlined above.

In theory, sending input can result in EAGAIN - this should happen only if not all output was received. You can use this to structure alternative decode or encode loops other than the one suggested above. For example, you could try sending new input on each iteration, and try to receive output if that returns EAGAIN.

End of stream situations. These require "flushing" (aka draining) the codec, as the codec might buffer multiple frames or packets internally for performance or out of necessity (consider B-frames). This is handled as follows:

Instead of valid input, send NULL to the avcodec_send_packet() (decoding) or avcodec_send_frame() (encoding) functions. This will enter draining mode.
Call avcodec_receive_frame() (decoding) or avcodec_receive_packet() (encoding) in a loop until AVERROR_EOF is returned. The functions will not return AVERROR(EAGAIN), unless you forgot to enter draining mode.
Before decoding can be resumed again, the codec has to be reset with avcodec_flush_buffers().

Using the API as outlined above is highly recommended. But it is also possible to call functions outside of this rigid schema. For example, you can call avcodec_send_packet() repeatedly without calling avcodec_receive_frame(). In this case, avcodec_send_packet() will succeed until the codec's internal buffer has been filled up (which is typically of size 1 per output frame, after initial input), and then reject input with AVERROR(EAGAIN). Once it starts rejecting input, you have no choice but to read at least some output.

Not all codecs will follow a rigid and predictable dataflow; the only guarantee is that an AVERROR(EAGAIN) return value on a send/receive call on one end implies that a receive/send call on the other end will succeed, or at least will not fail with AVERROR(EAGAIN). In general, no codec will permit unlimited buffering of input or output.

This API replaces the following legacy functions:

avcodec_decode_video2() and avcodec_decode_audio4(): Use avcodec_send_packet() to feed input to the decoder, then use avcodec_receive_frame() to receive decoded frames after each packet. Unlike with the old video decoding API, multiple frames might result from a packet. For audio, splitting the input packet into frames by partially decoding packets becomes transparent to the API user. You never need to feed an AVPacket to the API twice (unless it is rejected with AVERROR(EAGAIN) - then no data was read from the packet). Additionally, sending a flush/draining packet is required only once.
avcodec_encode_video2()/avcodec_encode_audio2(): Use avcodec_send_frame() to feed input to the encoder, then use avcodec_receive_packet() to receive encoded packets. Providing user-allocated buffers for avcodec_receive_packet() is not possible.
The new API does not handle subtitles yet.

Mixing new and old function calls on the same AVCodecContext is not allowed, and will result in undefined behavior.

Some codecs might require using the new API; using the old API will return an error when calling it. All codecs support the new API.

A codec is not allowed to return AVERROR(EAGAIN) for both sending and receiving. This would be an invalid state, which could put the codec user into an endless loop. The API has no concept of time either: it cannot happen that trying to do avcodec_send_packet() results in AVERROR(EAGAIN), but a repeated call 1 second later accepts the packet (with no other receive/flush API calls involved). The API is a strict state machine, and the passage of time is not supposed to influence it. Some timing-dependent behavior might still be deemed acceptable in certain cases. But it must never result in both send/receive returning EAGAIN at the same time at any point. It must also absolutely be avoided that the current state is "unstable" and can "flip-flop" between the send/receive APIs allowing progress. For example, it's not allowed that the codec randomly decides that it actually wants to consume a packet now instead of returning a frame, after it just returned AVERROR(EAGAIN) on an avcodec_send_packet() call.