[FFmpeg-devel] [PATCHv8] libavcodec: v4l2: add support for v4l2 mem2mem codecs

Jorge Ramirez jorge.ramirez-ortiz at linaro.org
Tue Sep 5 10:16:46 EEST 2017

On 09/05/2017 12:16 AM, Mark Thompson wrote:
> On 04/09/17 22:55, Jorge Ramirez wrote:
>> On 09/04/2017 11:29 PM, Mark Thompson wrote:
>>>> ... stuff ...
>>> So the sequence of calls is:
>>> send_frame(frame 0) -> success
>>> receive_packet() -> EAGAIN
>>> send_frame(frame 1) -> success
>>> receive_packet() -> EAGAIN
>>> ...
>>> send_frame(frame 15) -> success
>>> receive_packet() -> EAGAIN
>>> send_frame(frame 16) -> success
>>> receive_packet() -> packet 0
>>> receive_packet() -> packet 1
>>> ...
>>> receive_packet() -> packet 15
>>> receive_packet() -> packet 16
>>> receive_packet() -> blocks
>>> This appears correct to me - since EAGAIN has not been returned by a receive_packet() call, it can be called again repeatedly (as documented in avcodec.h).   Do you disagree?
>> I would have expected that after a packet is received, a new frame is enqueued instead of executing again receive_packet (why do we that? what is the benefit?)
>> under these circumsntances I can't block in receive_packet blindly, I have to track in user-space what the driver has in its guts and predict how much it needs to keep working....I dont think it is a good idea.
> Feel free to submit a patch to avcodec.h which changes the definition of the API.

no, that is ok. I can work around it easily (maybe v4l2 has special 
needs compared to the rest of ffmpeg)

>>> I think that the problem is that you are only polling on the frame buffer queue when blocking, so you don't see the packet buffers becoming free in the packet buffer queue - if you did see and dequeue them, you would then return EAGAIN to indicate that more input is needed.  (See comments in <e4c6a8d7-798a-dfdb-b748-42936e944829 at jkqxz.net>.)
>> I could manually track it that way with additional counters - so before blocking  I could see count many frames are enqueued in the input and if there is not at least one I could just return EAGAIN.
>> but the behaviour differs from encoding.
> The behaviour is identical.  You return the output buffer if there is one available on the output queue, otherwise you return EAGAIN if there is any space on the input queue, otherwise you block waiting for either of those to become possible.

but we must be talking about different things because what I observe in 
decode it is 180 degrees away from identical

In the decode case the receive_frame API (a single call for input and 
output) can implement a simple model [1]
  - receive a packet,
  - push it to the input queue  (push till the input queue is full at init)
  - block for data to be ready in the output queue.

static int v4l2m2m_receive_frame(AVCodecContext *avctx, AVFrame *frame)
     V4L2m2mContext *s = avctx->priv_data;
     V4L2Context *const capture = &s->capture;
     V4L2Context *const output = &s->output;
     AVPacket avpkt = {0};
     int timeout = -1;            /* block for capture ready */
     int ret;

     ret = ff_decode_get_packet(avctx, &avpkt);
     if (ret < 0 && ret != AVERROR_EOF)
         return ret;

     if (s->draining)
         goto dequeue;

     ret = ff_v4l2_enqueue_packet(output, &avpkt);
     if (ret < 0)
         return ret;

     if (avpkt.size) {
         ret = try_start(avctx);
         if (ret)
             return 0;

     if (!s->data_queued) {
         if (output->ops.get_buffer(output))
             return AVERROR(EAGAIN);
         else s->data_queued = 1;

     return ff_v4l2_dequeue_frame(capture, frame, timeout);

To sum up,. every time a packet is pushed to input, a frame can be dequeued.

However the encode API seems wrongly designed (heavy coupled) since 
receive_frame has to check if there is space in the input queue.
Receive frame should only care about the output queue - otherwise what 
is the reason for having two calls send_frame and receive_packet?

In my view, receive_packet should just block. And send_frame should be 
called everytime a packet is received in receive_packet.

that way I can
1. fill the input pipe
2. start streaming always blocking for output.

>> in decoding I queue all buffers and then block in capture; everytime it unblocks and returns a new frame back to ffmpeg, the next call just enqueues a packet (it doesnt try to dequeue another frame) so the pipeline never stops.
>> I think the code should to be symmetrical for encoding and decoding...
> Yes.


> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org
> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel

More information about the ffmpeg-devel mailing list