[FFmpeg-devel] [PATCH] lavc/libvpxenc: prevent fifo from filling up

[David Lemler]

Prevent the fifo used in encoding VPx videos from filling up and
stopping encode when it reaches 21845 items, which happens when the
video has more than that number of frames.

Incorporated suggestion from James Zern to prevent calling
frame_data_submit() at all when performing the first pass of a 2-pass
encode so the fifo is not filled at all; replaces original patch which
drained the fifo after filling to prevent it from becoming full.

Fixes the regression originally introduced in
5bda4ec6c3cb6f286bb40dee4457c3c26e0f78cb

Co-authored-by: James Zern <jzern at google.com>
Signed-off-by: David Lemler <david at lemler.family>
---
 libavcodec/libvpxenc.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/libavcodec/libvpxenc.c b/libavcodec/libvpxenc.c
index 8833df2d68..549ac55aaa 100644
--- a/libavcodec/libvpxenc.c
+++ b/libavcodec/libvpxenc.c
@@ -1780,9 +1780,11 @@ static int vpx_encode(AVCodecContext *avctx, AVPacket
*pkt,
             }
         }
 
-        res = frame_data_submit(avctx, ctx->fifo, frame);
-        if (res < 0)
-            return res;
+        if (!(avctx->flags & AV_CODEC_FLAG_PASS1)) {
+            res = frame_data_submit(avctx, ctx->fifo, frame);
+            if (res < 0)
+                return res;
+        }
     }
 
     // this is for encoding with preset temporal layering patterns defined
in
-- 
2.41.0.windows.1