[FFmpeg-devel] [PATCH] vaapi_encode: explicitly free buffers after vaEndPicture

[Will Kelleher]

Hi all -

I'm experiencing some significant heap growth when encoding with VAAPI on
my Ivy Bridge hardware.  Based on what I'm seeing from Massif, it seems like
the parameter buffers allocated in vaapi_encode_make_param_buffer are leaking.

I came across this thread [1] that indicates that vaEndPicture might not be
freeing the param buffers like the libva documentation says it should.

I also noticed that VLC [2] seems to explicitly call vaDestroyBuffer on the
param buffers after vaEndPicture.

When I try that, the leak is gone.

Thoughts?

Thanks,

will

1: https://lists.freedesktop.org/archives/libva/2012-July/000952.html
2: https://github.com/BtbN/vlc-vaapi-enc/blob/master/vlc-h264-vaapi-enc.c

---
 libavcodec/vaapi_encode.c | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/libavcodec/vaapi_encode.c b/libavcodec/vaapi_encode.c
index 329b33c..6fca815 100644
--- a/libavcodec/vaapi_encode.c
+++ b/libavcodec/vaapi_encode.c
@@ -370,6 +370,19 @@ static int vaapi_encode_issue(AVCodecContext *avctx,
         goto fail_at_end;
     }
 
+    // Although the documentation says that vaEndPicture will free these buffers,
+    // in practice that appears not to be the case. We need to explicitly free
+    // them to avoid a massive leak.
+    for(i = 0; i < pic->nb_param_buffers; i++) {
+        if (pic->param_buffers[i] != VA_INVALID_ID) {
+            vas = vaDestroyBuffer(ctx->hwctx->display, pic->param_buffers[i]);
+            if (vas != 0) {
+                av_log(avctx, AV_LOG_WARNING, "Failed to destroy param buffer %d\n",i);
+            }
+            pic->param_buffers[i] = VA_INVALID_ID;
+        }
+    }
+
     pic->encode_issued = 1;
 
     if (ctx->issue_mode == ISSUE_MODE_SERIALISE_EVERYTHING)
-- 
2.8.0