[Libav-user] Fwd: How to free an AVBuffer?
Wladislav Artsimovich
ffmpeg at frost.kiwi
Wed Mar 8 10:13:45 EET 2023
Dear Yurii,
ohh, that is correct! I don't need a separate buffer, a misunderstanding
from my side. Many thanks for the explanation!
Doing the following cut the memory leak in half. Only 10.5 MB per 100000
frames leaking remaining.
```C#
fixed (byte* pMessageData = message)
{
AVFrameSideData* sideData = ffmpeg.av_frame_new_side_data(&frame,
AVFrameSideDataType.AV_FRAME_DATA_SEI_UNREGISTERED, (ulong)message.Length);
System.Buffer.MemoryCopy(pMessageData, sideData->data,
(ulong)message.Length, (ulong)message.Length);
ffmpeg.avcodec_send_frame(_pCodecContext,
&frame).ThrowExceptionIfError();
ffmpeg.av_frame_remove_side_data(&frame,
AVFrameSideDataType.AV_FRAME_DATA_SEI_UNREGISTERED);
}
```
No leak happens with
```C#
fixed (byte* pMessageData = message)
{
//AVFrameSideData* sideData = ffmpeg.av_frame_new_side_data(&frame,
AVFrameSideDataType.AV_FRAME_DATA_SEI_UNREGISTERED, (ulong)message.Length);
//System.Buffer.MemoryCopy(pMessageData, sideData->data,
(ulong)message.Length, (ulong)message.Length);
ffmpeg.avcodec_send_frame(_pCodecContext,
&frame).ThrowExceptionIfError();
//ffmpeg.av_frame_remove_side_data(&frame,
AVFrameSideDataType.AV_FRAME_DATA_SEI_UNREGISTERED);
}
```
I still don't quite get, why this isn't enough and why ffmpeg insists on
leaking those remaining 10.5 MB per 100000 frames. Maybe I can somehow
compile libavXX.dll with debug symbols, so the VisualStudio 2022
profiler could actually tell me what's happening, instead of ignoring
what is happening inside those ffmpeg dlls.
> If you wish to save some allocations
Don't need it. Evidently, I can't even get the basics down, so this is
above my pay-grade.
> What is the lifetime of your frame?
Instinctively I would say, it's gone as soon as this encode one frame
ends. At least from caller side in my C# program, there never exist
multiple frames at once. But due to the language used in the docs in
regards to "flushing" aka "draining", I'm not so sure actually.
Especially, since the language used to say what a packet is vs what a
frame is highly confusing. ''For encoding, call
avcodec_receive_packet(). On success, it will return an AVPacket with a
compressed frame. [...] the codec will typically return 1 output
frame/packet"
The code used is based on
https://github.com/Ruslan-B/FFmpeg.AutoGen/blob/master/FFmpeg.AutoGen.Example/H264VideoStreamEncoder.cs
. `public void Encode(AVFrame frame)` receives an AVFrame, that was
given to it from the outside. By my logic, the AVFrame I created is not
needed anymore when `ffmpeg.avcodec_send_frame()` is called. After
returning from `Encode()`, the next AVFrame is created and submitted.
The AVFrame abstraction as created by FFmpeg.AutoGen isn't actually
`IDisposable`, so I can't explicitly `.Dispose()` it. Even so, I doubt
that it matters, since it goes out of scope and the profiler never
reports multiple AVFrame objects being alive.
The memory leak happens in the FFmpeg code, which I can't debug because
the profiler can't look into it. I do not know the lifetime of that and
the "frame/packet" language really doesn't explain to me, whether it is
a packet or a frame that continues on. (Though I presume, the frame is
gone, the thing that lives on is the packet) Finally, there is `public
void Drain()`. The language of flushing/draining suggests lifetimes
existing beyond. A long winded way to say, I'm not sure...
Do you have any idea what those remaining 10.5 mb of memory leak per
100000 franes are made of, after `ffmpeg.av_frame_remove_side_data()`
was called?
Best regards,
Vlad
On 3/8/2023 15:39, Yurii Monakov wrote:
>
>
> ---------- Пересылаемое сообщение ---------
> От: *Yurii Monakov* <monakov.y at gmail.com>
> Дата: Чт, 2 марта 2023 г. в 15:47
> Тема: Re: [Libav-user] How to free an AVBuffer?
> Кому: Wladislav Artsimovich <ffmpeg at frost.kiwi>
>
>
> Vlad,
>
> What is the lifetime of your frame? If it is persistent across calls
> then you allocate
> new side data in every call and ffmpeg simply grows its side data list.
>
> You can use av_frame_new_side_data, copy data to it, send a frame to
> codec and
> then av_frame_remove_side_data. There is no need to allocate a
> separate buffer,
> av_frame_new_side_data does this internally.
>
> If you wish to save some allocations, you need to check if side data
> of a given
> type is already present. If message.Length changes from call to call,
> you have to
> av_buffer_realloc sideData->buf and fix sideData->data/sideData->size
> fields.
>
> Regards,
> Yurii
>
>
> чт, 2 мар. 2023 г. в 07:06, Wladislav Artsimovich <ffmpeg at frost.kiwi>:
>
> Dear Yurii,
>
> thx for elaborating. Indeed reassigning the pointer was a logical
> error,
> thx for pointing (hah) it out. I tried using `av_buffer_create()`
> originally, but couldn't find a way of using that function pointer
> callback `av_buffer_default_free()` in C#.
> From C# pMessageData cannot be freed, but is discarded automatically
> when the fixed{} block ends. I managed to free that MetaDatabuffer
> and
> the associated memory leak is gone! But not via `av_free()` (which
> didn't work, no error, no freeing), but via `av_buffer_unref()`.
> However, there still remains another memory leak caused by
> `av_frame_new_side_data_from_buf()`
>
> How can I solve that? I tried following
> https://chromium.googlesource.com/chromium/third_party/ffmpeg/+/refs/heads/master/libavutil/frame.c#66
>
> and inserting `ffmpeg.av_buffer_unref(&sideData->buf);
> ffmpeg.av_dict_free(&sideData->metadata);
> ffmpeg.av_freep(sideData);` at
> different positions and `av_frame_remove_side_data()` at different
> positions, eg. before or after `ffmpeg.av_packet_free(&pPacket);`.
> The
> Encoder just gracefully stops without error, not processing any
> further
> frames.
>
> What would be the proper procedure to free that AVFrameSideData?
>
> Here is the full breakdown:
>
> 0 MB memory leaked per 100000 frames:
> fixed (byte* pMessageData = message)
> {
> //AVBufferRef* MetaDataBuffer =
> ffmpeg.av_buffer_alloc((ulong)message.Length);
> //System.Buffer.MemoryCopy(pMessageData, MetaDataBuffer->data,
> (ulong)message.Length, (ulong)message.Length);
> //AVFrameSideData* sideData =
> ffmpeg.av_frame_new_side_data_from_buf(&frame,
> AVFrameSideDataType.AV_FRAME_DATA_SEI_UNREGISTERED, MetaDataBuffer);
> ffmpeg.avcodec_send_frame(_pCodecContext,
> &frame).ThrowExceptionIfError();
> //ffmpeg.av_buffer_unref(&MetaDataBuffer);
> }
>
> 62 MB leaked per 100000 frames:
> fixed (byte* pMessageData = message)
> {
> AVBufferRef* MetaDataBuffer =
> ffmpeg.av_buffer_alloc((ulong)message.Length);
> System.Buffer.MemoryCopy(pMessageData, MetaDataBuffer->data,
> (ulong)message.Length, (ulong)message.Length);
> //AVFrameSideData* sideData =
> ffmpeg.av_frame_new_side_data_from_buf(&frame,
> AVFrameSideDataType.AV_FRAME_DATA_SEI_UNREGISTERED, MetaDataBuffer);
> ffmpeg.avcodec_send_frame(_pCodecContext,
> &frame).ThrowExceptionIfError();
> //ffmpeg.av_buffer_unref(&MetaDataBuffer);
> }
>
> 0 MB memory leaked per 100000 frames:
> fixed (byte* pMessageData = message)
> {
> AVBufferRef* MetaDataBuffer =
> ffmpeg.av_buffer_alloc((ulong)message.Length);
> System.Buffer.MemoryCopy(pMessageData, MetaDataBuffer->data,
> (ulong)message.Length, (ulong)message.Length);
> //AVFrameSideData* sideData =
> ffmpeg.av_frame_new_side_data_from_buf(&frame,
> AVFrameSideDataType.AV_FRAME_DATA_SEI_UNREGISTERED, MetaDataBuffer);
> ffmpeg.avcodec_send_frame(_pCodecContext,
> &frame).ThrowExceptionIfError();
> ffmpeg.av_buffer_unref(&MetaDataBuffer);
> }
>
> 24.5 MB leaked per 100000 frames:
> fixed (byte* pMessageData = message)
> {
> AVBufferRef* MetaDataBuffer =
> ffmpeg.av_buffer_alloc((ulong)message.Length);
> System.Buffer.MemoryCopy(pMessageData, MetaDataBuffer->data,
> (ulong)message.Length, (ulong)message.Length);
> AVFrameSideData* sideData =
> ffmpeg.av_frame_new_side_data_from_buf(&frame,
> AVFrameSideDataType.AV_FRAME_DATA_SEI_UNREGISTERED, MetaDataBuffer);
> ffmpeg.avcodec_send_frame(_pCodecContext,
> &frame).ThrowExceptionIfError();
> ffmpeg.av_buffer_unref(&MetaDataBuffer);
> }
>
> Best regards,
>
> Vlad
>
> On 3/2/2023 09:35, Yurii Monakov wrote:
> > av_buffer_alloc allocates buffer of given length (it’s data
> pointer is
> > not null). And after that you replace that pointer to with your own
> > pointer. So, the first allocation get definitely lost.
> > You should copy pMessageData to MetaDataBuffer->data (and free
> > pMessageData manually) or use av_buffer_create function.
> >
> > Yurii
> >
> > Ср, 1 марта 2023 г. в 06:55, Wladislav Artsimovich
> <ffmpeg at frost.kiwi>:
> >
> > Dear FFmpeg and libav users,
> >
> > I am using FFmpeg.AutoGen in my C# Program to write out frames
> > encoded
> > as a h.264 stream and inject per-frame metadata with an
> > unregistered SEI
> > Message. It works great, but I have a memory leak, which I
> don't know
> > how to address. Source file attached.
> >
> > Culprit is
> > ```C#
> > fixed (byte* pMessageData = message)
> > {
> > AVBufferRef* MetaDataBuffer =
> > ffmpeg.av_buffer_alloc((ulong)message.Length);
> > MetaDataBuffer->data = pMessageData;
> > AVFrameSideData* sideData =
> > ffmpeg.av_frame_new_side_data_from_buf(&frame,
> > AVFrameSideDataType.AV_FRAME_DATA_SEI_UNREGISTERED,
> MetaDataBuffer);
> > }
> > ```
> > I create an AVBuffer, as required by
> > av_frame_new_side_data_from_buf().
> > But I cannot free it, resulting in a memory leak. I went through
> > all of
> > https://ffmpeg.org/doxygen/trunk/group__lavu__buffer.html and
> > tried out
> > different av_freep() av_free(), av_buffer_unref() functions,
> changing
> > when to free etc. When I think I perform the free correctly,
> nothing
> > happens. No error, no freeing just nothing. Clearly I
> misunderstand
> > something.
> > To add insult to injury, the Visual Studio profiler cannot look
> > inside
> > the unmanaged memory of libav and reports, that the heap is fine
> > and not
> > growing, see attached screenshot.
> >
> > How and when can I free this memory created allocated by
> > av_buffer_alloc() in the attached source file?
> >
> > I asked the same question in the FFmpeg.AutoGen Questions
> Repo. Some
> > more context there:
> > https://github.com/Ruslan-B/FFmpeg.AutoGen.Questions/issues/36
> >
> > Best regards,
> >
> > Vlad
> > _______________________________________________
> > Libav-user mailing list
> > Libav-user at ffmpeg.org
> > https://ffmpeg.org/mailman/listinfo/libav-user
> >
> > To unsubscribe, visit link above, or email
> > libav-user-request at ffmpeg.org with subject "unsubscribe".
> >
> >
> > _______________________________________________
> > Libav-user mailing list
> > Libav-user at ffmpeg.org
> > https://ffmpeg.org/mailman/listinfo/libav-user
> >
> > To unsubscribe, visit link above, or email
> > libav-user-request at ffmpeg.org with subject "unsubscribe".
>
>
> _______________________________________________
> Libav-user mailing list
> Libav-user at ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/libav-user
>
> To unsubscribe, visit link above, or email
> libav-user-request at ffmpeg.org with subject "unsubscribe".
More information about the Libav-user
mailing list