[FFmpeg-devel] Memory leaks in Ffmpeg after changing vlc allocation approach

[Art Clarke]

Hi folks (well mostly Michael),

We have a fairly comprehensive memory leak test system for our software
here, and it's been impressive how solid ffmpeg has been in running through
it without leaking (way to go folks).  This is super important as our
current system design has Ffmpeg running inside a very-long-running server
process.

That said, Ffmpeg is now starting to show leaks for well-behaved
applications due to heap-allocated memory that cannot be freed via any
Ffmpeg API call.  Now the leaks themselves are one-time allocations that do
not grow, so it's not causing us problems.  But it is breaking our test
suite.

The specific change that caused this was made a few weeks ago to clean-up
how VLC tables were allocated to make it a lot cleaner and less hacky (see
bistream.c).  Unfortunately the new way of allocating still needs to keep
around static variables that now point to heap-allocated tables.  In our
memory allocation tests these show up as still-reachable-memory leaks on
exit:

For example:
==25405== 6,656 bytes in 5 blocks are still reachable in loss record 2 of 2
==25405==    at 0x40058FE: realloc (vg_replace_malloc.c:429)
==25405==    by 0x4749DFE: av_realloc (mem.c:110)
==25405==    by 0x4166AD3: ff_realloc_static (bitstream.c:47)
==25405==    by 0x4166E76: alloc_table (bitstream.c:122)
==25405==    by 0x4166EF7: build_table (bitstream.c:145)
==25405==    by 0x41672FE: build_table (bitstream.c:224)
==25405==    by 0x416744A: init_vlc_sparse (bitstream.c:295)
==25405==    by 0x432140B: h263_decode_init_vlc (h263.c:2910)
==25405==    by 0x431637C: ff_h263_decode_init (h263dec.c:117)

Now to be honest, there isn't much anyone can do about this currently since
after a call to av_register_all, any Ffmpeg call can be made that needs
those tables.

My specific question is have folks given any thoughts to creating a method
on the AV api such as: av_unregister_all()

Proposed design:
This method can be called by test-code doing memory-leak tests, and it would
have the purpose of calling any destruction code necessary for
pre-registered objects.  People who call this must guarantee they never call
another Ffmpeg function in that process.  This would allow the calls to
h263_decode_init_vlc (and any other function that allocates a global heap
object) to register clean-up handlers as follows:

av_register_static_object(vlc, destructor_to_be_fired_later);

and then when av_unregister_all is called, we could call all destructors.

Signatures might look like:
void av_unregister_all();
void av_register_static_object(void * obj, void
(destructor_to_be_fired*)(void*));

I'd be happy to hack this together but wanted to check first:
a) is there any other demand for this (if not, I'll just make sure we
suppress leaks as they show up in our test suite)?
b) is there any other purposed way to solve this?
c) does anyone read this far into an e-mail?

thoughts?

- Art