[FFmpeg-devel] Memory leaks in Ffmpeg after changing vlc allocation approach

[Michael Niedermayer]

On Thu, Jun 19, 2008 at 05:13:37PM -0400, Art Clarke wrote:
> Hi folks (well mostly Michael),
> 
> We have a fairly comprehensive memory leak test system for our software
> here, and it's been impressive how solid ffmpeg has been in running through
> it without leaking (way to go folks).  This is super important as our
> current system design has Ffmpeg running inside a very-long-running server
> process.
> 
> That said, Ffmpeg is now starting to show leaks for well-behaved
> applications due to heap-allocated memory that cannot be freed via any
> Ffmpeg API call.  Now the leaks themselves are one-time allocations that do
> not grow, so it's not causing us problems.  But it is breaking our test
> suite.
> 
> The specific change that caused this was made a few weeks ago to clean-up
> how VLC tables were allocated to make it a lot cleaner and less hacky (see
> bistream.c). 

> Unfortunately the new way of allocating still needs to keep
> around static variables that now point to heap-allocated tables.

No, go RTFsvnlog again!


>  In our
> memory allocation tests these show up as still-reachable-memory leaks on
> exit:
> 
> For example:
> ==25405== 6,656 bytes in 5 blocks are still reachable in loss record 2 of 2
> ==25405==    at 0x40058FE: realloc (vg_replace_malloc.c:429)
> ==25405==    by 0x4749DFE: av_realloc (mem.c:110)
> ==25405==    by 0x4166AD3: ff_realloc_static (bitstream.c:47)
> ==25405==    by 0x4166E76: alloc_table (bitstream.c:122)
> ==25405==    by 0x4166EF7: build_table (bitstream.c:145)
> ==25405==    by 0x41672FE: build_table (bitstream.c:224)
> ==25405==    by 0x416744A: init_vlc_sparse (bitstream.c:295)
> ==25405==    by 0x432140B: h263_decode_init_vlc (h263.c:2910)
> ==25405==    by 0x431637C: ff_h263_decode_init (h263dec.c:117)
> 
> Now to be honest, there isn't much anyone can do about this currently since
> after a call to av_register_all, any Ffmpeg call can be made that needs
> those tables.
> 

> My specific question is have folks given any thoughts to creating a method
> on the AV api such as: av_unregister_all()

Yes, theres no need for such a method at all. Because there should not be any
heap allocated tables in static variables. lavc* thread safety requires all
static tables to be constant thus their size is constant as well. Now when
their size is constant it should be
static array[123];
not
static *array= av_malloc();

Not all code in lav* has been changed, some still uses such mallocs, patches
removing such heap allocated statics are welcome. (note many small patches
prefered over 1 large to avoid you wasting time solving it incorrectly)


[...]
> b) is there any other purposed way to solve this?

see above


> c) does anyone read this far into an e-mail?

no, i started skiping randomly where you proposed to put the hacks back that
i removed.

[...]
-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

Awnsering whenever a program halts or runs forever is
On a turing machine, in general impossible (turings halting problem).
On any real computer, always possible as a real computer has a finite number
of states N, and will either halt in less than N cycles or never halt.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
URL: <http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/attachments/20080620/c5267bf5/attachment.pgp>