[FFmpeg-devel] [PATCH] ffv1enc: reduce stack usage.

[Reimar Döffinger]

On Tue, Sep 02, 2014 at 09:54:36PM +0200, Reimar Döffinger wrote:
> On Tue, Sep 02, 2014 at 09:28:27PM +0200, wm4 wrote:
> > I don't see anything wrong in the patch (well, maybe you should switch
> > the code to the "goto fail;" idiom). I was just wondering whether there
> > was a specific reason for this. Did it fail on a certain system?
> 
> Btw. while I did not test it, and ignoring HP-UX and AIX (16k and 96k
> stack it seems), OpenBSD 4.6 release notes say:
> Increased the default thread stack size to 256k for 32-bit hosts and to
> 512k on 64-bit hosts.
> 
> Before it was 64k, so it should fail on an OpenBSD system from 2009 :)

And sorry if I write far more than you asked for, but there is
also a security aspect to it.
Those large objects on stack are usually arrays.
If anything you do depends on user input, there is a risk
that you will have a buffer overflow.
At least without stack smashing protection (which I think we do not
enable ourselves, though many distributions do by default),
stack buffer overflows are near trivial to exploit.
A head buffer overflow is at least more work, plus you can surround
every allocation with guard pages if you are paranoid, which stops them
- even if at a possibly massive cost in (virtual and physical) memory usage.
I believe this was the reasoning behind the OpenBSD 64kB limit, stopping
people from putting buffers on the stack.