[FFmpeg-devel] [RFC] Sechole in gcc 3.3+ and 4.*

[Michael Niedermayer]

Hi

Lars T?uber, told me about a sechole in gcc 4.2+
http://www.heise.de/newsticker/GCC-optimiert-Sicherheitspruefungen-weg--/meldung/106097
http://www.kb.cert.org/vuls/id/162289

The issue described though is rather harmless.
That is i think a check like buf+len < buf makes little sense.

Out of boredom i tried

int func(int len){
    if(len+100 < len)
        return 1;
    return 0;
}

Just to find out that all versions of gcc i have installed except 2.95
optimize this away at -O2 or higher.

If one uses -fwrapv than gcc 4.* seems to leave the check in there but
3.4 does still remove it and 3.3 does not support -fwrapv.

AFAIK addition of signed integers is not an undefined operation in C.
Rather the exact representaton being implementation defined, that is
implementation must define if they are twos complement, ones complement
or sign+magnitude. Now we know everything is twos complement but gcc
does not seem to behave like that ...

I also failed to find a mentioning of signed integer addition in the
list of undefined things in the C standard, it only mentions
pointer+integer being undefined in some cases.

Comments welcome...

-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

While the State exists there can be no freedom; when there is freedom there
will be no State. -- Vladimir Lenin
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
URL: <http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/attachments/20080407/74040fa2/attachment.pgp>