[FFmpeg-devel] Upgrade Trouble

Reinhard Tartler siretart
Fri Dec 4 21:24:46 CET 2009

Michael Niedermayer <michaelni at gmx.at> writes:

> what i suggest is that the linker behaves sanely without bandaids.
> what it does is a breadth first search from the application, what it
> should do is a breadth first search from the object where a symbol is
> used.
> iam no expert in all these corner cases of shared lib ABI compatibility
> but then its clear the others in this thread arent either.
> Possibly RTLD_DEEPBIND could solve the problem. Drepper is quite vocal
> on arguing against it which is a sign that this might be the right
> direction.

RTLD_DEEPBIND is a flag for dlopen(). We are not talking about
applications that load ffmpeg this way but are dynamically linked.

Moreover, that flag wouldn't help even if we would find some way to make
the dynamic linker (ld-linux.so or similar) use that flag. Since it
applies to intra-library symbol lookups only, it does not help solving
the problems caused by transitive dependency problems that we are
currently discussing in this thread.

> Either way, ffmpeg isnt the only thing that seems to have been hit by this
> A->B1,C1 B1->C2 symbol shadowing bug so i really think that a generic
> solution at the linker/loader level would be best. It might solve this
> problem once and for all.

Correct. other libraries that got hit by this problem started
implementing proper symbol versioning.  Prominent examples for this
include the gnu libc and libdb (berkely db).  A more complete explanation
why symbol versioning helps in the situation of is explained in Dreppers
DSO howto[1], section "3.3 ABI Versioning", p.35f.

> Of course iam not against other solutions if a linker/loader based solution
> is not possible. symbol versioning seems like an option, ...

Indeed, however, proper symbol versioning has additionally the advantage
that the ffmpeg libraries could properly prevent exporting non-public
symbols.  This has been previously attempted by Diego, but his patch was
unfortunately never finished.

Let's recapitulate, what is symbol versioning after all?  It basically
means that a special linker script is used to assign "version tags" to
symbols in an ELF DSO. That version script classifies symbols either as
"global" or "local":

  * local symbols are marked as 'hidden' and cannot be used outside the
    DSO.  This is an effective way to hide private symbols

  * global symbols are part of the public ABI

The "assignment" of tags is similar to mechanically adding prefixes to
symbol names. This effectively makes the assigned symbol part of the
symbol name and thus, part of the ABI! This means that symbol versioning
should be done carefully.

> ... though here some extra work is needed to manually patch up the
> corners like symbols that have been moved from lavf -> lavc.

Indeed. The approach to statically prefix the SONAME to each symbol is
trivial to implement, but leads to the situation that moving symbols
from lavf to lavc effectively results in an ABI break that would require
an SONAME bump.  This might be an acceptable compromise, but can only be
avoided by using a more sophisticated approach for symbol versioning.

The safest approach to avoid this would probably be a version script,
in that each maintainer needs to carfully maintain version numbers for
each publicly exported symbol.

> Bumping tha soname of a lib that has not changed its interface is something
> id like to avoid.

I absolutely agree.  This is the only way applications can benefit from
improvments in the library without needing to be recompiled!

> I dont think lavc/f exports parts of lavu, if they do that would need a
> soname bump of course.
> That said iam open to all solutions if the nicer ones are exhausted, the
> problem is real and a solution needed.

I'm happy to read that you take this problem seriously.  I'd be willing
to help with implementing symbol versioning in FFmpeg, if you can give
me hints what kind of work would be helpful.

> Also there is another aspect that should be mentioned, debians dependancy
> system is buggy if it does not detect this.
> I mean if the linker/loader used by debian cannot handle 2 libs with
> different major abi versions linked together it should not allow this
> situation to arise for any application.
> something like:
> lavu=123
> conflicts_when_loaded_togeher: lavu!=123

That's trivial to implement by making a potential libavutil50 package
declare a 'Breaks' or 'Conflicts' relationship with libavutil49.  Please
note that no packaging system can have the clairvoyance skills for
this.  I'm currently badly hit by ignorant 3rd party maintainers that
prefer to ignore this issue and just let users suffer and report bugs
against distro packages.

Anyway, this approach would require to recompile the world. However,
that would have an effect worse than bumping SONAME for all libraries:
Binaries that have been compiled against the new ffmpeg couldn't be
installed along with binaries not yet recompiled against the new ffmpeg.

[1] http://people.redhat.com/drepper/dsohowto.pdf

Reinhard Tartler, KeyID 945348A4

More information about the ffmpeg-devel mailing list