[FFmpeg-devel] Reintroducing FFmpeg to Debian

[Andreas Cadhalpun]

Hi Reinhard,

On 10.08.2014 15:10, Reinhard Tartler wrote:
> On Sun, Aug 10, 2014 at 3:01 AM, Matthias Urlichs <matthias at urlichs.de> wrote:
>> IMHO it's reasonable to expect core APIs to be upwards-compatible and keep
>> deprecated interfaces around for another release or two.
>
> This is exactly what Libav is doing: The deprecation process for
> symbols, APIs, enums, etc. takes *years*, because so many software
> packages in Debian and else where use them, and it is so believably
> painful to change them. Just have a look at the last two Libav
> transitions, and the massive amount of patches it took to get packages
> fixed and eventually to get Debian to the new Libav release.
>
> Now enter FFmpeg.

FFmpeg also has a deprecation process and keeps deprecated features 
around longer than Libav. For example, avcodec_encode_video, 
av_close_input_file and avcodec_decode_audio3 are still present in 
FFmpeg, but already removed from Libav.

> FFmpeg has a significant higher release frequency, (it seems to me
> about every 3-4 months), so that you would get a deprecation cycle
> that is considerably less than a year.

The deprecation cycle is not related to the release frequency, as many 
FFmpeg release are API/ABI backwards compatible with the previous one.

> In practice, the deprecation
> cycle more or less seems to match Libav's cycle, because at least
> right now, FFmpeg  tracks Libav's API. If that were not the case (and
> I promise you FFmpeg would stop tracking Libav as soon as it replaces
> Libav in Debian), I can almost guarantee [1] you that FFmpeg would
> very much prefer to resume to the deprecation cycle the project
> before: None, i.e., every piece of software is expected to keep up
> with FFmpeg's master branch for reasons Jean-Yves outlines.

I think you won't be able to keep that promise, because it wouldn't make 
much sense to stop merging changes from Libav (especially, if they are 
useful) after doing it for such a long time. Even in the unlikely event 
that this might happen, there is no reason to change the handling of 
deprecations.

> [1] at least statements such as
> https://ffmpeg.org/pipermail/ffmpeg-devel/2014-August/160876.html
> strongly suggest this (at least if you have followed  the
> libavresample/libswresample mess).

I'm understanding this mail differently: What Michael is explaining is 
that it is more difficult for FFmpeg to change things in libavresample 
than in libswresample, because Libav is unlikely to merge back changes, 
but FFmpeg tries to be compatible with Libav.

In reality, there hasn't been any backwards incompatible change in 
libswresample (still soversion 0), but there has been one in 
libavresample (now at soversion 1).

>>> Keeping your own static version is the only reasonable approach.
>>
>> That may be OK on Windows. However, a proper Linux distribution is supposed
>> to be an integrated whole and not a haphazard collection of programs, each
>> bringing along their own copy of core libraries and their own un- or
>> semi-fixed security problems.
>>
>
> BTW Jean-Yves outlines an approach that used to be very common on the
> past: Pick some particular snapshot of FFmpeg and maintain that in a
> downstream project, and expect users to use that because it is too
> much effort to keep up with FFmpeg's release frequency.

It is easy to 'keep up' with releases that are API/ABI compatible, which 
many FFmpeg releases are.
One doesn't even have to recompile dependent programs (if they are not 
buggy), one can just install the new version of the libraries.

> Prominent
> examples of projects that did this (and actually, still do) include
> xine-lib, mplayer, xbmc, and many more. This lead exactly to the mess
> I was talking about in my previous email: dozens of embedded
> code-copies that were accepted into the Debian archive.

As you must know, xine-lib and xbmc are - and mplayer was - compiled 
against the system version of Libav in Debian.

> Over many years, I've spearheaded a significant effort in Debian with
> packaging and in upstream with introducing a release culture in FFmpeg
> (as release manager) to get to somewhat same release frequencies, so
> that downstream projects at least had a chance to agree on common
> versions of FFmpeg. At the time of the split, I was worried that this
> work would have been in vain.

It appears your work has not have been in vain, as FFmpeg's current 
release culture takes into account that any backwards incompatible API 
change means a lot of work for everyone using it. I believe this is 
handled now much better than in the times before the 0.5 release.

If you are unhappy with how the releases are managed in FFmpeg, you can 
always send your concerns to ffmpeg-devel (and I think you still have 
commit rights for FFmpeg's git repository as well).

> Considering that most active developers
> of FFmpeg at that time (which coincidentally supported my approach to
> release management and frequency) joined what is now known as Libav, I
> continued my work as upstream release manager in Libav, because I
> consider Libav as much more suited for Debian than FFmpeg. Today, I
> still firmly believe that this was the right move for Debian as a
> project.

Conversely, I firmly believe, that the absence of FFmpeg from the 
archive is bad for Debian's users. The reasons for that can be found in 
my initial mail in this thread.

> I do strongly believe that projects that require people to use FFmpeg
> actually mean to use their own private fork (cf. the mythtv debacle),

As far as I know, MythTV is the only example for a program that cannot 
be compiled with a system version of FFmpeg and instead uses its own fork.

> and given the amount of packages in Debian, it would significantly
> much more effort to "port" (or "patch" as Andreas is phrasing it) them
> to some common version of FFmpeg than doing what we are doing now:

As I explained in my initial mail in this thread [1], most of these 
patches are only necessary for switching the packages to pkg-config to 
detect the FFmpeg linker flags.
If the FFmpeg in Debian would use the upstream library names, i.e. the 
same as Libav, these patches would not be necessary. One still can avoid 
soname clashes by using the --enable-raise-major configure option and 
thus bumping all major soversion by 100, but this is not done in order 
to not use the same namespace that Libav in Debian currently uses and 
because some programs (I only know of vlc) check for a maximum version 
of the libraries.

There are only a few other changes necessary, and most of them are also 
needed for Libav 10 (or even totally unrelated to FFmpeg/Libav).
For the benefit of everyone, here is a comprehensive list of those:
  * acoustid-fingerprinter: Still uses CodecID in a codepath not used,
    when compiled against Libav.
  * bino: Currently fails to build due to a gettext version mismatch [2].
  * dvswitch: Still uses CodecID (and also avcodec_encode_video, but
    that is still present in FFmpeg.) [3]
  * ffdiaporama: The libav10.patch introduces incompatible API
    (avfilter_graph_parse).
  * k3b: Still uses CodecID (and also av_close_input_file and
    avcodec_decode_audio3, but these are still present in FFmpeg.)
    Therefore the FFmpeg plugin has been disabled. [4]
  * libextractor: Misses build-dependencies for the FFmpeg plugin. [5]
  * miro: Still uses CodecID (and also fails to build with Libav 10). [6]
  * mplayer2: MP_INPUT_BUFFER_PADDING_SIZE has to be increased to 32,
    because FFmpeg needs 32-byte padding.
  * netgen: Still uses CodecID, av_get_pict_type_char,
    avcodec_alloc_context and avcodec_open, but the linking with libav
    libraries has been disabled, see [7].
  * renpy: Still uses AVCODEC_MAX_AUDIO_FRAME_SIZE in a codepath not
    used, when compiled against Libav.
  * xbmc: The '--enable-libav-compat' needs to be removed, when compiling
    against FFmpeg.
  * zoneminder: Misses a build-dependency on libgcrypt11-dev. [8]

As you seem to not believe me, I'm going to send all the patches to the 
BTS, so that everyone can judge the amount of work for themselves.

> Making sure they work with the version of Libav's libavcodec.so
> implementation.

This seems like more work, because, for example, I don't know about any 
patches fixing the problems XMBC has with Libav and they don't seem to 
be straightforward to fix.

> This thread has shown a couple of examples that
> support this argument: Mythtv, but also mplayer that claims to work
> with a system libavcodec.so, which is true as long as it matches the
> version that is was built against.

MythTV shows nothing, because it uses its own fork and MPlayer works 
fine with a system FFmpeg. There is no need to recompile it, except for 
a major soversion bump, unless it has bugs I don't know about. Do you 
know of such bugs?

I'm using a version of MPlayer compiled against FFmpeg 2.2 with the 
FFmpeg libraries from 2.3 and it works for me.

> This is what makes mplayer so hard
> to package, and was ultimately the reason why I requested mplayer's
> removal (which is more than ironic, given that back then, I fought
> with ftp-master for many years to get it included into Debian in the
> first place).

I have read MPlayer's removal bug and it shows clearly that the only 
reason for its removal was, that it can't be compiled with Libav and 
upstream is not interested in working on this, because of the missing 
features in Libav.

> On a related note: Most  Libav developers are very tired of the
> constant flamewars and defamation attempts that arises from FFmpeg.

I'm also very tired of flamewars in general and happy that this 
discussion hasn't become one.

> Over years, Libav tries to convinced everyone by providing usable
> software releases.

That didn't succeed very well, at least not for me and all the other 
people for which Libav doesn't work correctly.

> Nevertheless, this particular debate is very
> worrisome: The silence from the Libav camp seems to not to be taken as
> consent. Quite the contrary is true.

Do you want to say here, that there are many people preferring Libav, 
who haven't participated in this discussion?

This might or might not be true, but I would welcome anyone, who makes a 
technical argument.

> How to proceed from here? TBH, I'm not sure. Ideally, both projects
> would find some common ground and "just merge" (however that would
> technically look like).

Indeed, that would be the ideal solution, ...

> However, this very debate within Debian shows
> that this is unlikely to happen anytime soon: There is way to much
> disagreement on very fundamental questions that require agreement
> within a free software project, and the hostile and aggressive tone
> the majority of participants in this debate exhibit does not help with
> making progress on that front either.

... but I'm also under the impression that there is still too much bad 
blood between both upstreams for this to happen.

On 10.08.2014 16:18, Reinhard Tartler wrote:
 > On Fri, Aug 8, 2014 at 2:06 PM, Andreas Cadhalpun
 > <andreas.cadhalpun at googlemail.com> wrote:
 >> It would be nice, if you could also spent some time thinking about
 >> the possibility of having both FFmpeg and Libav in Debian.
 >
 > Please believe me that I did spend a lot of time thinking about this.
 > I firmly do not think that this is a reasonable approach. Please see
 > my other email on debian-devel as explanation.

To me it seems your other mail (above) doesn't mention the possibility 
of having both, only:
"FFmpeg would stop tracking Libav as soon as it replaces Libav in Debian"

But if you are opposed to having both and also opposed to letting FFmpeg 
replace Libav, it seems that it will be impossible to get a consensus here.

Therefore the technical committee will have to decide between:
a) having both FFmpeg and Libav in Debian
b) only having FFmpeg in Debian
c) only having Libav in Debian

As I haven't seen any convincing technical argument for only having 
Libav, I doubt the technical committee's decision will be c).

Best regards,
Andreas


1: I'm getting tired of this phrase...
2: https://bugs.debian.org/757280
3: https://bugs.debian.org/747868
4: https://bugs.debian.org/739312
5: https://bugs.debian.org/755810
6: https://bugs.debian.org/748861
7: https://bugs.debian.org/751344
8: https://bugs.debian.org/745819