[FFmpeg-devel] [PATCH 00/13] [RFC] Reduce unnecessary recompilation

[Martin Storsjö]

On Wed, 23 Feb 2022, Martin Storsjö wrote:

> When updating the ffmpeg source, one quite often ends up in a situation
> where practically all of the codebase (or all of a library) gets rebuilt,
> due to updates to headers that are included in most files.
>
> In some cases, full rebuilds are warranted of course, but they could also
> be avoided in many cases - e.g. such as if the minor/micro version of
> a library has been bumped, or if a new component (codec, demuxer, filter
> etc) has been enabled (or removed if reconfiguring with an older source
> version). Very few source files are affected by exactly what the full
> library version is, or by the full list of enabled components.
>
> To avoid such rebuilds, I've got a proof of concept patchset that
> splits headers, so that most source files avoid including the bits that
> change often and that shouldn't affect how they are built.
>
> - The version.h headers are split into a separate version_major.h which
>  contains only the major library version, and accompanying FF_API_*
>  defines. The main library headers only include version_major.h, and
>  files that need the exact version (e.g. LIB<name>_VERSION* or
>  LIB<name>_IDENT) can include version.h explicitly. This is a minor
>  break of the public API though, as definitions that used to be available
>  no longer are.
>
>  This works mostly fine for most libraries, but there's little point in
>  splitting libavutil/version.h, because LIBAVUTIL_VERSION_INT is used
>  in every source file that defines an AVClass.
>
>  By splitting version.h, and update to the minor/micro version numbers
>  of all libraries except avutil now would require recompiling 30
>  files instead of 1653 before.
>
>  (This change also should lower the barrier to and reduce the risk of
>  forgetting to bump the version numbers, which one otherwise often
>  postpones while working on a patch, as it forces rebuilds.)
>
> - config.h is split into a separate config_components.h that includes the
>  list of enabled/disabled components (corresponding to $ALL_COMPONENTS
>  in configure). One could consider splitting up config.h even more, but
>  that probably gives less benefit compared to the amount of churn.
>
>  Surprisingly, a nontrivial number of source files do depend on the
>  state of specific encoders/decoders/components, so quite a few files
>  do end up requiring including config_components.h. (Also this change
>  can possibly break compilation of source files that require external
>  dependencies that I haven't tested.)
>
>  In practice, this reduces the number of rebuilt source files from
>  1979 to 193, if there's a change to the list of enabled components
>  but not to the rest of config.h.
>
> What do you think - is it worth the slight churn to avoid pointless
> rebuilds?

Ping - I never got any feedback on the general concept of this patchset; 
is either of the refactorings worthwhile?

I'll post a rebased version of the patchset, which also should fix the 
build errors that Michael noted. (It's quite expected that the last patch, 
splitting up config.h, will require changes in some conditionally compiled 
files that haven't been included in my test builds.)

// Martin