[FFmpeg-devel] coverity testing of FFmpeg

[Timothy Gu]

On Sun, Nov 27, 2016 at 1:13 PM Philip Langdale <philipl at overt.org> wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On Sun, 27 Nov 2016 21:26:13 +0100
> Michael Niedermayer <michael at niedermayer.cc> wrote:
>
> > On Sun, Nov 27, 2016 at 11:00:21AM -0800, Philip Langdale wrote:
> > > -----BEGIN PGP SIGNED MESSAGE-----
> > > Hash: SHA1
> > >
> > > On Sat, 26 Nov 2016 23:55:17 +0100
> > > Michael Niedermayer <michael at niedermayer.cc> wrote:
> > >
> > > > Hi all
> > > >
> > > > The machine on which the coverity stuff is is old, both hw and OS.
> > > > the OS will no longer get security updates in a few months and
> > > > the hw does not always boot and its switched off most of the time.
> > > > and it has no other use anymore than running coverity. Ive tried
> > > > to find someone a while ago to take coverity testing over and i
> > > > thought timothy would maybe do it but he seems to not have had any
> > > > time to look into it ...
> > > > and de facto ive not run it regularly in the recent months.
> > > > So this is kinda a louder announcement that if you care about
> > > > coverity testing, you need to do something ...
> > > >
> > > > thx
> > > >
> > > > PS: work would involve installing every optional dependancy of
> > > > FFmpeg (and keep them updated as needed)
> > > > and regularly either manually or automatically git pull, build
> > > > with the coverity tools and upload Its not a huge amount of work
> > > > but it is work
> > > >
> > >
> > > Hi Michael,
> > >
> > > I think we could do this using travis-ci.
> > >
> > > https://scan.coverity.com/travis_ci
> > >
> > > travis can be directly connected to the github mirror and then we
> > > set up a coverity job as covered in this doc.
> > >
> > > It wouldn't even need to be configured to build - just run the
> > > coverity scan. I'd be happy to investigate this, but I'd need admin
> > > permissions on github to configure travis integration.
> >
> > The way coverity works is by wraping around the compiler or something
> > gether lots of info and that is then uploaded
> > so you need configure and build
> >
> > for coverity to check all optional code it needs all the headers
> > from nvidia, sdl, x11 to the most obscure.
> > if they arent installed coverity wouldnt be able to test these
> > components as the code would not be complete
> >
> > so in whatever envirnment this is done all optional dependancies
> > must be installed one way or another.
> >
> > i assume in the example on https://scan.coverity.com/travis_ci
> > build_command_prepend: ./configure
> > would be a very long command with many --enable*
> >
> > also therese that limit:
> > https://scan.coverity.com/faq#frequency
> >     Up to 12 builds per week, with a maximum of 3 builds per day, for
> > projects with fewer than 100K lines of code Up to   8 builds per
> > week, with a maximum of 2 builds per day, for projects with 100K to
> > 500K lines of code Up to   4 builds per week, with a maximum of 1
> > build per day, for projects with 500K to 1 million lines of code Up
> > to   2 builds per week, with a maximum of 1 build per day, for
> > projects with more than 1 million lines of code
> >
> > so we cannot do coverity after every commit, a cronjob might be a
> > simpler choice that runs at the frequency that we can submit
> >
> > Is there still an advantage in travis ci / would that still be
> > preferred?
> >
> > [...]
>
> The recommended configuration on that page is to have it watch a branch
> and you push master to the branch when you want a scan. We can do that
> whenever we want - once a month or whatever. Right now you are doing it
> ad-hoc, and we could still do it ad-hoc. Equally, a cron job running
> somewhere could push to the branch on whatever frequency we want.
>
> The goal here is to remove the need for the dedicated build machine, not
> to improve any other part of the process - although there will be
> opportunities for that.
>
> I've looked at how you prepare a build environment in travis and it's
> simple to install dependencies; that won't be a problem.
>

Libav has had used Travis for a while now for FATE, and I personally have
attempted to set up the same for FFmpeg, without much success.

The problem with using Travis for Coverity (versus a local machine) is
still the issue of dependencies. While system-level deps like vdpau can be
installed easily, there are three classes of other deps that need to be
installed to make Coverity the most useful:

1. Dependencies very few people use, and therefore not packaged by Ubuntu
(e.g. libnut, libxavs, zimg)
2. Dependencies in active development, only the newest versions of which
are accepted by FFmpeg's build system (there currently isn't any examples
but it's not impossible to see this in the future)
3. Proprietary dependencies, which may or may not currently be an issue
anymore. Philip and Timo, how easy is it to get cuda/nvenc/cuvid/npp to
compile.

Timothy