[FFmpeg-devel] core infrastructure badge for FFmpeg
Ganesh Ajjanagadde
gajjanag at mit.edu
Wed Jul 6 00:45:19 EEST 2016
05.07.2016, 17:29, "Michael Niedermayer" <michael at niedermayer.cc>:
> On Mon, Jul 04, 2016 at 09:15:27PM -0400, Ganesh Ajjanagadde wrote:
>> 04.07.2016, 15:55, "Ronald S. Bultje" <rsbultje at gmail.com>:
>> > Hi,
>> >
>> > On Mon, Jul 4, 2016 at 3:44 PM, Ganesh Ajjanagadde <gajjanag at mit.edu> wrote:
>> >
>> >> 04.07.2016, 15:36, "Ronald S. Bultje" <rsbultje at gmail.com>:
>> >> > Hi,
>> >> >
>> >> > On Mon, Jul 4, 2016 at 3:29 PM, Ganesh Ajjanagadde <gajjanag at mit.edu>
>> >> wrote:
>> >> >
>> >> >> 04.07.2016, 10:33, "Clément Bœsch" <u at pkh.me>:
>> >> >> > On Mon, 4 Jul 2016 at 13:41 Ganesh Ajjanagadde <gajjanag at mit.edu>
>> >> >> wrote:
>> >> >> >> Hi,
>> >> >> >>
>> >> >> >> https://bestpractices.coreinfrastructure.org/.
>> >> >> >>
>> >> >> >> Thoughts on getting this done for FFmpeg?
>> >> >> >
>> >> >> > Any thing we need to adjust in the project? I don't see much things
>> >> to
>> >> >> > change by looking at
>> >> >> https://bestpractices.coreinfrastructure.org/projects/1
>> >> >>
>> >> >> I could not either.
>> >> >> It was something I noticed a week back, judged low-hanging and of some
>> >> >> benefit to the project.
>> >> >> I thus am willing to do it, provided there are no objections.
>> >> >
>> >> > I think if any changes are necessary to the website or documentation or
>> >> > code, these would simply go through the relevant review process, right?
>> >> Or
>> >> > am I missing something?
>> >>
>> >> True. At the moment, it seems like the relevant forms can be filled in
>> >> directly from existing information,
>> >> and no changes are necessary.
>> >>
>> >> I can send a screenshot of the filled out form before submitting if people
>> >> want.
>> >
>> > Oh I see, I misunderstood. I personally don't have objections to that.
>>
>> Turns out that the first few pages are easy to fill and we already achieve them.
>> The last few are much harder, and FFmpeg does not currently meet all of them.
>>
>> Here is the progress so far: https://bestpractices.coreinfrastructure.org/projects/235.
>>
>> I have filled them to the best of my ability.
>> Here is a summary of where we fail to meet the criteria,
>> and some suggestions for what we could do.
>>
>> The "MUST" constraints are the ones where FFmpeg needs to improve,
>> assuming the project wishes to get to 100%.
>
>> 1. The project MUST have evidence that such tests are being added in the most recent major changes to the project.
>> Suggestion: enforce tests for all new filters, muxers, etc. At least 6 months back, this was not enforced for lavfi.
>
> The details for this reads "Major functionality would typically be mentioned in the ChangeLog. (Perfection is not required, merely evidence that tests are typically being added in practice.) "
>
> diffstat between 3.0 and 3.1 of make fatelist shows
> fatelist-3.1 | 281 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++--
> 1 file changed, 275 insertions(+), 6 deletions(-)
>
> so there are 269 more tests in 3.1 than 3.0
>
> limiting this to filters shows
> grep filter fatelist-3.0> fatelist-3.0-filter
> grep filter fatelist-3.1> fatelist-3.1-filter
> diff -u fatelist-3.0-filter fatelist-3.1-filter | diffstat
> fatelist-3.1-filter | 25 +++++++++++++++++++++++++
> 1 file changed, 25 insertions(+)
>
> we have 15 filters in the 3.0->3.1 changelog if i didnt miscount
>
> so i would say we are maybe a bit behind with adding tests but they
> do get typically added eventually even for libavfilter.
> No question, it would be better if tests would be added quicker ...
I do not doubt this, but at the moment we do not enforce it.
Do you see any trouble in enforcing this requirement from major release to next major release?
This should provide sufficient time.
Really, all it requires is changing the "informal/soft constraint" to a "hard" constraint,
and accordingly noted in the dev docs.
I treated the "must" here as a hard constraint.
>
> [...]
>
>> 5. The project security mechanisms MUST use default keylengths that at least meet the NIST minimum requirements through the year 2030 (as stated in 2012).
>> Suggestion: add --enable-unsafe-crypt to configure, and by default not enable it.
>> Change API's and functions accordingly, document it.
>> Note: strictly speaking, per the sentence above, it is ok as we do not really have "default" keylengths.
>> But the extended rationale shows why we fail.
>
> "For example, FFmpeg's DES API accepts 64 bit keys, and there is no configuration option for disabling this."
>
> The low level DES API may be part of a security mechanism, for example
> 3DES used for encrypting something which is having sufficiently sized
> keys. Also the low level APIs are used for format interoperability
> outside any "security mechanisms"
Ok.
>
>> 6. The default project security mechanisms MUST NOT depend on cryptographic algorithms that are broken
>> (e.g., MD4, MD5, single DES, RC4, or Dual_EC_DRBG).
>> Suggestion: same as 5 above.
>
> "For example, FFmpeg provides a DES API, and there is no configuration option for disabling this."
>
> the point above speaks about the "default", i would argue a user using
> the DES API explicitly has made a choice and is not using a default
>
>> 7. The project security mechanisms SHOULD NOT by default depend on cryptographic algorithms with known serious weaknesses (e.g., SHA-1).
>> Suggestion: same as 5 above.
>
>> 8. At least one static code analysis tool MUST be applied to any proposed major production release of the software before its release,
>> if there is at least one FLOSS tool that implements this criterion in the selected language.
>> Suggestion: force the use of coverity before release. AFAIK, we do not clean this up fully before releases.
>
> we do run coverity around the release time generally, the point does
> not seem to require strictly fixing every warning it finds, we fix most
>
> more regular testing and testing with more tools would be a very good
> idea.
> Some tools though are very verbose and the warnings produced are of
> rather low quality, such tools should be avoided as they would
> misdirect manhours ...
>
>> 9. All medium and high severity exploitable vulnerabilities discovered with static code analysis MUST be fixed in a timely way after they are confirmed.
>> Suggestion: this is related to 8, same idea.
>
> which "exploitable vulnerabilities discovered" have not been fixed ?
I don't know. Since you seem to think this is done,
sure, I will change it.
>
>> 10. It is SUGGESTED that static source code analysis occur on every commit or at least daily.
>> Suggestion: ignore, or if we have the resources and it is deemed worthy, get more regular coverity reports.
>
>> 11. It is SUGGESTED that at least one dynamic analysis tool be applied to any proposed major production release of the software before its release.
>> Suggestion: agree upon some fuzzer, and run it on a variety of inputs before release. Can be a fate-fuzz.
>
> There are some researchers which regularly check FFmpeg with fuzzers
> this does not match our release shedule though
I can add info to that effect, but as you point out this does not match the wording.
Any chance we can get a regular fuzzer?
>
>> 12. It is SUGGESTED that if the software is application-level software written using a memory-unsafe language (e.g., C or C++)
>> then at least one dynamic tool (e.g., a fuzzer or web application scanner)
>> be routinely used with a mechanism to detect memory safety problems such as buffer overwrites.
>> Suggestion: AFAIK this is not regular, simply get someone to regularly run a fuzzer, especially before release.
>>
>> 13. All medium and high severity exploitable vulnerabilities discovered with dynamic code analysis MUST be fixed in a timely way after they are confirmed.
>> Suggestion: Force the fixing of fuzzed problems before release, make this part of release checklist.
>> No idea if we have the resources to enforce this without delaying a release too much.
>>
>> There are some others where I lack knowledge, and either have not filled yet or am not satisfied with my response.
>> Help would be greatly appreciated.
>>
>> 14. The project MUST have at least one primary developer who knows how to design secure software.
>> I marked this as yes, and gave Nicholas and Michael as examples.
>> I apologize to any who are offended by it.
>> Alternative wording would be nice.
>>
>> 15. The project SHOULD implement perfect forward secrecy for key agreement protocols
>> so a session key derived from a set of long-term keys cannot be compromised if one of the long-term keys is compromised in the future.
>> No idea
>
>> 16. If passwords are stored for authentication of external users, the project MUST store them as iterated hashes
>> with a per-user salt by using a key stretching (iterated) algorithm (e.g., PBKDF2, Bcrypt or Scrypt).
>> No idea
>
> I dont think we store any "passwords for authentication of external users"
Ok, will update.
>
>> 17. There MUST be no unpatched vulnerabilities of medium or high severity that have been publicly known for more than 60 days.
>> Do we guarantee this?
>
> I think we fix public vulnerabilities much quicker than 60 days in
> practice
Ok, will update.
>
>> 20. (Future criterion) It is SUGGESTED that the project have a reproducible build.
>> AFAIK we can do this easily by fixing on a particular toolchain,
>> unless I am missing something.
>
> I think andreas worked on reproducible builds as required by debian
> for something, thus i would assume that does work
> this should be in the discussions around the src/ directory IIRC
I will contact Andreas and/or check this, and update accordingly.
>
> Thanks alot for your work!
Thanks for your help!
>
> [...]
>
> --
> Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB
>
> Breaking DRM is a little like attempting to break through a door even
> though the window is wide open and the only thing in the house is a bunch
> of things you dont want and which you would get tomorrow for free anyway
> ,
>
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org
> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
More information about the ffmpeg-devel
mailing list