[FFmpeg-trac] #3871(avcodec:new): FFmpeg MD5 output different with same data #2

[FFmpeg]

#3871: FFmpeg MD5 output different with same data #2
--------------------------------------+-----------------------------------
             Reporter:  ahthovaikied  |                    Owner:
                 Type:  defect        |                   Status:  new
             Priority:  normal        |                Component:  avcodec
              Version:  2.2.4         |               Resolution:
             Keywords:  md5 aac h264  |               Blocked By:
             Blocking:                |  Reproduced by developer:  0
Analyzed by developer:  0             |
--------------------------------------+-----------------------------------

Comment (by ahthovaikied):

 Replying to [comment:7 cehoyos]:
 > My question was: Why do want to calculate the MD5 of the demuxer output?
 What kind of bugs (problems, regressions) are you hoping to find or to
 avoid?
 I am not doing this to find regressions. For reasons I won't develop here,
 I need to calculate a checksum unique for media streams of a file, and I
 need to reproduce the calculation on various environments (different CPUs
 and build options, I can freeze the FFmpeg version however). This allows
 me for example to detect a single bit corruption of the file.
 I could calculate the MD5 of the whole file, but:
 * I need to embed the MD5 in the file once it's calculated
 * I want to be able to alter metadata without changing the MD5

 Replying to [comment:7 cehoyos]:
 > > For a same input file, I expect the MD5 calculation to be identical
 across different machines unless I alter the file by adding, removing,
 modifying or reordering streams.
 >
 > As said, this is missing several conditions like FFmpeg version and
 compilation options.
 >
 > > The MD5 calculation should also be reproducible across different
 FFmpeg versions, otherwise I think it's a bug (fixed or introduced).
 >
 > Why?
 > As said, behaviour changes are possible without a bug being fixed or
 introduced.
 > This is of course different for decoder output if a specification exists
 that requests bitexact output as for H.264 or a sample implementation as
 for VP8.
 >
 > > Replying to [comment:2 cehoyos]:
 > > > I didn't try to reproduce yet, but at least for some input files you
 can certainly get different md5 outputs with your exact command line if
 you are using different configure lines. The md5 values may also change
 depending on the version you test (again without changing the command
 line). All this cannot be surprising so I wonder now what exactly you want
 to test...
 > > I am not following you here. Why should the FFmpeg build configuration
 have any influence on the MD5 produced?
 >
 > Since libavformat (the demuxer) depends on libavcodec you shouldn't be
 surprised that demuxers produce different output depending on the
 compilation options used.
 >
 > > I do not decode the streams, only feed them through the MD5
 calculation. If I have the MKV demuxer and the ability to calculate the
 MD5 in an FFmpeg build, it should absolutely ALWAYS produce the same MD5.
 >
 > No.
 OK, it seems we disagree on only one thing:
 You are saying that a demuxer does not necessarily have a bit exact,
 reproducible output, and that it could change depending on build options
 or CPU, am I correct?
 I can perfectly understand that a '''decoder''' can have a non
 reproducible output, be cause of floating point errors, integer vs
 floating point calculations, or because a decoder decides to ignore that
 type of frame to go faster, etc.
 But how can a '''demuxer''' not be bit exact?
 Isn't there a standard that precisely describes for example that in a
 Matroska file, if byte w has value x, then the chunk from offset y to z is
 part of a video stream? Then how can the implementation have any latitude
 to ignore, add or change some bytes of that stream?

 Replying to [comment:7 cehoyos]:
 > Concerning the bisect: Did you find a version that produces the output
 you want and a version that produces a different output on the same system
 and with the same compilation options?
 Yes, see my comment above (https://trac.ffmpeg.org/ticket/3871#comment:4
 ), tests were done on the same core i7 PC, with the same build options.

--
Ticket URL: <https://trac.ffmpeg.org/ticket/3871#comment:8>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker