[FFmpeg-trac] #2280(avformat:new): establish versioning with framemd5 and framecrc outputs

FFmpeg trac at avcodec.org
Mon Feb 18 18:53:40 CET 2013


#2280: establish versioning with framemd5 and framecrc outputs
-------------------------------------+-------------------------------------
               Reporter:  dericed    |                  Owner:
                   Type:             |                 Status:  new
  enhancement                        |              Component:  avformat
               Priority:  normal     |               Keywords:
                Version:             |  framemd5,framecrc
  unspecified                        |               Blocking:
             Blocked By:             |  Analyzed by developer:  0
Reproduced by developer:  0          |
-------------------------------------+-------------------------------------
 Summary of the bug:

 I utilize the framemd5 output in a digital preservation context. For
 instance I'll produce a set of framemd5 files and traditional whole-file
 checksums for a set of video files and then write framemd5, the checksum,
 and the media out to LTO data tape. If, years later, I recover the video
 files and there is a checksum mismatch then I can compare the stored
 framemd5 to a newly produced framemd5 to determine which frame(s) have a
 digital error.

 I've found that the framemd5 format undergoes gradual changes so that a
 framemd5 output produced by an older ffmpeg may not match that of a newer
 ffmpeg. I propose to add a version number to framemd5 as a format and
 allow the version to be called in the command line.

 For instance if we consider the git-master version of framemd5 to be
 version 1 and then a future revision to the encoder makes a change to the
 format, then the new version would get a new version number which would
 become the default, and we could still create the older version by
 specifying -version 1.

 How to reproduce:

 Produce framemd5 output with 20101118 build of ffmpeg
 {{{
 ./ffmpeg -i sample.mov -f framemd5 sample_20101118_build.txt
 FFmpeg version SVN-r25762, Copyright (c) 2000-2010 the FFmpeg developers
   built on Nov 18 2010 04:11:35 with gcc 4.2.1 (Apple Inc. build 5659)
   configuration: --enable-gpl --enable-version3 --enable-libgsm --enable-
 pthreads --enable-libvorbis --enable-libtheora --enable-libspeex --enable-
 libmp3lame --enable-libopenjpeg --enable-libschroedinger --enable-
 libopencore_amrwb --enable-libopencore_amrnb --enable-libvpx --disable-
 decoder=libvpx --arch=x86 --enable-runtime-cpudetect --enable-libx264
 --enable-librtmp --extra-libs='-lrtmp -lssl -lcrypto' --target-os=darwin
 --cross-prefix=x86_64-apple-darwin10- --cc='ccache x86_64-apple-
 darwin10-gcc'
   libavutil     50.33. 0 / 50.33. 0
   libavcore      0.13. 0 /  0.13. 0
   libavcodec    52.96. 0 / 52.96. 0
   libavformat   52.84. 0 / 52.84. 0
   libavdevice   52. 2. 2 / 52. 2. 2
   libavfilter    1.62. 0 /  1.62. 0
   libswscale     0.12. 0 /  0.12. 0
 Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'sample.mov':
   Metadata:
     major_brand     : qt
     minor_version   : 512
     compatible_brands: qt
     encoder         : Lavf54.63.100
   Duration: 00:00:00.20, start: 0.000000, bitrate: 1273 kb/s
     Stream #0.0(eng): Video: h264, yuv420p, 640x480 [PAR 1:1 DAR 4:3],
 1238 kb/s, 25 fps, 25 tbr, 12800 tbn, 50 tbc
 [buffer @ 0x101301aa0] w:640 h:480 pixfmt:yuv420p
 Output #0, framemd5, to 'sample_20101118_build.txt':
   Metadata:
     encoder         : Lavf52.84.0
     Stream #0.0(eng): Video: rawvideo, yuv420p, 640x480 [PAR 1:1 DAR 4:3],
 q=2-31, 200 kb/s, 90k tbn, 25 tbc
 Stream mapping:
   Stream #0.0 -> #0.0
 Press [q] to stop encoding
 frame=    5 fps=  0 q=0.0 Lsize=       0kB time=0.20 bitrate=  10.0kbits/s
 video:2250kB audio:0kB global headers:0kB muxing overhead -99.989193%
 }}}

 and produce the framemd5 output from the same source with git-master

 {{{
 ffmpeg -i sample.mov -f framemd5 sample_gitmaster.txt
 ffmpeg version 1.1.git Copyright (c) 2000-2013 the FFmpeg developers
   built on Feb 18 2013 12:16:42 with Apple clang version 3.1
 (tags/Apple/clang-318.0.61) (based on LLVM 3.1svn)
   configuration: --prefix=/usr/local/Cellar/ffmpeg/HEAD --enable-shared
 --enable-pthreads --enable-gpl --enable-version3 --enable-nonfree
 --enable-hardcoded-tables --enable-avresample --cc=cc --host-cflags=
 --host-ldflags= --enable-libx264 --enable-libfaac --enable-libmp3lame
 --enable-libxvid --enable-libfreetype --enable-libass --enable-ffplay
 --enable-libopenjpeg --extra-
 cflags='-I/usr/local/Cellar/openjpeg/1.5.1/include/openjpeg-1.5 '
   libavutil      52. 17.102 / 52. 17.102
   libavcodec     54. 92.100 / 54. 92.100
   libavformat    54. 63.100 / 54. 63.100
   libavdevice    54.  3.103 / 54.  3.103
   libavfilter     3. 38.103 /  3. 38.103
   libswscale      2.  2.100 /  2.  2.100
   libswresample   0. 17.102 /  0. 17.102
   libpostproc    52.  2.100 / 52.  2.100
 Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'sample.mov':
   Metadata:
     major_brand     : qt
     minor_version   : 512
     compatible_brands: qt
     encoder         : Lavf54.63.100
   Duration: 00:00:00.20, start: 0.000000, bitrate: 1273 kb/s
     Stream #0:0(eng): Video: h264 (High) (avc1 / 0x31637661), yuv420p,
 640x480 [SAR 1:1 DAR 4:3], 1238 kb/s, 25 fps, 25 tbr, 12800 tbn, 50 tbc
     Metadata:
       handler_name    : DataHandler
 Output #0, framemd5, to 'sample_gitmaster.txt':
   Metadata:
     major_brand     : qt
     minor_version   : 512
     compatible_brands: qt
     encoder         : Lavf54.63.100
     Stream #0:0(eng): Video: rawvideo (I420 / 0x30323449), yuv420p,
 640x480 [SAR 1:1 DAR 4:3], q=2-31, 200 kb/s, 25 tbn, 25 tbc
     Metadata:
       handler_name    : DataHandler
 Stream mapping:
   Stream #0:0 -> #0:0 (h264 -> rawvideo)
 Press [q] to stop, [?] for help
 frame=    5 fps=0.0 q=0.0 Lsize=       0kB time=00:00:00.20 bitrate=
 16.5kbits/s
 video:2250kB audio:0kB subtitle:0 global headers:0kB muxing overhead
 -99.982118%
 }}}

 cat sample_gitmaster.txt
 {{{
 #tb 0: 1/25
 0,          0,          0,        1,   460800,
 3108af6b1d4031964e8d704dff24ea9d
 0,          1,          1,        1,   460800,
 889bb8610de6cdf5b6d756e7ebf0d0fb
 0,          2,          2,        1,   460800,
 030e150d6c13c3acd3b396c4630c87dd
 0,          3,          3,        1,   460800,
 865d8fede003c0ddaac6457f40290144
 0,          4,          4,        1,   460800,
 6081e7d4e77c23818d9e64b3d4076d7a
 }}}

 cat sample_20101118_build.txt
 {{{
 0, 0, 460800, 3108af6b1d4031964e8d704dff24ea9d
 0, 3600, 460800, 889bb8610de6cdf5b6d756e7ebf0d0fb
 0, 7200, 460800, 030e150d6c13c3acd3b396c4630c87dd
 0, 10800, 460800, 865d8fede003c0ddaac6457f40290144
 0, 14400, 460800, 6081e7d4e77c23818d9e64b3d4076d7a
 }}}

 The framemd5 reports are different but not versioned. I propose an output
 like:
 {{{
 #format: framemd5
 #version: 1.0.0
 #tb 0: 1/25
 #stream_index, packet_dts, packet_pts, packet_duration, packet_size, MD5
 0,          0,          0,        1,   460800,
 3108af6b1d4031964e8d704dff24ea9d
 0,          1,          1,        1,   460800,
 889bb8610de6cdf5b6d756e7ebf0d0fb
 0,          2,          2,        1,   460800,
 030e150d6c13c3acd3b396c4630c87dd
 0,          3,          3,        1,   460800,
 865d8fede003c0ddaac6457f40290144
 0,          4,          4,        1,   460800,
 6081e7d4e77c23818d9e64b3d4076d7a
 }}}

 I added the column headers as well. Since the columns themselves may
 change over time I think placing the column names in the header makes the
 format more self-descriptive.

-- 
Ticket URL: <https://ffmpeg.org/trac/ffmpeg/ticket/2280>
FFmpeg <http://ffmpeg.org>
FFmpeg issue tracker


More information about the FFmpeg-trac mailing list