[FFmpeg-trac] #7088(ffmpeg:new): Dropped video frames from iOS 11 Screen Record Videos lead to an Unsynced video

FFmpeg trac at avcodec.org
Sat Mar 17 00:03:10 EET 2018


#7088: Dropped video frames from iOS 11 Screen Record Videos lead to an Unsynced
video
-------------------------------------+-------------------------------------
             Reporter:  jonlam92     |                     Type:  defect
               Status:  new          |                 Priority:  important
            Component:  ffmpeg       |                  Version:  3.2.4
             Keywords:  ios 11,      |               Blocked By:
  iphone, quicktime, ffmpeg,         |  Reproduced by developer:  0
  avconv, negative start timestamp,  |
  map, mp4, mpeg, copy               |
             Blocking:               |
Analyzed by developer:  0            |
-------------------------------------+-------------------------------------
 It seems like ffmpeg does not know how to handle videos that were recorded
 on iOS 11 through the native screen recording functionality.

 We were taking the video stream and copying the second audio stream (from
 the microphone) and stitch it together so web players would be able to
 handle the audio portion properly.

 We discovered that frames were dropped at the beginning of the converted
 video, but the audio still began on time. It seems like everything before
 the 2nd keyframe is dropped, as it can't locate the first keyframe. We
 believe this occurs due to a negative start timestamp on the video.

 How to reproduce:
 {{{
 ffmpeg -i input.mp4 -acodec copy -vcodec copy -map 0:v:0 -map 0:a:1? -f
 mp4 output.mp4
 }}}

 Output from `ffmpeg -i input.mp4`
 {{{
 ffmpeg version 3.4.2 Copyright (c) 2000-2018 the FFmpeg developers
   built with Apple LLVM version 9.0.0 (clang-900.0.39.2)
   configuration: --prefix=/usr/local/Cellar/ffmpeg/3.4.2 --enable-shared
 --enable-pthreads --enable-version3 --enable-hardcoded-tables --enable-
 avresample --cc=clang --host-cflags= --host-ldflags= --disable-jack
 --enable-gpl --enable-libmp3lame --enable-libx264 --enable-libxvid
 --enable-opencl --enable-videotoolbox --disable-lzma
   libavutil      55. 78.100 / 55. 78.100
   libavcodec     57.107.100 / 57.107.100
   libavformat    57. 83.100 / 57. 83.100
   libavdevice    57. 10.100 / 57. 10.100
   libavfilter     6.107.100 /  6.107.100
   libavresample   3.  7.  0 /  3.  7.  0
   libswscale      4.  8.100 /  4.  8.100
   libswresample   2.  9.100 /  2.  9.100
   libpostproc    54.  7.100 / 54.  7.100
 Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'input.mp4':
   Metadata:
     major_brand     : mp42
     minor_version   : 1
     compatible_brands: mp41mp42isom
     creation_time   : 2018-03-16T18:56:20.000000Z
   Duration: 00:03:54.07, start: -0.015941, bitrate: 2456 kb/s
     Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661),
 yuvj420p(pc), 1080x1920, 2319 kb/s, 23.03 fps, 60 tbr, 600 tbn, 1200 tbc
 (default)
     Metadata:
       creation_time   : 2018-03-16T18:56:21.000000Z
       handler_name    : Core Media Video
     Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz,
 stereo, fltp, 2 kb/s (default)
     Metadata:
       creation_time   : 2018-03-16T18:56:21.000000Z
       handler_name    : Core Media Audio
     Stream #0:2(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz,
 stereo, fltp, 103 kb/s (default)
     Metadata:
       creation_time   : 2018-03-16T18:56:21.000000Z
       handler_name    : Core Media Audio
 }}}

 Output of this video has dropped video frames (approximately 10 seconds
 worth), while keeping the audio. This leads to an unsynced video. Even
 with flags like `-copyts` and other fixes we found online, we still ended
 up with dropped frames at the start of the video, i.e. a black screen.

 We tried a fork of ffmpeg called libav, which seems to handle this file
 properly and doesn’t drop the frames at the start of the output video,
 between the first and second keyframe.

 Output from `avconv -i input.mp4`
 {{{
 avconv version 12.3, Copyright (c) 2000-2018 the Libav developers
   built on Feb 13 2018 11:14:42 with Apple LLVM version 9.0.0
 (clang-900.0.39.2)
 Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'input.mp4':
   Metadata:
     major_brand     : mp42
     minor_version   : 1
     compatible_brands: mp41mp42isom
     creation_time   : 2018-03-16 18:56:20
   Duration: 00:03:56.66, start: 0.062381, bitrate: 2429 kb/s
     Stream #0:0(und): Video: h264 (High) [avc1 / 0x31637661]
       yuvj420p, pc, 1080x1920, 2319 kb/s
       46.92 fps, 600 tbn (default)
     Metadata:
       creation_time   : 2018-03-16 18:56:21
       handler_name    : Core Media Video
     Stream #0:1(und): Audio: aac (LC) [mp4a / 0x6134706D]
       44100 Hz, stereo, fltp, 2 kb/s (default)
     Metadata:
       creation_time   : 2018-03-16 18:56:21
       handler_name    : Core Media Audio
     Stream #0:2(und): Audio: aac (LC) [mp4a / 0x6134706D]
       44100 Hz, stereo, fltp, 103 kb/s (default)
     Metadata:
       creation_time   : 2018-03-16 18:56:21
       handler_name    : Core Media Audio
 }}}

 Notice the start timestamps for the same video is different.

 ffmpeg start timestamp: ` -0.015941`
 avconv start timestamp: `0.062381`

--
Ticket URL: <https://trac.ffmpeg.org/ticket/7088>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker


More information about the FFmpeg-trac mailing list