[FFmpeg-trac] #7088(ffmpeg:new): Dropped video frames from iOS 11 Screen Record Videos lead to an Unsynced video
FFmpeg
trac at avcodec.org
Sat Mar 17 00:03:10 EET 2018
#7088: Dropped video frames from iOS 11 Screen Record Videos lead to an Unsynced
video
-------------------------------------+-------------------------------------
Reporter: jonlam92 | Type: defect
Status: new | Priority: important
Component: ffmpeg | Version: 3.2.4
Keywords: ios 11, | Blocked By:
iphone, quicktime, ffmpeg, | Reproduced by developer: 0
avconv, negative start timestamp, |
map, mp4, mpeg, copy |
Blocking: |
Analyzed by developer: 0 |
-------------------------------------+-------------------------------------
It seems like ffmpeg does not know how to handle videos that were recorded
on iOS 11 through the native screen recording functionality.
We were taking the video stream and copying the second audio stream (from
the microphone) and stitch it together so web players would be able to
handle the audio portion properly.
We discovered that frames were dropped at the beginning of the converted
video, but the audio still began on time. It seems like everything before
the 2nd keyframe is dropped, as it can't locate the first keyframe. We
believe this occurs due to a negative start timestamp on the video.
How to reproduce:
{{{
ffmpeg -i input.mp4 -acodec copy -vcodec copy -map 0:v:0 -map 0:a:1? -f
mp4 output.mp4
}}}
Output from `ffmpeg -i input.mp4`
{{{
ffmpeg version 3.4.2 Copyright (c) 2000-2018 the FFmpeg developers
built with Apple LLVM version 9.0.0 (clang-900.0.39.2)
configuration: --prefix=/usr/local/Cellar/ffmpeg/3.4.2 --enable-shared
--enable-pthreads --enable-version3 --enable-hardcoded-tables --enable-
avresample --cc=clang --host-cflags= --host-ldflags= --disable-jack
--enable-gpl --enable-libmp3lame --enable-libx264 --enable-libxvid
--enable-opencl --enable-videotoolbox --disable-lzma
libavutil 55. 78.100 / 55. 78.100
libavcodec 57.107.100 / 57.107.100
libavformat 57. 83.100 / 57. 83.100
libavdevice 57. 10.100 / 57. 10.100
libavfilter 6.107.100 / 6.107.100
libavresample 3. 7. 0 / 3. 7. 0
libswscale 4. 8.100 / 4. 8.100
libswresample 2. 9.100 / 2. 9.100
libpostproc 54. 7.100 / 54. 7.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'input.mp4':
Metadata:
major_brand : mp42
minor_version : 1
compatible_brands: mp41mp42isom
creation_time : 2018-03-16T18:56:20.000000Z
Duration: 00:03:54.07, start: -0.015941, bitrate: 2456 kb/s
Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661),
yuvj420p(pc), 1080x1920, 2319 kb/s, 23.03 fps, 60 tbr, 600 tbn, 1200 tbc
(default)
Metadata:
creation_time : 2018-03-16T18:56:21.000000Z
handler_name : Core Media Video
Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz,
stereo, fltp, 2 kb/s (default)
Metadata:
creation_time : 2018-03-16T18:56:21.000000Z
handler_name : Core Media Audio
Stream #0:2(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz,
stereo, fltp, 103 kb/s (default)
Metadata:
creation_time : 2018-03-16T18:56:21.000000Z
handler_name : Core Media Audio
}}}
Output of this video has dropped video frames (approximately 10 seconds
worth), while keeping the audio. This leads to an unsynced video. Even
with flags like `-copyts` and other fixes we found online, we still ended
up with dropped frames at the start of the video, i.e. a black screen.
We tried a fork of ffmpeg called libav, which seems to handle this file
properly and doesn’t drop the frames at the start of the output video,
between the first and second keyframe.
Output from `avconv -i input.mp4`
{{{
avconv version 12.3, Copyright (c) 2000-2018 the Libav developers
built on Feb 13 2018 11:14:42 with Apple LLVM version 9.0.0
(clang-900.0.39.2)
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'input.mp4':
Metadata:
major_brand : mp42
minor_version : 1
compatible_brands: mp41mp42isom
creation_time : 2018-03-16 18:56:20
Duration: 00:03:56.66, start: 0.062381, bitrate: 2429 kb/s
Stream #0:0(und): Video: h264 (High) [avc1 / 0x31637661]
yuvj420p, pc, 1080x1920, 2319 kb/s
46.92 fps, 600 tbn (default)
Metadata:
creation_time : 2018-03-16 18:56:21
handler_name : Core Media Video
Stream #0:1(und): Audio: aac (LC) [mp4a / 0x6134706D]
44100 Hz, stereo, fltp, 2 kb/s (default)
Metadata:
creation_time : 2018-03-16 18:56:21
handler_name : Core Media Audio
Stream #0:2(und): Audio: aac (LC) [mp4a / 0x6134706D]
44100 Hz, stereo, fltp, 103 kb/s (default)
Metadata:
creation_time : 2018-03-16 18:56:21
handler_name : Core Media Audio
}}}
Notice the start timestamps for the same video is different.
ffmpeg start timestamp: ` -0.015941`
avconv start timestamp: `0.062381`
--
Ticket URL: <https://trac.ffmpeg.org/ticket/7088>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker
More information about the FFmpeg-trac
mailing list