[FFmpeg-user] help with live capture and audio sync issues

Roger Pack rogerdpack2 at gmail.com
Wed Dec 23 01:37:48 CET 2015


I'm almost tempted to come up with a "sync" filter (unless one already
exists) that would force things back into sync (live streams that is)

On 9/3/15, Ricardo Kleemann <ricardo at americasnet.com> wrote:
> Hi,
>
> I have very inconsistent results with doing live capture. Sometimes the
> resulting file has perfect audio sync, but sometimes it doesn’t. I’ve tried
> different combinations but the results are never consistent.
>
> I am capturing video from a black magic design UltraStudio Mini Recorder and
> the audio from an external Mic.
>
> This is done on a MacBook Pro.
>
> Here’s the capture command:
>
> ffmpeg -y \
> 	-thread_queue_size 2048 -f avfoundation -async 1 -i none:'C-Media USB Audio
> Device' \
> 	-thread_queue_size 2048 -f decklink -vsync 0 -i 'UltraStudio Mini
> Recorder at 12':none\
> 	-c:v libx264 -preset veryfast -profile:v baseline -vf scale=640:360 \
> 	-pix_fmt yuv420p \
> 	-c:a libfdk_aac -b:a 80k \
> 	-map 1:1 -map 0:0 \
> 	video_tests/test-avsync.mp4
>
>
> Here’s the ffmpeg output:
>
> ffmpeg version N-73104-g7604358 Copyright (c) 2000-2015 the FFmpeg
> developers
>   built with Apple LLVM version 6.1.0 (clang-602.0.53) (based on LLVM
> 3.6.0svn)
>   configuration: --prefix=/usr/local --enable-gpl --enable-nonfree
> --enable-ffplay --enable-libass --enable-libfdk-aac --enable-libfreetype
> --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis
> --enable-libvpx --enable-libx264 --enable-libxvid --enable-decklink
> --extra-cflags=-I/Users/ptl/blackmagicsdk/Mac/include
> --extra-ldflags=-L/Users/ptl/blackmagicsdk/Mac/include
>   libavutil      54. 27.100 / 54. 27.100
>   libavcodec     56. 44.100 / 56. 44.100
>   libavformat    56. 38.100 / 56. 38.100
>   libavdevice    56.  4.100 / 56.  4.100
>   libavfilter     5. 17.100 /  5. 17.100
>   libswscale      3.  1.101 /  3.  1.101
>   libswresample   1.  2.100 /  1.  2.100
>   libpostproc    53.  3.100 / 53.  3.100
> Input #0, avfoundation, from 'none:C-Media USB Audio Device':
>   Duration: N/A, start: 1057.070635, bitrate: 1411 kb/s
>     Stream #0:0: Audio: pcm_f32le, 44100 Hz, mono, flt, 1411 kb/s
> [decklink @ 0x7fdc23843600] Found Decklink mode 1920 x 1080 with rate
> 29.97(i)
> Guessed Channel Layout for  Input Stream #1.0 : stereo
> Input #1, decklink, from 'UltraStudio Mini Recorder at 12:none':
>   Duration: N/A, start: 0.000000, bitrate: 1536 kb/s
>     Stream #1:0: Audio: pcm_s16le, 48000 Hz, 2 channels, s16, 1536 kb/s
>     Stream #1:1: Video: rawvideo (UYVY / 0x59565955), uyvy422, 1920x1080, -5
> kb/s, 29.97 tbr, 1000k tbn, 29.97 tbc
> -async is forwarded to lavfi similarly to -af
> aresample=async=1:min_hard_comp=0.100000:first_pts=0.
> [libx264 @ 0x7fdc23845c00] using cpu capabilities: MMX2 SSE2Fast SSSE3
> SSE4.2 AVX
> [libx264 @ 0x7fdc23845c00] profile Constrained Baseline, level 3.0
> [libx264 @ 0x7fdc23845c00] 264 - core 144 r2533 c8a773e - H.264/MPEG-4 AVC
> codec - Copyleft 2003-2015 - http://www.videolan.org/x264.html - options:
> cabac=0 ref=1 deblock=1:0:0 analyse=0x1:0x111 me=hex subme=2 psy=1
> psy_rd=1.00:0.00 mixed_ref=0 me_range=16 chroma_me=1 trellis=0 8x8dct=0
> cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=0 threads=6
> lookahead_threads=2 sliced_threads=0 nr=0 decimate=1 interlaced=0
> bluray_compat=0 constrained_intra=0 bframes=0 weightp=0 keyint=250
> keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=10 rc=crf mbtree=1
> crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
> Output #0, mp4, to 'video_tests/test-avsync.mp4':
>   Metadata:
>     encoder         : Lavf56.38.100
>     Stream #0:0: Video: h264 (libx264) ([33][0][0][0] / 0x0021), yuv420p,
> 640x360, q=-1--1, 29.97 fps, 30k tbn, 29.97 tbc
>     Metadata:
>       encoder         : Lavc56.44.100 libx264
>     Stream #0:1: Audio: aac (libfdk_aac) ([64][0][0][0] / 0x0040), 44100 Hz,
> mono, s16, 80 kb/s
>     Metadata:
>       encoder         : Lavc56.44.100 libfdk_aac
> Stream mapping:
>   Stream #1:1 -> #0:0 (rawvideo (native) -> h264 (libx264))
>   Stream #0:0 -> #0:1 (pcm_f32le (native) -> aac (libfdk_aac))
> Press [q] to stop, [?] for help
> frame=   18 fps=0.0 q=29.0 size=       0kB time=00:00:00.03 bitrate=
> 11.5kbits/frame=   33 fps= 33 q=29.0 size=      79kB time=00:00:00.53
> bitrate=1216.8kbits/frame=   48 fps= 32 q=29.0 size=     148kB
> time=00:00:01.03 bitrate=1173.1kbits/frame=   63 fps= 31 q=29.0 size=
> 245kB time=00:00:01.53 bitrate=1305.0kbits/frame=   79 fps= 31 q=29.0 size=
>    369kB time=00:00:02.06 bitrate=1460.7kbits/frame=   94 fps= 31 q=29.0
> size=     491kB time=00:00:02.56 bitrate=1565.9kbits/frame=  109 fps= 31
> q=29.0 size=     593kB time=00:00:03.06 bitrate=1583.1kbits/frame=  124 fps=
> 31 q=29.0 size=     691kB time=00:00:03.57 bitrate=1585.9kbits/frame=  139
> fps= 31 q=29.0 size=     785kB
> ...
> time=00:00:26.79 bitrate=1108.6kbits/frame=  822 fps= 30 q=-1.0 Lsize=
> 3732kB time=00:00:27.42 bitrate=1114.6kbits/s
> video:3445kB audio:263kB subtitle:0kB other streams:0kB global headers:0kB
> muxing overhead: 0.636530%
> [libx264 @ 0x7fdc23845c00] frame I:9     Avg QP:25.83  size: 18818
> [libx264 @ 0x7fdc23845c00] frame P:813   Avg QP:27.67  size:  4130
> [libx264 @ 0x7fdc23845c00] mb I  I16..4: 32.9%  0.0% 67.1%
> [libx264 @ 0x7fdc23845c00] mb P  I16..4:  5.6%  0.0%  1.3%  P16..4: 36.7%
> 12.2%  4.7%  0.0%  0.0%    skip:39.5%
> [libx264 @ 0x7fdc23845c00] coded y,uvDC,uvAC intra: 39.7% 51.8% 17.8% inter:
> 21.0% 15.6% 0.4%
> [libx264 @ 0x7fdc23845c00] i16 v,h,dc,p: 29% 44% 18%  9%
> [libx264 @ 0x7fdc23845c00] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 14% 30% 18%  6%
> 6%  6%  8%  5%  6%
> [libx264 @ 0x7fdc23845c00] i8c dc,h,v,p: 47% 32% 15%  6%
> [libx264 @ 0x7fdc23845c00] kb/s:1028.74
>
>
> Here’s the ffprobe output for the resulting video_tests/test-avsync.mp4
> file:
>
> ffprobe -i video_tests/test-avsync.mp4
> ffprobe version N-73104-g7604358 Copyright (c) 2007-2015 the FFmpeg
> developers
>   built with Apple LLVM version 6.1.0 (clang-602.0.53) (based on LLVM
> 3.6.0svn)
>   configuration: --prefix=/usr/local --enable-gpl --enable-nonfree
> --enable-ffplay --enable-libass --enable-libfdk-aac --enable-libfreetype
> --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis
> --enable-libvpx --enable-libx264 --enable-libxvid --enable-decklink
> --extra-cflags=-I/Users/ptl/blackmagicsdk/Mac/include
> --extra-ldflags=-L/Users/ptl/blackmagicsdk/Mac/include
>   libavutil      54. 27.100 / 54. 27.100
>   libavcodec     56. 44.100 / 56. 44.100
>   libavformat    56. 38.100 / 56. 38.100
>   libavdevice    56.  4.100 / 56.  4.100
>   libavfilter     5. 17.100 /  5. 17.100
>   libswscale      3.  1.101 /  3.  1.101
>   libswresample   1.  2.100 /  1.  2.100
>   libpostproc    53.  3.100 / 53.  3.100
> Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'video_tests/test-avsync.mp4':
>   Metadata:
>     major_brand     : isom
>     minor_version   : 512
>     compatible_brands: isomiso2avc1mp41
>     encoder         : Lavf56.38.100
>   Duration: 00:00:16.08, start: 0.046440, bitrate: 1406 kb/s
>     Stream #0:0(und): Video: h264 (Constrained Baseline) (avc1 /
> 0x31637661), yuv420p, 640x360, 1321 kb/s, 29.97 fps, 29.97 tbr, 30k tbn,
> 59.94 tbc (default)
>     Metadata:
>       handler_name    : VideoHandler
>     Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, mono,
> fltp, 80 kb/s (default)
>     Metadata:
>       handler_name    : SoundHandler
>
>
> I can see that there’s an offset of 0.046440, but I’m not sure if that’s a
> consistent result. I’m trying to get the capture to produce a result without
> sync delay but in any case, why would the results be inconsistent?
>
> The MacBook is always pretty idle with no memory issues when doing the
> capture, and ffmpeg isn’t consuming too much cpu.
>
> Any suggestions on what else I can try to get consistent sync?
>
> thanks
> Ricardo
>
>
>
> _______________________________________________
> ffmpeg-user mailing list
> ffmpeg-user at ffmpeg.org
> http://ffmpeg.org/mailman/listinfo/ffmpeg-user
>


More information about the ffmpeg-user mailing list