[FFmpeg-user] How does [a]select work from an output perspective?
Rob Hallam
robert at roberthallam.com
Fri Jan 21 18:28:05 EET 2022
Context: I am making videos by combining two sets of game footage, one
I record myself and I grab the other one which my friend streamed to
Twitch. I want to get the two as closely 'aligned' (in A/V sync) as
possible to save time when I work with clips from the footage later in
an NLE (Blender VSE).
https://ffmpeg.org/ffmpeg-filters.html#select_002c-aselect
the documentation for the filter indicates frames not matching are 'discarded'.
I grab footage and create a test clip to check sync using a command like this:
ffmpeg -y -ss 48:41 -i
https://d2vjef5jvl6bfs.cloudfront.net/04a2d491ed12d9666418_idiotictendencies_45210723181_1642625711/chunked/index-dvr.m3u8
-t 10 -vf select='gt(t\,7.51)',scale=1920:1080:flags=lanczos,fps=fps=60
-c:v h264 -crf 19 -af aselect='gt(t\,7.51)' -c:a aac -shortest
grumround.mkv
(output appended)
using:
-ss to jump to the nearest I-frame to the point where I want to sync
for a 10 second clip
-vf: select after 7.51 seconds*, then scale and convert output
framerate to match my own footage
-h264 + aac encoding options
This produces a video which when played appears to be in 'sync' with
my footage. However, the video is 10 seconds long and starts at the
7th (ish) second. This is due to 'Duration: 00:00:10.02, start:
7.488000,' as reported by ffprobe (output appended) as far as I can
tell.
In my ignorance of how these options work, I figured it would either
produce a 2.5 second clip, ie 10 seconds of input filtered to only use
those after 7.5; or a full 10 second clip, starting at (-ss + gt)
time.
* I am actually kinda surprised that "gt 7.5" works at all in this
context, as I figured the frame timestamps in the streamed video would
be higher than that.
In getting this to work, I have also tried plain input seeking (-ss
<time> -i <input>) and combined seeking (-ss <time> -i <input> -ss
<time2>) but this produces an output video which is 10 seconds long
but with blank video until it reaches the desired point. This I think
I understand, as the seeking gets to a valid audio time, but doesn't
get to a usable video frame until further on. These aren't usable for
my purposes.
I would be grateful for a bit of insight so that I understand what I
am doing a little better. Comments about alternatives or how to better
achieve what I am doing -- ie cut a video segment out of an online VOD
to about 0.2s accuracy -- are of course welcome too!
Output appended.
Cheers for your time,
Rob
----
(not git HEAD, I can compile but it takes a while and arch's version
should be reasonably up to date)
$ ffmpeg -y -ss 48:41 -i
https://d2vjef5jvl6bfs.cloudfront.net/04a2d491ed12d9666418_idiotictendencies_45210723181_1642625711/chunked/index-dvr.m3u8
-t 10 -vf select='gt(t\,7.51)',scale=1920:1080:flags=lanczos,fps=fps=60
-c:v h264 -crf 19 -af aselect='gt(t\,7.51)' -c:a aac -shortest
grumround.mkv
ffmpeg version n4.4.1 Copyright (c) 2000-2021 the FFmpeg developers
built with gcc 11.1.0 (GCC)
configuration: --prefix=/usr --disable-debug --disable-static
--disable-stripping --enable-amf --enable-avisynth -
-enable-cuda-llvm --enable-lto --enable-fontconfig --enable-gmp
--enable-gnutls --enable-gpl --enable-ladspa --enabl
e-libaom --enable-libass --enable-libbluray --enable-libdav1d
--enable-libdrm --enable-libfreetype --enable-libfribi
di --enable-libgsm --enable-libiec61883 --enable-libjack
--enable-libmfx --enable-libmodplug --enable-libmp3lame --e
nable-libopencore_amrnb --enable-libopencore_amrwb
--enable-libopenjpeg --enable-libopus --enable-libpulse --enable-
librav1e --enable-librsvg --enable-libsoxr --enable-libspeex
--enable-libsrt --enable-libssh --enable-libsvtav1 --en
able-libtheora --enable-libv4l2 --enable-libvidstab --enable-libvmaf
--enable-libvorbis --enable-libvpx --enable-lib
webp --enable-libx264 --enable-libx265 --enable-libxcb
--enable-libxml2 --enable-libxvid --enable-libzimg --enable-n
vdec --enable-nvenc --enable-shared --enable-version3
libavutil 56. 70.100 / 56. 70.100
libavcodec 58.134.100 / 58.134.100
libavformat 58. 76.100 / 58. 76.100
libavdevice 58. 13.100 / 58. 13.100
libavfilter 7.110.100 / 7.110.100
libswscale 5. 9.100 / 5. 9.100
libswresample 3. 9.100 / 3. 9.100
libpostproc 55. 9.100 / 55. 9.100
[hls @ 0x55ed8a6e3240] Skip ('#EXT-X-VERSION:3')
[hls @ 0x55ed8a6e3240] Skip ('#ID3-EQUIV-TDTG:2022-01-19T22:43:46')
[hls @ 0x55ed8a6e3240] Skip ('#EXT-X-TWITCH-ELAPSED-SECS:0.000')
[hls @ 0x55ed8a6e3240] Skip ('#EXT-X-TWITCH-TOTAL-SECS:6505.818')
[hls @ 0x55ed8a6e3240] Opening
'https://d2vjef5jvl6bfs.cloudfront.net/04a2d491ed12d9666418_idiotictendencies_4521072
3181_1642625711/chunked/0.ts' for reading
[hls @ 0x55ed8a6e3240] Opening
'https://d2vjef5jvl6bfs.cloudfront.net/04a2d491ed12d9666418_idiotictendencies_4521072
3181_1642625711/chunked/1.ts' for reading
Input #0, hls, from
'https://d2vjef5jvl6bfs.cloudfront.net/04a2d491ed12d9666418_idiotictendencies_45210723181_164262
5711/chunked/index-dvr.m3u8':
Duration: 01:48:25.82, start: 64.665000, bitrate: 0 kb/s
Program 0
Metadata:
variant_bitrate : 0
Stream #0:0: Audio: aac (LC) ([15][0][0][0] / 0x000F), 44100 Hz,
stereo, fltp
Metadata:
variant_bitrate : 0
Stream #0:1: Video: h264 (High) ([27][0][0][0] / 0x001B),
yuv420p(tv, smpte170m/smpte170m/bt470m), 1280x720 [SAR 1
:1 DAR 16:9], 30 tbr, 90k tbn, 60 tbc
Metadata:
variant_bitrate : 0
Stream #0:2: Data: timed_id3 (ID3 / 0x20334449)
Metadata:
variant_bitrate : 0
Stream mapping:
Stream #0:1 -> #0:0 (h264 (native) -> h264 (libx264))
Stream #0:0 -> #0:1 (aac (native) -> aac (native))
Press [q] to stop, [?] for help
[hls @ 0x55ed8a6e3240] Opening
'https://d2vjef5jvl6bfs.cloudfront.net/04a2d491ed12d9666418_idiotictendencies_4521072
3181_1642625711/chunked/269.ts' for reading
[hls @ 0x55ed8a6e3240] Opening
'https://d2vjef5jvl6bfs.cloudfront.net/04a2d491ed12d9666418_idiotictendencies_4521072
3181_1642625711/chunked/270.ts' for reading
[libx264 @ 0x55ed8ae79080] using SAR=1/1
[libx264 @ 0x55ed8ae79080] using cpu capabilities: MMX2 SSE2Fast SSSE3
SSE4.2 AVX FMA3 BMI2 AVX2
[libx264 @ 0x55ed8ae79080] profile High, level 4.2, 4:2:0, 8-bit
[libx264 @ 0x55ed8ae79080] 264 - core 163 r3060 5db6aa6 - H.264/MPEG-4
AVC codec - Copyleft 2003-2021 - http://www.v
ideolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0
analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00
mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0
deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 thread
s=18 lookahead_threads=3 sliced_threads=0 nr=0 decimate=1 interlaced=0
bluray_compat=0 constrained_intra=0 bframes=3
b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0
weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_r
efresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=19.0 qcomp=0.60 qpmin=0
qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, matroska, to 'grumround.mkv':
Metadata:
encoder : Lavf58.76.100
Stream #0:0: Video: h264 (H264 / 0x34363248), yuv420p(tv,
smpte170m/smpte170m/bt470m, progressive), 1920x1080 [SAR
1:1 DAR 16:9], q=2-31, 60 fps, 1k tbn
Metadata:
variant_bitrate : 0
encoder : Lavc58.134.100 libx264
Side data:
cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A
Stream #0:1: Audio: aac (LC) ([255][0][0][0] / 0x00FF), 44100 Hz,
stereo, fltp, 128 kb/s
Metadata:
variant_bitrate : 0
encoder : Lavc58.134.100 aac
frame= 148 fps= 36 q=-1.0 Lsize= 3478kB time=00:00:09.99
bitrate=2850.7kbits/s speed=2.41x
video:3437kB audio:39kB subtitle:0kB other streams:0kB global
headers:0kB muxing overhead: 0.077687%
[libx264 @ 0x55ed8ae79080] frame I:1 Avg QP:20.69 size:204703
[libx264 @ 0x55ed8ae79080] frame P:39 Avg QP:21.65 size: 67001
[libx264 @ 0x55ed8ae79080] frame B:108 Avg QP:24.52 size: 6489
[libx264 @ 0x55ed8ae79080] consecutive B-frames: 1.4% 2.7% 4.1% 91.9%
[libx264 @ 0x55ed8ae79080] mb I I16..4: 6.8% 71.2% 21.9%
[libx264 @ 0x55ed8ae79080] mb P I16..4: 1.8% 9.7% 2.1% P16..4:
31.1% 23.7% 12.1% 0.0% 0.0% skip:19.7%
[libx264 @ 0x55ed8ae79080] mb B I16..4: 0.2% 0.7% 0.2% B16..8:
17.9% 3.8% 0.8% direct: 0.7% skip:75.7% L0:
54.1% L1:40.1% BI: 5.8%
[libx264 @ 0x55ed8ae79080] 8x8 transform intra:70.7% inter:66.3%
[libx264 @ 0x55ed8ae79080] coded y,uvDC,uvAC intra: 68.7% 70.9% 34.3%
inter: 11.4% 8.9% 1.4%
[libx264 @ 0x55ed8ae79080] i16 v,h,dc,p: 14% 42% 7% 38%
[libx264 @ 0x55ed8ae79080] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 19% 24% 13%
5% 8% 7% 9% 6% 9%
[libx264 @ 0x55ed8ae79080] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 20% 25% 10%
6% 10% 8% 9% 6% 7%
[libx264 @ 0x55ed8ae79080] i8c dc,h,v,p: 50% 25% 17% 8%
[libx264 @ 0x55ed8ae79080] Weighted P-Frames: Y:0.0% UV:0.0%
[libx264 @ 0x55ed8ae79080] ref P L0: 62.4% 19.5% 15.0% 3.0%
[libx264 @ 0x55ed8ae79080] ref B L0: 90.5% 7.4% 2.1%
[libx264 @ 0x55ed8ae79080] ref B L1: 99.6% 0.4%
[libx264 @ 0x55ed8ae79080] kb/s:11411.55
[aac @ 0x55ed8ad0c240] Qavg: 179.170
$ ffprobe grumround.mkv
ffprobe version n4.4.1 Copyright (c) 2007-2021 the FFmpeg developers
built with gcc 11.1.0 (GCC)
configuration: --prefix=/usr --disable-debug --disable-static
--disable-stripping --enable-amf --enable-avisynth -
-enable-cuda-llvm --enable-lto --enable-fontconfig --enable-gmp
--enable-gnutls --enable-gpl --enable-ladspa --enabl
e-libaom --enable-libass --enable-libbluray --enable-libdav1d
--enable-libdrm --enable-libfreetype --enable-libfribi
di --enable-libgsm --enable-libiec61883 --enable-libjack
--enable-libmfx --enable-libmodplug --enable-libmp3lame --e
nable-libopencore_amrnb --enable-libopencore_amrwb
--enable-libopenjpeg --enable-libopus --enable-libpulse --enable-
librav1e --enable-librsvg --enable-libsoxr --enable-libspeex
--enable-libsrt --enable-libssh --enable-libsvtav1 --en
able-libtheora --enable-libv4l2 --enable-libvidstab --enable-libvmaf
--enable-libvorbis --enable-libvpx --enable-lib
webp --enable-libx264 --enable-libx265 --enable-libxcb
--enable-libxml2 --enable-libxvid --enable-libzimg --enable-n
vdec --enable-nvenc --enable-shared --enable-version3
libavutil 56. 70.100 / 56. 70.100
libavcodec 58.134.100 / 58.134.100
libavformat 58. 76.100 / 58. 76.100
libavdevice 58. 13.100 / 58. 13.100
libavfilter 7.110.100 / 7.110.100
libswscale 5. 9.100 / 5. 9.100
libswresample 3. 9.100 / 3. 9.100
libpostproc 55. 9.100 / 55. 9.100
Input #0, matroska,webm, from 'grumround.mkv':
Metadata:
ENCODER : Lavf58.76.100
Duration: 00:00:10.02, start: 7.488000, bitrate: 2844 kb/s
Stream #0:0: Video: h264 (High), yuv420p(tv,
smpte170m/smpte170m/bt470m, progressive), 1920x1080 [SAR 1:1 DAR 16:9
], 60 fps, 60 tbr, 1k tbn, 120 tbc (default)
Metadata:
VARIANT_BITRATE : 0
ENCODER : Lavc58.134.100 libx264
DURATION : 00:00:10.000000000
Stream #0:1: Audio: aac (LC), 44100 Hz, stereo, fltp (default)
Metadata:
VARIANT_BITRATE : 0
ENCODER : Lavc58.134.100 aac
DURATION : 00:00:10.019000000
More information about the ffmpeg-user
mailing list