[FFmpeg-user] Use source wallclock (NTP) in RTSP/RTCP streams to setpts for inter-media synchronization
David Gessel
gessel at blackrosetech.com
Mon Jan 18 00:38:02 EET 2021
-------- Original Message --------
Subject: Re: [FFmpeg-user] Use source wallclock (NTP) in RTSP/RTCP streams to setpts for inter-media synchronization
From: Carl Eugen Hoyos <ceffmpeg at gmail.com>
To: FFmpeg user questions <ffmpeg-user at ffmpeg.org>
Date: 2021-01-18 00:24+0300
> Am Sa., 16. Jan. 2021 um 23:49 Uhr schrieb David Gessel
> <gessel at blackrosetech.com>:
>
>> ffmpeg -max_delay 500000 -reorder_queue_size 10000 \
>> -fflags nobuffer -re -rtsp_transport udp -an -flags low_delay -strict experimental\
>> -fflags nobuffer -re -thread_queue_size 1024 -i rtsp://192.168.100.34:554/stream/profile0\
>> -fflags nobuffer -re -thread_queue_size 1024 -i rtsp://192.168.100.57:554/stream/profile0\
>> -fflags nobuffer -re -thread_queue_size 1024 -i rtsp://192.168.100.48:554/stream/profile0\
>> -fflags nobuffer -re -thread_queue_size 1024 -i rtsp://192.168.100.34:554/stream/profile1\
>> -fflags nobuffer -re -thread_queue_size 1024 -i rtsp://192.168.100.57:554/stream/profile1\
>> -fflags nobuffer -re -thread_queue_size 1024 -i rtsp://192.168.100.48:554/stream/profile1\
>> -filter_complex "
>> nullsrc=size=3554x480 [base];
>> [0:v] setpts=N/(8.33*TB) [CAM1];
>> [1:v] setpts=N/(8.33*TB) [CAM2];
>> [2:v] setpts=N/(8.33*TB) [CAM3];
>> [3:v] setpts=N/(8.33*TB) [CAM4];
>> [4:v] setpts=N/(8.33*TB) [CAM5];
>> [5:v] setpts=N/(8.33*TB) [CAM6];
>> [base][CAM1] overlay=x=0 [tmp1];
>> [tmp1][CAM2] overlay=x=583 [tmp2];
>> [tmp2][CAM3] overlay=x=1165 [tmp3];
>> [tmp3][CAM4] overlay=x=1748 [tmp4];
>> [tmp4][CAM5] overlay=x=2331 [tmp5];
>> [tmp5][CAM6] overlay=x=2914 " \
>> -c:v libx264 -tune zerolatency -an -preset ultrafast -crf 22 -f matroska - |\
>> ffplay -framedrop -sync ext -probesize 32 -
>
> You should really test with file output and complete, uncut console
> output missing.
Thanks for the help Carl,
The console output is attached below
>
> As long as you are using the setpts filter, you disable all
> synchronisation and you
> cannot get the output you want.
Makes sense, thanks for the hint. A link to a screen cap of the command without setpts attached below. The time code is burned in at the sources. I let it run for a few minutes and the streams did not sync.
https://www.brt.llc/insecure/ffmpeg_sync_issue.webp
(Babri the kitty says hi)
Console output follows:
$ ffmpeg -max_delay 500000 -reorder_queue_size 10000 \
> -fflags nobuffer -re -rtsp_transport udp -an -flags low_delay -strict experimental\
> -fflags nobuffer -re -thread_queue_size 1024 -i rtsp://192.168.100.34:554/stream/profile0\
> -fflags nobuffer -re -thread_queue_size 1024 -i rtsp://192.168.100.57:554/stream/profile0\
> -fflags nobuffer -re -thread_queue_size 1024 -i rtsp://192.168.100.48:554/stream/profile0\
> -fflags nobuffer -re -thread_queue_size 1024 -i rtsp://192.168.100.34:554/stream/profile1\
> -fflags nobuffer -re -thread_queue_size 1024 -i rtsp://192.168.100.57:554/stream/profile1\
> -fflags nobuffer -re -thread_queue_size 1024 -i rtsp://192.168.100.48:554/stream/profile1\
> -filter_complex "
> nullsrc=size=3554x480 [base];
> [base][0:v] overlay=x=0 [tmp1];
> [tmp1][1:v] overlay=x=583 [tmp2];
> [tmp2][2:v] overlay=x=1165 [tmp3];
> [tmp3][3:v] overlay=x=1748 [tmp4];
> [tmp4][4:v] overlay=x=2331 [tmp5];
> [tmp5][5:v] overlay=x=2914 " \
> -c:v libx264 -tune zerolatency -an -preset ultrafast -crf 22 -f matroska - |\
> ffplay -framedrop -sync ext -probesize 32 -
ffmpeg version N-100596-geacad34 Copyright (c) 2000-2021 the FFmpeg developers
built with gcc 9 (Ubuntu 9.3.0-17ubuntu1~20.04)
configuration: --prefix=/home/gessel/ffmpeg_build --pkg-config-flags=--static --extra-cflags=-I/home/gessel/ffmpeg_build/include --extra-ldflags=-L/home/gessel/ffmpeg_build/lib --extra-libs='-lpthread -lm' --bindir=/home/gessel/bin --enable-gpl --enable-libass --enable-libfdk-aac --enable-libfreetype --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 --enable-nonfree
libavutil 56. 63.100 / 56. 63.100
libavcodec 58.115.102 / 58.115.102
libavformat 58. 65.101 / 58. 65.101
libavdevice 58. 11.103 / 58. 11.103
libavfilter 7. 95.100 / 7. 95.100
libswscale 5. 8.100 / 5. 8.100
libswresample 3. 8.100 / 3. 8.100
libpostproc 55. 8.100 / 55. 8.100
ffplay version 4.2.4-1ubuntu0.1 Copyright (c) 2003-2020 the FFmpeg developers
built with gcc 9 (Ubuntu 9.3.0-10ubuntu2)
configuration: --prefix=/usr --extra-version=1ubuntu0.1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-nvenc --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
libavutil 56. 31.100 / 56. 31.100
libavcodec 58. 54.100 / 58. 54.100
libavformat 58. 29.100 / 58. 29.100
libavdevice 58. 8.100 / 58. 8.100
libavfilter 7. 57.100 / 7. 57.100
libavresample 4. 0. 0 / 4. 0. 0
libswscale 5. 5.100 / 5. 5.100
libswresample 3. 5.100 / 3. 5.100
libpostproc 55. 5.100 / 55. 5.100
Input #0, rtsp, from 'rtsp://192.168.100.34:554/stream/profile0':
Metadata:
title : Mobotix IP-Camera
comment : Profile profile0
Duration: N/A, start: 0.000000, bitrate: N/A
Stream #0:0: Video: mjpeg (Baseline), yuvj420p(pc, bt470bg/unknown/unknown), 640x480 [SAR 1:1 DAR 4:3], 8.33 fps, 8.33 tbr, 90k tbn, 90k tbc
Input #1, rtsp, from 'rtsp://192.168.100.57:554/stream/profile0':
Metadata:
title : Mobotix IP-Camera
comment : Profile profile0
Duration: N/A, start: 0.000000, bitrate: N/A
Stream #1:0: Video: mjpeg (Baseline), yuvj420p(pc, bt470bg/unknown/unknown), 640x480 [SAR 1:1 DAR 4:3], 8.33 fps, 8.33 tbr, 90k tbn, 90k tbc
Input #2, rtsp, from 'rtsp://192.168.100.48:554/stream/profile0':
Metadata:
title : Mobotix IP-Camera
comment : Profile profile0
Duration: N/A, start: 0.000000, bitrate: N/A
Stream #2:0: Video: mjpeg (Baseline), yuvj420p(pc, bt470bg/unknown/unknown), 640x480 [SAR 1:1 DAR 4:3], 8.33 fps, 8.33 tbr, 90k tbn, 90k tbc
Input #3, rtsp, from 'rtsp://192.168.100.34:554/stream/profile1':
Metadata:
title : Mobotix IP-Camera
comment : Profile profile1
Duration: N/A, start: 0.000000, bitrate: N/A
Stream #3:0: Video: mjpeg (Baseline), yuvj420p(pc, bt470bg/unknown/unknown), 640x480 [SAR 1:1 DAR 4:3], 8.33 fps, 8.33 tbr, 90k tbn, 90k tbc
Input #4, rtsp, from 'rtsp://192.168.100.57:554/stream/profile1':
Metadata:
title : Mobotix IP-Camera
comment : Profile profile1
Duration: N/A, start: 0.000000, bitrate: N/A
Stream #4:0: Video: mjpeg (Baseline), yuvj420p(pc, bt470bg/unknown/unknown), 640x480 [SAR 1:1 DAR 4:3], 8.33 fps, 8.33 tbr, 90k tbn, 90k tbc
Input #5, rtsp, from 'rtsp://192.168.100.48:554/stream/profile1':
Metadata:
title : Mobotix IP-Camera
comment : Profile profile1
Duration: N/A, start: 0.000000, bitrate: N/A
Stream #5:0: Video: mjpeg (Baseline), yuvj420p(pc, bt470bg/unknown/unknown), 640x480 [SAR 1:1 DAR 4:3], 8.33 fps, 8.33 tbr, 90k tbn, 90k tbc
Stream mapping:
Stream #0:0 (mjpeg) -> overlay:overlay
Stream #1:0 (mjpeg) -> overlay:overlay
Stream #2:0 (mjpeg) -> overlay:overlay
Stream #3:0 (mjpeg) -> overlay:overlay
Stream #4:0 (mjpeg) -> overlay:overlay
Stream #5:0 (mjpeg) -> overlay:overlay
overlay -> Stream #0:0 (libx264)
Press [q] to stop, [?] for help
[swscaler @ 0x5615e8adc0c0] deprecated pixel format used, make sure you did set range correctly
[swscaler @ 0x5615e8afce40] deprecated pixel format used, make sure you did set range correctly
[swscaler @ 0x5615e8b1e340] deprecated pixel format used, make sure you did set range correctly
[swscaler @ 0x5615e8b3f9c0] deprecated pixel format used, make sure you did set range correctly
[swscaler @ 0x5615e8b61240] deprecated pixel format used, make sure you did set range correctly
[swscaler @ 0x5615e8b82b80] deprecated pixel format used, make sure you did set range correctly
[rtsp @ 0x5615e87a56c0] max delay reached. need to consume packet
[rtsp @ 0x5615e87a56c0] RTP: missed 2152 packets
[rtsp @ 0x5615e875c400] max delay reached. need to consume packet
[rtsp @ 0x5615e875c400] RTP: missed 5058 packets
[rtsp @ 0x5615e875c400] RTP: PT=1a: bad cseq bbf9 expected=a837
[rtsp @ 0x5615e875c400] max delay reached. need to consume packet
[rtsp @ 0x5615e875c400] RTP: missed 5059 packets
[rtsp @ 0x5615e875c400] RTP timestamps don't match.
[rtsp @ 0x5615e875c400] Received packet without a start chunk; dropping frame.
Last message repeated 10 times
[rtsp @ 0x5615e87e54c0] max delay reached. need to consume packet
[rtsp @ 0x5615e87e54c0] RTP: missed 1058 packets
[rtsp @ 0x5615e875c400] Received packet without a start chunk; dropping frame.
Last message repeated 52 times 0KB vq= 0KB sq= 0B f=0/0
[libx264 @ 0x5615e88a4300] using SAR=1/1
[libx264 @ 0x5615e88a4300] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
[libx264 @ 0x5615e88a4300] profile Constrained Baseline, level 4.0
[libx264 @ 0x5615e88a4300] 264 - core 155 r2917 0a84d98 - H.264/MPEG-4 AVC codec - Copyleft 2003-2018 - http://www.videolan.org/x264.html - options: cabac=0 ref=1 deblock=0:0:0 analyse=0:0 me=dia subme=0 psy=1 psy_rd=1.00:0.00 mixed_ref=0 me_range=16 chroma_me=1 trellis=1 8x8dct=0 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=0 threads=7 lookahead_threads=7 sliced_threads=1 slices=7 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=0 weightp=0 keyint=250 keyint_min=25 scenecut=0 intra_refresh=0 rc=crf mbtree=0 crf=22.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=0
Output #0, matroska, to 'pipe:':
Metadata:
title : Mobotix IP-Camera
comment : Profile profile0
encoder : Lavf58.65.101
Stream #0:0: Video: h264 (H264 / 0x34363248), yuv420p(progressive), 3554x480 [SAR 1:1 DAR 1777:240], q=2-31, 25 fps, 1k tbn (default)
Metadata:
encoder : Lavc58.115.102 libx264
Side data:
cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A
[rtsp @ 0x5615e8761400] max delay reached. need to consume packet 4856.0kbits/s speed=0.00769x
[rtsp @ 0x5615e8761400] RTP: missed 1424 packets
[matroska,webm @ 0x7f5a40000bc0] decoding for stream 0 failed=0/0
Input #0, matroska,webm, from 'pipe:':
Metadata:
title : Mobotix IP-Camera
COMMENT : Profile profile0
ENCODER : Lavf58.65.101
Duration: N/A, bitrate: N/A
Stream #0:0: Video: h264 (Constrained Baseline), yuv420p(progressive), 3554x480 [SAR 1:1 DAR 1777:240], 25 fps, 25 tbr, 1k tbn, 50 tbc (default)
Metadata:
ENCODER : Lavc58.115.102 libx264
[rtsp @ 0x5615e8730080] max delay reached. need to consume packet
[rtsp @ 0x5615e8730080] RTP: missed 5791 packets
[rtsp @ 0x5615e8730080] RTP: PT=1a: bad cseq 32e9 expected=1c4a
[rtsp @ 0x5615e8730080] max delay reached. need to consume packet
[rtsp @ 0x5615e8730080] RTP: missed 5792 packets
[rtsp @ 0x5615e8730080] RTP timestamps don't match.
[rtsp @ 0x5615e8730080] Received packet without a start chunk; dropping frame.
Last message repeated 58 times 0KB vq= 5KB sq= 0B f=0/0
Error writing trailer of pipe:: Broken pipe 0KB sq= 0B f=0/0 18908.6kbits/s speed=0.998x
frame= 877 fps= 25 q=17.0 Lsize= 80881kB time=00:00:35.04 bitrate=18908.6kbits/s speed=0.983x
video:80935kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
[libx264 @ 0x5615e88a4300] frame I:4 Avg QP:16.75 size:532250
[libx264 @ 0x5615e88a4300] frame P:873 Avg QP:18.00 size: 92495
[libx264 @ 0x5615e88a4300] mb I I16..4: 100.0% 0.0% 0.0%
[libx264 @ 0x5615e88a4300] mb P I16..4: 0.8% 0.0% 0.0% P16..4: 62.2% 0.0% 0.0% 0.0% 0.0% skip:37.0%
[libx264 @ 0x5615e88a4300] coded y,uvDC,uvAC intra: 60.5% 89.4% 86.4% inter: 41.4% 57.7% 38.1%
[libx264 @ 0x5615e88a4300] i16 v,h,dc,p: 31% 18% 38% 13%
[libx264 @ 0x5615e88a4300] i8c dc,h,v,p: 44% 20% 22% 14%
[libx264 @ 0x5615e88a4300] kb/s:18900.13
Exiting normally, received signal 2.
More information about the ffmpeg-user
mailing list