[FFmpeg-trac] #9081(undetermined:new): Progress reporting regression for content with large frame gaps (was: Progress reporting regression in 4.3.x for content with large frame gaps)
FFmpeg
trac at avcodec.org
Thu Feb 11 13:20:35 EET 2021
#9081: Progress reporting regression for content with large frame gaps
-------------------------------------+-------------------------------------
Reporter: zrumford | Owner:
Type: defect | Status: new
Priority: important | Component:
| undetermined
Version: git-master | Resolution:
Keywords: fps | Blocked By:
regression |
Blocking: | Reproduced by developer: 0
Analyzed by developer: 0 |
-------------------------------------+-------------------------------------
Changes (by cehoyos):
* keywords: regression => fps regression
* priority: normal => important
* component: ffmpeg => undetermined
Old description:
> Summary of the bug:
> I noticed that the "progress" update in the console output no longer
> refreshes as of ver 4.3.x (I tested in 4.2.0 to confirm the regression)
> for certain content.
>
> For reference the "status update" I am talking about is this bit of the
> console output:
> frame= 1106 fps=157 q=26.0 size= 1024kB time=00:00:35.06 bitrate=
> 239.2kbits/s speed=4.97x
>
> The input content has large periods with no frames. The command I am
> using fixes this my forcing the output framerate to 30. When I do this in
> v4.2.1 (and prior) the "speed" and "frame" values written to the console
> continue to update even though the location in the source is not moving
> forward (during the large gaps). In 4.3.1, it looks like the progress
> update only occurs when the source location has changed.
>
> Why does this matter? My app does "hang detection" and this triggers a
> false positive hang. I can always bump my hang detection timer so it is
> not a huge deal for me but no matter what I can always encounter content
> which has longer and longer time between frames (think of a slide
> presentation where an 8 hour long video may have 10 frames).
>
> FWIW I tried on windows and linux builds and got the same result.
>
> How to reproduce:
> c:\ffmpegs\ffmpeg-4.2.1-win64-static\ffmpeg-4.2.1-win64-static>ffmpeg -i
> input.mp4 -an -map 0:v:0 -vf fps=fps=30,scale=1656:1246,yadif=deint=1
> -colorspace bt709 -color_primaries bt709 -color_trc bt709 -pix_fmt
> yuv420p -c:v libx264 -profile:v main -level:v 4.2 -preset medium -crf 20
> -x264-params bframes=0:scenecut=0:aud=1:cabac=0:colormatrix=bt709
> -force_key_frames expr:gte(t,n_forced*3) -b:v 2560000 -maxrate:v 3840000
> -bufsize 11520000 -max_muxing_queue_size 9999
> d8ba422f4-cbf7-4986-a2b2-acbc004ba739.m4v
> ffmpeg version 4.3.1 Copyright (c) 2000-2020 the FFmpeg developers
> built with gcc 10.2.1 (GCC) 20200726
> configuration: --enable-gpl --enable-version3 --enable-sdl2 --enable-
> fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-
> libdav1d --enable-libbluray --enable-libfreetype --enable-libmp3lame
> --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-
> libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy
> --enable-libsoxr --enable-libsrt --enable-libtheora --enable-libtwolame
> --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264
> --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma
> --enable-zlib --enable-gmp --enable-libvidstab --enable-libvmaf --enable-
> libvorbis --enable-libvo-amrwbenc --enable-libmysofa --enable-libspeex
> --enable-libxvid --enable-libaom --enable-libgsm --enable-librav1e
> --disable-w32threads --enable-libmfx --enable-ffnvcodec --enable-cuda-
> llvm --enable-cuvid --enable-d3d11va --enable-nvenc --enable-nvdec
> --enable-dxva2 --enable-avisynth --enable-libopenmpt --enable-amf
> libavutil 56. 51.100 / 56. 51.100
> libavcodec 58. 91.100 / 58. 91.100
> libavformat 58. 45.100 / 58. 45.100
> libavdevice 58. 10.100 / 58. 10.100
> libavfilter 7. 85.100 / 7. 85.100
> libswscale 5. 7.100 / 5. 7.100
> libswresample 3. 7.100 / 3. 7.100
> libpostproc 55. 7.100 / 55. 7.100
> Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'input.mp4':
> Metadata:
> major_brand : mp42
> minor_version : 1
> compatible_brands: isommp41mp42
> creation_time : 2021-01-20T16:18:49.000000Z
> Duration: 00:27:44.94, start: 0.000000, bitrate: 419 kb/s
> Stream #0:0(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz,
> stereo, fltp, 247 kb/s (default)
> Metadata:
> creation_time : 2021-01-20T16:18:49.000000Z
> handler_name : Core Media Audio
> Stream #0:1(und): Video: h264 (High) (avc1 / 0x31637661),
> yuvj420p(pc, bt709/bt709/iec61966-2-1), 1440x1920, 169 kb/s, 6.99 fps, 60
> tbr, 600 tbn, 1200 tbc (default)
> Metadata:
> rotate : 270
> creation_time : 2021-01-20T16:18:49.000000Z
> handler_name : Core Media Video
> Side data:
> displaymatrix: rotation of 90.00 degrees
> Stream mapping:
> Stream #0:1 -> #0:0 (h264 (native) -> h264 (libx264))
> Press [q] to stop, [?] for help
> [swscaler @ 00000203a4821a00] deprecated pixel format used, make sure you
> did set range correctly
> [libx264 @ 00000203a0601340] using cpu capabilities: MMX2 SSE2Fast SSSE3
> SSE4.2 AVX FMA3 BMI2 AVX2
> [libx264 @ 00000203a0601340] profile Main, level 4.2, 4:2:0, 8-bit
> [libx264 @ 00000203a0601340] 264 - core 161 - H.264/MPEG-4 AVC codec -
> Copyleft 2003-2020 - http://www.videolan.org/x264.html - options: cabac=0
> ref=3 deblock=1:0:0 analyse=0x1:0x111 me=hex subme=7 psy=1
> psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=0
> cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=12
> lookahead_threads=2 sliced_threads=0 nr=0 decimate=1 interlaced=0
> bluray_compat=0 constrained_intra=0 bframes=0 weightp=2 keyint=250
> keyint_min=25 scenecut=0 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1
> crf=20.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 vbv_maxrate=3840
> vbv_bufsize=11520 crf_max=0.0 nal_hrd=none filler=0 ip_ratio=1.40
> aq=1:1.00
> Output #0, ipod, to 'd8ba422f4-cbf7-4986-a2b2-acbc004ba739.m4v':
> Metadata:
> major_brand : mp42
> minor_version : 1
> compatible_brands: isommp41mp42
> encoder : Lavf58.45.100
> Stream #0:0(und): Video: h264 (libx264) (avc1 / 0x31637661),
> yuv420p(bt709), 1656x1246, q=-1--1, 2560 kb/s, 30 fps, 15360 tbn, 30 tbc
> (default)
> Metadata:
> encoder : Lavc58.91.100 libx264
> creation_time : 2021-01-20T16:18:49.000000Z
> handler_name : Core Media Video
> Side data:
> cpb: bitrate max/min/avg: 3840000/0/2560000 buffer size: 11520000
> vbv_delay: N/A
> displaymatrix: rotation of -0.00 degrees
> [out_0_0 @ 00000203a471f1c0] 100 buffers queued in out_0_0, something may
> be wrong.peed=2.47x
> frame= 290 fps=113 q=26.0 size= 0kB time=00:00:07.86 bitrate=
> 0.0kbits/s speed=3.06x
>
> Notes:
> I used this version
> https://github.com/BtbN/FFmpeg-
> Builds/releases/tag/autobuild-2021-01-26-12-37 (I tried many prior 4.3.1
> vernos too)
>
> I would submit my input.mp4 file which repros it but it has customer
> content so I cannot. I have probably two dozen other files (all with
> large gaps between the video frames) but I have not got approval to share
> externally.
New description:
Summary of the bug:
I noticed that the "progress" update in the console output no longer
refreshes as of ver 4.3.x (I tested in 4.2.0 to confirm the regression)
for certain content.
For reference the "status update" I am talking about is this bit of the
console output:
frame= 1106 fps=157 q=26.0 size= 1024kB time=00:00:35.06 bitrate=
239.2kbits/s speed=4.97x
The input content has large periods with no frames. The command I am using
fixes this my forcing the output framerate to 30. When I do this in v4.2.1
(and prior) the "speed" and "frame" values written to the console continue
to update even though the location in the source is not moving forward
(during the large gaps). In 4.3.1, it looks like the progress update only
occurs when the source location has changed.
Why does this matter? My app does "hang detection" and this triggers a
false positive hang. I can always bump my hang detection timer so it is
not a huge deal for me but no matter what I can always encounter content
which has longer and longer time between frames (think of a slide
presentation where an 8 hour long video may have 10 frames).
FWIW I tried on windows and linux builds and got the same result.
How to reproduce:
{{{
c:\ffmpegs\ffmpeg-4.2.1-win64-static\ffmpeg-4.2.1-win64-static>ffmpeg -i
input.mp4 -an -map 0:v:0 -vf fps=fps=30,scale=1656:1246,yadif=deint=1
-colorspace bt709 -color_primaries bt709 -color_trc bt709 -pix_fmt
yuv420p -c:v libx264 -profile:v main -level:v 4.2 -preset medium -crf 20
-x264-params bframes=0:scenecut=0:aud=1:cabac=0:colormatrix=bt709
-force_key_frames expr:gte(t,n_forced*3) -b:v 2560000 -maxrate:v 3840000
-bufsize 11520000 -max_muxing_queue_size 9999
d8ba422f4-cbf7-4986-a2b2-acbc004ba739.m4v
ffmpeg version 4.3.1 Copyright (c) 2000-2020 the FFmpeg developers
built with gcc 10.2.1 (GCC) 20200726
configuration: --enable-gpl --enable-version3 --enable-sdl2 --enable-
fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-
libdav1d --enable-libbluray --enable-libfreetype --enable-libmp3lame
--enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg
--enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr
--enable-libsrt --enable-libtheora --enable-libtwolame --enable-libvpx
--enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265
--enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp
--enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libvo-
amrwbenc --enable-libmysofa --enable-libspeex --enable-libxvid --enable-
libaom --enable-libgsm --enable-librav1e --disable-w32threads --enable-
libmfx --enable-ffnvcodec --enable-cuda-llvm --enable-cuvid --enable-
d3d11va --enable-nvenc --enable-nvdec --enable-dxva2 --enable-avisynth
--enable-libopenmpt --enable-amf
libavutil 56. 51.100 / 56. 51.100
libavcodec 58. 91.100 / 58. 91.100
libavformat 58. 45.100 / 58. 45.100
libavdevice 58. 10.100 / 58. 10.100
libavfilter 7. 85.100 / 7. 85.100
libswscale 5. 7.100 / 5. 7.100
libswresample 3. 7.100 / 3. 7.100
libpostproc 55. 7.100 / 55. 7.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'input.mp4':
Metadata:
major_brand : mp42
minor_version : 1
compatible_brands: isommp41mp42
creation_time : 2021-01-20T16:18:49.000000Z
Duration: 00:27:44.94, start: 0.000000, bitrate: 419 kb/s
Stream #0:0(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz,
stereo, fltp, 247 kb/s (default)
Metadata:
creation_time : 2021-01-20T16:18:49.000000Z
handler_name : Core Media Audio
Stream #0:1(und): Video: h264 (High) (avc1 / 0x31637661), yuvj420p(pc,
bt709/bt709/iec61966-2-1), 1440x1920, 169 kb/s, 6.99 fps, 60 tbr, 600 tbn,
1200 tbc (default)
Metadata:
rotate : 270
creation_time : 2021-01-20T16:18:49.000000Z
handler_name : Core Media Video
Side data:
displaymatrix: rotation of 90.00 degrees
Stream mapping:
Stream #0:1 -> #0:0 (h264 (native) -> h264 (libx264))
Press [q] to stop, [?] for help
[swscaler @ 00000203a4821a00] deprecated pixel format used, make sure you
did set range correctly
[libx264 @ 00000203a0601340] using cpu capabilities: MMX2 SSE2Fast SSSE3
SSE4.2 AVX FMA3 BMI2 AVX2
[libx264 @ 00000203a0601340] profile Main, level 4.2, 4:2:0, 8-bit
[libx264 @ 00000203a0601340] 264 - core 161 - H.264/MPEG-4 AVC codec -
Copyleft 2003-2020 - http://www.videolan.org/x264.html - options: cabac=0
ref=3 deblock=1:0:0 analyse=0x1:0x111 me=hex subme=7 psy=1
psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=0
cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=12
lookahead_threads=2 sliced_threads=0 nr=0 decimate=1 interlaced=0
bluray_compat=0 constrained_intra=0 bframes=0 weightp=2 keyint=250
keyint_min=25 scenecut=0 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1
crf=20.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 vbv_maxrate=3840
vbv_bufsize=11520 crf_max=0.0 nal_hrd=none filler=0 ip_ratio=1.40
aq=1:1.00
Output #0, ipod, to 'd8ba422f4-cbf7-4986-a2b2-acbc004ba739.m4v':
Metadata:
major_brand : mp42
minor_version : 1
compatible_brands: isommp41mp42
encoder : Lavf58.45.100
Stream #0:0(und): Video: h264 (libx264) (avc1 / 0x31637661),
yuv420p(bt709), 1656x1246, q=-1--1, 2560 kb/s, 30 fps, 15360 tbn, 30 tbc
(default)
Metadata:
encoder : Lavc58.91.100 libx264
creation_time : 2021-01-20T16:18:49.000000Z
handler_name : Core Media Video
Side data:
cpb: bitrate max/min/avg: 3840000/0/2560000 buffer size: 11520000
vbv_delay: N/A
displaymatrix: rotation of -0.00 degrees
[out_0_0 @ 00000203a471f1c0] 100 buffers queued in out_0_0, something may
be wrong.peed=2.47x
frame= 290 fps=113 q=26.0 size= 0kB time=00:00:07.86 bitrate=
0.0kbits/s speed=3.06x
}}}
Notes:
I used this version
https://github.com/BtbN/FFmpeg-
Builds/releases/tag/autobuild-2021-01-26-12-37 (I tried many prior 4.3.1
vernos too)
I would submit my input.mp4 file which repros it but it has customer
content so I cannot. I have probably two dozen other files (all with large
gaps between the video frames) but I have not got approval to share
externally.
--
Comment:
Probably a regression since b794df43f3a521985f27d40974316db40b64cc88
I tested with a simplified command line, and the issue I see is not
(primarily) a problem with the status line but that my system goes oom:
{{{
$ ffmpeg -i ticket9081.mp4 -vf fps=30 -qscale 2 -an out.avi
ffmpeg version N-101060-gbd50e715a9 Copyright (c) 2000-2021 the FFmpeg
developers
built with gcc 10 (SUSE Linux)
configuration: --enable-gpl
libavutil 56. 65.100 / 56. 65.100
libavcodec 58.122.100 / 58.122.100
libavformat 58. 67.100 / 58. 67.100
libavdevice 58. 11.103 / 58. 11.103
libavfilter 7.105.100 / 7.105.100
libswscale 5. 8.100 / 5. 8.100
libswresample 3. 8.100 / 3. 8.100
libpostproc 55. 8.100 / 55. 8.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'ticket9081.mp4':
Metadata:
major_brand : mp42
minor_version : 1
compatible_brands: isommp41mp42
creation_time : 2021-02-02T23:52:38.000000Z
Duration: 00:12:45.87, start: 0.000000, bitrate: 458 kb/s
Stream #0:0(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo,
fltp, 217 kb/s (default)
Metadata:
creation_time : 2021-02-02T23:52:38.000000Z
handler_name : Core Media Audio
vendor_id : [0][0][0][0]
Stream #0:1(und): Video: h264 (High) (avc1 / 0x31637661), yuvj420p(pc,
bt709/bt709/iec61966-2-1), 1440x1920, 239 kb/s, 1.95 fps, 60 tbr, 600 tbn,
1200 tbc (default)
Metadata:
rotate : 270
creation_time : 2021-02-02T23:52:38.000000Z
handler_name : Core Media Video
vendor_id : [0][0][0][0]
Side data:
displaymatrix: rotation of 90.00 degrees
Please use -q:a or -q:v, -qscale is ambiguous
Stream mapping:
Stream #0:1 -> #0:0 (h264 (native) -> mpeg4 (native))
Press [q] to stop, [?] for help
[swscaler @ 0x413a5c0] deprecated pixel format used, make sure you did set
range correctly
Output #0, avi, to 'out.avi':
Metadata:
major_brand : mp42
minor_version : 1
compatible_brands: isommp41mp42
ISFT : Lavf58.67.100
Stream #0:0(und): Video: mpeg4 (FMP4 / 0x34504D46), yuv420p(tv,
bt709/bt709/iec61966-2-1, progressive), 1920x1440, q=2-31, 200 kb/s, 30
fps, 30 tbn (default)
Metadata:
encoder : Lavc58.122.100 mpeg4
creation_time : 2021-02-02T23:52:38.000000Z
handler_name : Core Media Video
vendor_id : [0][0][0][0]
Side data:
cpb: bitrate max/min/avg: 0/0/200000 buffer size: 0 vbv_delay: N/A
displaymatrix: rotation of -0.00 degrees
[out_0_0 @ 0x376bdc0] 100 buffers queued in out_0_0, something may be
wrong.ts/s speed= 3.3x
[out_0_0 @ 0x376bdc0] 1000 buffers queued in out_0_0, something may be
wrong.s/s speed=3.31x
Killed
}}}
--
Ticket URL: <https://trac.ffmpeg.org/ticket/9081#comment:7>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker
More information about the FFmpeg-trac
mailing list