[FFmpeg-trac] #9485(undetermined:new): Video frame drops when composing video from multiple generative video filters

[FFmpeg]

#9485: Video frame drops when composing video from multiple generative video
filters
-------------------------------------+-------------------------------------
             Reporter:  megpoid      |                    Owner:  (none)
                 Type:  defect       |                   Status:  new
             Priority:  normal       |                Component:
                                     |  undetermined
              Version:  git-master   |               Resolution:
             Keywords:               |               Blocked By:
             Blocking:               |  Reproduced by developer:  0
Analyzed by developer:  0            |
-------------------------------------+-------------------------------------
Description changed by megpoid:

Old description:

> Summary of the bug:
> When trying to use multiple audio visualizer filters and compose them
> together to a single video stream, some filters will fail to output at
> the requested frame rate and some images coming from the filter will be
> duplicated.
>
> How to reproduce:
> {{{
> % ffmpeg -v verbose -i input.mp3 -filter_complex
> "[0:a]avectorscope=s=256x824:r=30[a];
> [0:a]showcqt=s=1664x1080:text=0:r=30[b];
> [0:a]showwaves=s=256x256:mode=line:draw=full:r=30[c]; [a][c]vstack[d];
> [b][d]hstack[out]" -map "[out]" -map 0:a -c:a libopus -vcodec libx264
> -preset ultrafast -pix_fmt yuv444p10le -t 60 -crf 23 -r 30 out0.mkv
> 2>log0.txt
>

> }}}
> When watching the output video, between timestamps 00:00.307 and
> 00:00:340, or timestamps 00:03:407 and 00:03.440, you can see that
> showcqt filter's output on the left has duplicate frames. Between
> timestamps 00:03.874 and 00:03.907, the showwaves filter's output in the
> bottom-right corner also froze for a frame.
> ffmpeg version N-104443-gdd7a61f62f-g783935de18+1
> built on 2021 October 29th
>
> Pastebin of the error log file: https://pastebin.com/raw/4VdUPrD3
>

>
> Known workaround
> This issue can be worked around by first encoding each individual filter
> into a PNG image sequence, then composing them in a second run,  then
> converting to the desired format in a third run, like so:
>
> {{{
> ffmpeg -i input.mp3 -filter_complex
> "[0:a]avectorscope=s=256x824:r=30[out]" -map "[out]" -t 60 -r 30
> a_%05d.png
>
> ffmpeg -i input.mp3 -filter_complex
> "[0:a]showcqt=s=1664x1080:text=0:r=30[out]" -map "[out]" -t 60 -r 30
> b_%05d.png
>
> ffmpeg -i input.mp3 -filter_complex
> "[0:a]showwaves=s=256x256:mode=line:draw=full:r=30[out]" -map "[out]" -t
> 60 -r 30 c_%05d.png
>
> ffmpeg -r 30 -i a_%05d.png -r 30 -i b_%05d.png -r 30 -i c_%05d.png
> -filter_complex "[0:v][2:v]vstack[a]; [1:v][a]hstack[out]" -map "[out]"
> -t 60 -r 30 d_%05d.png
>
> ffmpeg -r 30 -i d_%05d.png -i input.mp3 -acodec libopus -vcodec libx264
> -preset ultrafast -pix_fmt yuv444p10le -t 60 -crf 23 -r 30 output.mkv
> }}}
>

>

>
> I have permission in writing to redistribute the attached audio file as
> long as credit is given: https://soundcloud.com/polyak

New description:

 Summary of the bug:
 When trying to use multiple audio visualizer filters and compose them
 together to a single video stream, some filters will fail to output at the
 requested frame rate and some images coming from the filter will be
 duplicated.

 How to reproduce:
 {{{
 % ffmpeg -v verbose -i input.mp3 -filter_complex
 "[0:a]avectorscope=s=256x824:r=30[a];
 [0:a]showcqt=s=1664x1080:text=0:r=30[b];
 [0:a]showwaves=s=256x256:mode=line:draw=full:r=30[c]; [a][c]vstack[d];
 [b][d]hstack[out]" -map "[out]" -map 0:a -c:a libopus -vcodec libx264
 -preset ultrafast -pix_fmt yuv444p10le -t 60 -crf 23 -r 30 out0.mkv
 2>log0.txt


 }}}
 When watching the output video, between timestamps 00:00.307 and
 00:00:340, or timestamps 00:03:407 and 00:03.440, you can see that showcqt
 filter's output on the left has duplicate frames. Between timestamps
 00:03.874 and 00:03.907, the showwaves filter's output in the bottom-right
 corner also froze for a frame.
 ffmpeg version N-104443-gdd7a61f62f-g783935de18+1
 built on 2021 October 29th

 Pastebin of the error log file: https://pastebin.com/5RRWFjzm or
 https://crypt.is-
 by.us/?5e76d7f27a7e50a7#Eqmqrdaj2Q5hNuCOMnycs6jYx1KnYm5FniNE3B5OA1o=
 I have uploaded a file named ticket9485out01.10mb.mkv that demonstrates
 the issues described above.


 Known workaround
 This issue can be worked around by first encoding each individual filter
 into a PNG image sequence, then composing them in a second run,  then
 converting to the desired format in a third run, like so:

 {{{
 ffmpeg -i input.mp3 -filter_complex
 "[0:a]avectorscope=s=256x824:r=30[out]" -map "[out]" -t 60 -r 30
 a_%05d.png

 ffmpeg -i input.mp3 -filter_complex
 "[0:a]showcqt=s=1664x1080:text=0:r=30[out]" -map "[out]" -t 60 -r 30
 b_%05d.png

 ffmpeg -i input.mp3 -filter_complex
 "[0:a]showwaves=s=256x256:mode=line:draw=full:r=30[out]" -map "[out]" -t
 60 -r 30 c_%05d.png

 ffmpeg -r 30 -i a_%05d.png -r 30 -i b_%05d.png -r 30 -i c_%05d.png
 -filter_complex "[0:v][2:v]vstack[a]; [1:v][a]hstack[out]" -map "[out]" -t
 60 -r 30 d_%05d.png

 ffmpeg -r 30 -i d_%05d.png -i input.mp3 -acodec libopus -vcodec libx264
 -preset ultrafast -pix_fmt yuv444p10le -t 60 -crf 23 -r 30 output.mkv
 }}}





 I have permission in writing to redistribute the attached audio file as
 long as credit is given: https://soundcloud.com/polyak

--
-- 
Ticket URL: <https://trac.ffmpeg.org/ticket/9485#comment:1>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker