[FFmpeg-user] Concatenation Much Slower with Process Substitution for File List

Scott Colby scott at scolby.com
Thu Feb 15 08:33:44 EET 2024


I am a bit perplexed about the difference in speed between these two commands:

ffmpeg \
    -f concat -safe 0 -i <(
        find . -type f -iname '*.mp3' -maxdepth 1 -print0 |
            xargs -0 -- realpath |
            sort -n |
            sed -e "s/'/'\\\''/g" -e "s/^/file '/" -e "s/$/'/" |
            tee flist
    ) \
    -i cover.jpg \
    -c:a copy -c:v copy \

ffmpeg -f concat -safe 0 -i flist -i cover.jpg -c:a copy -c:v copy out.mp3

The former runs at about speed=30x while the latter runs at speed=1.29e+04x.

(Note that I'm using the same file list for the second command from the output
of tee in the first command.)

If I remove the `-i cover.jpg` part from either command, they both run at about
speed=5.39e+03x. (It's strange that it's slower when doing less work, but that's
not the main question I'm asking here.)

Changing the order of the inputs to put the cover image first does not affect
the speed.

I am running with bash 3.2.57(1)-release (arm64-apple-darwin23) on macOS 14.2.1.

When I run the fish-equivalent command with fish 3.7.0, I get the high speed

ffmpeg -f concat -safe 0 -i (
    find . -type f -iname '*.mp3' -maxdepth 1 -print0 |
    xargs -0 -- realpath |
    sort -n |
    sed -e "s/'/'\\\''/g" -e "s/^/file '/" -e "s/\%/'/" |
) -i cover.jpg -c:a copy -c:v copy out.mp3

I believe this is because fish uses a temporary file for process substitution
instead of /dev/fd/63 like bash does.

I have run all these commands with -loglevel debug and do not see any
significant differences in the log output; the only differences are in the exact
values in the log tags at the beginning of the lines (e.g. [mp3 @ 0x130005a90]),
but those are different for every run anyway.

Why is there this vast difference in performance between the normal file input
and the process substitution input? Is there another flag that I can pass to get
the high-speed processing when using a process substitution?


More information about the ffmpeg-user mailing list