[FFmpeg-devel] [PATCH] avfilter/asubprocess: add audio subprocess filter

[Stefan Westerfeld]

    Hi!

Am 27.06.24 um 23:35 schrieb Ramiro Polla:
> On Thu, Jun 27, 2024 at 6:32 PM Stefan Westerfeld <stefan at space.twc.de> wrote:
>> This will send the audio stream to an external command as wav file and
>> read back the output of the subprocess which is also supposed to be a
>> wav file of the same format and length.
> 
> Subprocesses are a pita to support and maintain in a cross-platform
> project. This approach would just open up a whole new can of worms to
> achieve something that FFmpeg can already do with a pipe, multiple
> inputs, and -map_metadata.

Not sure if FFmpeg can do it "properly" already. One of the requirements 
I have is that I want to progressively apply a watermark to a file which 
I get on stdin. I've tried to make a script which uses ffmpeg to do 
this, the closest working solution I have been able to get is

mkfifo ffmpeg.wm
tee ffmpeg.wm | ffmpeg -i - -f wav - | audiowmark add - - f0 |
   ffmpeg -i pipe:3 3<ffmpeg.wm -i - -filter_complex amix=weights="0 1" 
-f mp3 -

I use tee to have one stream to progressively get the metadata from and 
the watermarked stream. I use a filter here to make ffmpeg read the two 
inputs at the same speed, to avoid stalling the fifo or the watermarker. 
But this is not perfect really, so if the watermarker would buffer lots 
of data, I guess this would simply hang.

Note that this isn't much different than what my asubprocess filter 
would do. However it is less readable and maintainable, and just leaves 
subprocess management to the shell.

As for portability of my asubprocess filter, if building conditionally 
for unix-like-systems only is not an option for you, maybe the code 
could use a library for subprocess management, so that we don't need to 
reinvent the wheel here. I didn't deeply research this yet to evaluate 
all alternatives, but for instance glib-2.0 has portable subprocesses:

https://docs.gtk.org/gio/class.Subprocess.html

so if I would use this, the code should be portable and the ugly parts 
would live in glib-2.0 (where they already have been tested for a long 
time).

> Since your project already uses FFmpeg, perhaps you could use it to
> encode the output and keep the metadata as well. Or provide a simple
> script that would invoke FFmpeg with the correct parameters.

Yes, using the FFmpeg libs for reading the input and writing the output 
is something I considered before working on the asubprocess filter. It 
would probably be an alternative. However, there are cases where you 
have a video with two audio tracks as input and want to watermark it 
preserving tags in the audio tracks, so in the end I would have to 
implement code to handle video streams as well, which I'd like to avoid, 
especially since the ffmpeg command line utility is very flexible and 
powerful when dealing with video/audio stream transformations.

As for using a "simple" script, the complexity of doing this in a script 
and managing lots of audiowmark subprocesses there is also something I'd 
rather avoid, if ffmpeg could do it cleanly, making writing this script 
a lot easier, and avoiding temp files or processing everything in 
multiple passes.

The script for video watermarking we currently have uses temp files and 
multiple passes:

https://github.com/swesterfeld/audiowmark/blob/b73adb92b1ae24ddf5133412fe7f8a08dae75054/src/videowmark#L87

    Cu... Stefan
-- 
Stefan Westerfeld, http://space.twc.de/~stefan