[FFmpeg-devel] [Consulting] Converting an Audio Visualization application to CLI using FFMpeg

[James Darnley]

On 2016-02-27 04:09, Ryan Schott wrote:
> Hello,
> 
> I am not sure if this is the right page to post this, but your consulting
> page recommended I use this list. I recent built an audio visualization app
> using html5. I'm currently using that app with xsplit to stream music to an
> RTMP endpoint on either twitch.tv or YouTube.
> 
> For an example check out: https://youtu.be/Zmb_fy528Uo
> 
> I would like to be able to mimic this functionality on a server without
> using window capture and a live browser window. I'm a fan of the
> flexibility that my html5 solution gives me, but there's no way to spawn
> streams from a CLI. I know I will most likely have to throw out my HTML5
> solution to get the data to something that can run on a command line
> without a full browser.
> 
> My dream is that I can somehow stream the audio and video that would be
> coming from the browser normally to an input in ffmpeg and then output it
> to an RTMP endpoint. If I could get to that point, I could easily build a
> controller application with my skillset to allow any number of users to
> arbitrarily stream music with visualization from our central server to
> their twitch and youtube channels.
> 
> Another use case I have is mass creating visualized mp4's from mp3's. I
> currently have  a static image script working using ffmpeg to batch create
> videos from mp3's, but we'd rather have visualized videos than static image
> videos.
> 
> Overall, I'm looking to see if it's feasible to pipe the html5 video and
> audio output to an rtmp endpoint without a browser, and if it's not what my
> options are to construct visualizations in code without a desktop.

If you want exactly what was shown then it can't be done right now.

In the mean time you can have a look at some of FFmpeg's existing vis
filters here
> https://trac.ffmpeg.org/wiki/FancyFilteringExamples#Audio

And you can checkout my work-in-progress vis library here
> https://gitlab.com/J_Darnley/Advanced-Visualization-Studio
Although this library can't t do it either, it does have a very similar
spectrograph display, all it needs is a small gap between the bands.
Unfortunately I need to update the libavfilter wrapper for recent
changes I made.

When you have the video you want you can pad the edges and draw the
text, both of which can be done right now with ffmpeg.

If you want to render the video using your existing HTML5 code then I
find it hard to believe that nobody has yet written a headless browser
which can write to some sort of output stream, stdout at the very least.


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 603 bytes
Desc: OpenPGP digital signature
URL: <http://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20160227/67137c64/attachment.sig>