[FFmpeg-devel] [PATCH] OpenAL 1.1 Capture Support

[Stefano Sabatini]

On date Saturday 2011-06-04 23:50:17 -0400, Jonathan Baldwin encoded:
> I'm not sure if I'm doing it right, but I plan to make my first
> contribution to any open source project right now.
> 
> Attached is a patch which adds an OpenAL 1.1 audio capture input
> device to libavdevice. This means that audio capture in ffmpeg should
> now be possible on any platform with a working OpenAL 1.1
> implementation. Right now, there are three major OpenAL
> implementations:
> OpenAL Soft
>   Portable software implementation with multiple backends, currently
> ALSA, OSS, PulseAudio, Solaris, DirectSound, PortAudio. LGPL.
> Creative
>   Creative Labs' official implementation for Windows with hardware
> acceleration and software fallback. Proprietary.
> Apple
>   Implementation included with recent versions of Mac OS X.
> 
> So far, I've only tested it on Kubuntu Linux 11.04 using the OpenAL
> Soft implementation, and it works great. But it should work on Windows
> and Mac OS X too. I'll test it on Windows if I have to (I don't have a
> Mac).
> 
> What this code does NOT do is provide an output device for playing
> audio. OpenAL is more or less overkill for this anyways, as it
> provides a 3D audio environment.
> 
> Please tell me what I can do to improve the code, after all, I am only
> a high school student with too much time on my hands ;-)

> diff --git a/Changelog b/Changelog
> index a2018dd..d809be5 100644
> --- a/Changelog
> +++ b/Changelog
> @@ -21,6 +21,7 @@ version <next>:
>  - split filter added
>  - select filter added
>  - sdl output device added
> +- openal input device added
>  
>  
>  version 0.7_beta1:
> diff --git a/configure b/configure
> index 62b7e2f..f8077cd 100755
> --- a/configure
> +++ b/configure
> @@ -1470,6 +1470,8 @@ dv1394_indev_deps="dv1394 dv_demuxer"
>  fbdev_indev_deps="linux_fb_h"
>  jack_indev_deps="jack_jack_h sem_timedwait"
>  libdc1394_indev_deps="libdc1394"
> +openal_indev_deps="AL_al_h AL_alc_h"
> +openal_indev_extralibs="-lopenal"
>  oss_indev_deps_any="soundcard_h sys_soundcard_h"
>  oss_outdev_deps_any="soundcard_h sys_soundcard_h"
>  sdl_outdev_deps="sdl"
> @@ -2970,6 +2972,12 @@ check_header linux/videodev.h
>  check_header linux/videodev2.h
>  check_header sys/videoio.h
>  
> +enabled openal_indev &&
> +check_cpp_condition "AL/al.h" "defined(AL_VERSION_1_1)" "" -lopenal &&
> +check_lib2 "AL/al.h" alGetError -lopenal &&
> +check_header "AL/al.h" &&
> +check_header "AL/alc.h"
> +

I think this could be simplified by using:
enabled libopenal && require AL/al.h alGetError -lopenal &&
                      { check_cpp_condition AL/al.h "defined(AL_VERSION_1_1)" ||
                        die "ERROR: libopenal version must be 1.1"; }
...
openal_indev_deps="libopenal"

>  check_func_headers "windows.h vfw.h" capCreateCaptureWindow "$vfwcap_indev_extralibs"
>  # check that WM_CAP_DRIVER_CONNECT is defined to the proper value
>  # w32api 3.12 had it defined wrong
> diff --git a/doc/indevs.texi b/doc/indevs.texi
> index 0487108..c946867 100644
> --- a/doc/indevs.texi
> +++ b/doc/indevs.texi
> @@ -137,6 +137,47 @@ For more information read:
>  
>  IIDC1394 input device, based on libdc1394 and libraw1394.
>  
> + at section openal
> +
> +The OpenAL input device provides audio capture on all systems with a
> +working OpenAL 1.1 implementation.
> +
> +To enable this input device during configuration, you will need the
> +OpenAL headers and libraries. These are provided by the OpenAL Soft
> +implementation and as part of an SDK from Creative Labs.
> +
> +To capture from a specific sound device, provide the OpenAL name of
> +that device as the filename:
> + at example

> +$> ffmpeg -f openal -i 'DR-BT101 via PulseAudio' /tmp/out.ogg

"$" alone should be more clear.

> + at end example
> +Note that the device names used are different for each OpenAL
> +implementation. OpenAL Soft provides a tool called openal-info which

nit: @file{openal-info}

> +can be used to list the devices on a system.
> +
> +Or, just pass a blank string as the filename to use the default
> +device:
> + at example
> +$> ffmpeg -f openal -i '' /tmp/out.ogg
> + at end example
> +
> +Some OpenAL implementations (such as OpenAL Soft) support capturing
> +from two devices simultaneously within a single process:
> + at example
> +$> ffmpeg -f openal -i 'DR-BT101 via PulseAudio' /tmp/out1.ogg -f openal -i 'ALSA Default' /tmp/out2.ogg
> + at end example

> +If this doesn't work (crashes, duplicate audio, or worse) it is your
> +OpenAL implementation; try using the latest version of OpenAL Soft

it is your OpenAL implementation => it may be fault of your OpenAL implemention

since we're no perfect.

> +and see if that works.
> +
> +Links:
> + at itemize
> + at item
> +Creative Labs OpenAL Homepage @url{http://openal.org}
> + at item
> +OpenAL Soft @url{http://kcat.strangesoft.net/openal.html}
> + at end itemize
> +
>  @section oss
>  
>  Open Sound System input device.
> diff --git a/libavdevice/Makefile b/libavdevice/Makefile
> index 60103a4..4d3c1ae 100644
> --- a/libavdevice/Makefile
> +++ b/libavdevice/Makefile
> @@ -19,6 +19,7 @@ OBJS-$(CONFIG_DSHOW_INDEV)               += dshow.o dshow_enummediatypes.o \
>  OBJS-$(CONFIG_DV1394_INDEV)              += dv1394.o
>  OBJS-$(CONFIG_FBDEV_INDEV)               += fbdev.o
>  OBJS-$(CONFIG_JACK_INDEV)                += jack_audio.o
> +OBJS-$(CONFIG_OPENAL_INDEV)              += openal-dec.o
>  OBJS-$(CONFIG_OSS_INDEV)                 += oss_audio.o
>  OBJS-$(CONFIG_OSS_OUTDEV)                += oss_audio.o
>  OBJS-$(CONFIG_SDL_OUTDEV)                += sdl.o
> diff --git a/libavdevice/alldevices.c b/libavdevice/alldevices.c
> index 7846704..ef302d7 100644
> --- a/libavdevice/alldevices.c
> +++ b/libavdevice/alldevices.c
> @@ -44,6 +44,7 @@ void avdevice_register_all(void)
>      REGISTER_INDEV    (DV1394, dv1394);
>      REGISTER_INDEV    (FBDEV, fbdev);
>      REGISTER_INDEV    (JACK, jack);
> +    REGISTER_INDEV    (OPENAL, openal);
>      REGISTER_INOUTDEV (OSS, oss);
>      REGISTER_OUTDEV   (SDL, sdl);
>      REGISTER_INOUTDEV (SNDIO, sndio);
> diff --git a/libavdevice/avdevice.h b/libavdevice/avdevice.h
> index be56be4..7cb8f54 100644
> --- a/libavdevice/avdevice.h
> +++ b/libavdevice/avdevice.h
> @@ -23,7 +23,7 @@
>  #include "libavformat/avformat.h"
>  
>  #define LIBAVDEVICE_VERSION_MAJOR 53
> -#define LIBAVDEVICE_VERSION_MINOR  1
> +#define LIBAVDEVICE_VERSION_MINOR  2
>  #define LIBAVDEVICE_VERSION_MICRO  0
>  
>  #define LIBAVDEVICE_VERSION_INT AV_VERSION_INT(LIBAVDEVICE_VERSION_MAJOR, \
> diff --git a/libavdevice/openal-dec.c b/libavdevice/openal-dec.c
> new file mode 100644
> index 0000000..5c3eeee
> --- /dev/null
> +++ b/libavdevice/openal-dec.c
> @@ -0,0 +1,269 @@
> +/*

> + * OpenAL 1.1 capture device for libavdevice

Nit++: create a dedication /** @file ... */ doxy for this

> + * Copyright (c) 2011 Jonathan Baldwin
> + *
> + * Permission is hereby granted, free of charge, to any person obtaining a copy
> + * of this software and associated documentation files (the "Software"), to deal
> + * in the Software without restriction, including without limitation the rights
> + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
> + * copies of the Software, and to permit persons to whom the Software is
> + * furnished to do so, subject to the following conditions:
> + *
> + * The above copyright notice and this permission notice shall be included in
> + * all copies or substantial portions of the Software.
> + *
> + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
> + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
> + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
> + * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
> + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
> + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
> + * THE SOFTWARE.

OK, if you want this to be released as public domain (rather than the
standard LGPL adopted by most FFmpeg codebase). 

> + */
> +
> +#include "libavformat/avformat.h"
> +#include "libavutil/opt.h"
> +#include <AL/al.h>
> +#include <AL/alc.h>
> +
> +#define SSIZE 2048
> +
> +typedef struct
> +{

Style nits: 
* "typedef struct {" on the same line
* if (foo) {         on the same line, (foo) is preferred over ( foo )
* switch (foo) {     on the same line

> +    ALCdevice *device;
> +
> +    ALCchar buffer[32768];
> +    ALCint buffer_capacity;
> +    ALCint sample_size;
> +
> +    int channels, sample_rate;
> +    ALCenum sample_format;
> +} al_data;
> +

> +/**
> + * Gets the corresponding codecid for an AL_FORMAT.
> + */
> +static inline enum CodecID al_format_codecid ( ALCenum alf )
> +{
> +    switch ( alf )
> +    {
> +    case AL_FORMAT_MONO8:
> +    case AL_FORMAT_STEREO8:
> +        return CODEC_ID_PCM_S8;
> +    case AL_FORMAT_MONO16:
> +    case AL_FORMAT_STEREO16:

> +#if HAVE_BIGENDIAN
> +        return CODEC_ID_PCM_S16BE;
> +#else
> +        return CODEC_ID_PCM_S16LE;
> +#endif

AV_NE(CODEC_ID_PCM_S16BE, CODEC_ID_PCM_S16LE),

> +    }
> +    return CODEC_ID_NONE;
> +}
> +
> +/**
> + * Gets the number of channels for an AL_FORMAT.
> + */
> +static inline int al_format_channels ( ALCenum alf )
> +{
> +    switch ( alf )
> +    {
> +    case AL_FORMAT_MONO8:
> +    case AL_FORMAT_MONO16:
> +        return 1;
> +    case AL_FORMAT_STEREO8:
> +    case AL_FORMAT_STEREO16:
> +        return 2;
> +    }
> +    return 0;
> +}
> +
> +/**
> + * Gets the sample size for an AL_FORMAT.
> + */
> +static inline int al_format_sample_size ( ALCenum alf )
> +{
> +    switch ( alf )
> +    {
> +    case AL_FORMAT_MONO8:
> +        return 1;
> +    case AL_FORMAT_MONO16:
> +    case AL_FORMAT_STEREO8:
> +        return 2;
> +    case AL_FORMAT_STEREO16:
> +        return 4;
> +    }
> +    return 0;
> +}

This information can be stored in a table

struct al_format_entry {
   ALCenum al_fmt, enum AVSampleFormat sample_fmt; int nb_channels; int sample_size;
} al_format_entries[] = {
  ...
}

sample_size can be computed from nb_channels and sample_fmt by using
av_get_bits_per_sample_fmt().

> +
> +/**
> + * Get the OpenAL error state.

> + * Returns true if the device has an error, false otherwise.

@return


> + */
> +static inline int al_get_error ( ALCdevice *device, int *error_ret, const char **error_message_ret )

Maybe you can directly return the error code, and somehow simplify the
interface.

> +{
> +    ALCenum error = alcGetError ( device );
> +    switch ( error )
> +    {
> +    case ALC_NO_ERROR:
> +        return 0;
> +    case ALC_INVALID_DEVICE:
> +        *error_ret = AVERROR ( ENODEV );
> +        break;
> +    case ALC_INVALID_CONTEXT:
> +    case ALC_INVALID_ENUM:
> +    case ALC_INVALID_VALUE:
> +        *error_ret = AVERROR ( EINVAL );
> +        break;
> +    case ALC_OUT_OF_MEMORY:
> +        *error_ret = AVERROR ( ENOMEM );
> +        break;
> +    default:
> +        *error_ret = AVERROR ( ~0 );
> +    }
> +    *error_message_ret = alcGetString ( device, error );
> +    return *error_ret;
> +}
> +
> +/**
> + * Sets up the audio device and related data.
> + */
> +static int read_header ( AVFormatContext* ctx, AVFormatParameters *ap )
> +{
> +    al_data *ad = ctx->priv_data;
> +    int error=0;
> +    const char *error_message=0;

> +    AVStream *st = 0;
> +    AVCodecContext *codec = 0;

NULL is preferred over 0 for struct pointers.

> +
> +    ad->sample_format = ( ad->channels==1 ) ? AL_FORMAT_MONO16 : AL_FORMAT_STEREO16;
> +

> +#if 0 && FF_API_FORMAT_PARAMETERS
> +    if ( ap->sample_rate )
> +        ad->sample_rate = ap->sample_rate;
> +    if ( ap->channels )
> +        ad->sample_format = ( ap->channels==1 ) ? AL_FORMAT_MONO16 : AL_FORMAT_STEREO16;
> +#endif

maybe this can be safely skipped

> +
> +    /* Open device for capture */
> +    ad->device = alcCaptureOpenDevice (
> +                     ctx->filename[0] ? ctx->filename : 0,
> +                     ad->sample_rate,
> +                     ad->sample_format,
> +                     SSIZE
> +                 );
> +
> +    /* Did we fail? */
> +    if ( al_get_error ( ad->device, &error, &error_message ) ) goto fail1;
> +
> +    /* Create stream */
> +    if ( ! ( st=av_new_stream ( ctx, 0 ) ) )
> +    {
> +        error=AVERROR ( ENOMEM );
> +        goto fail2;
> +    }
> +
> +    /* We work in microseconds */
> +    av_set_pts_info ( st, 64, 1, 1000000 );
> +
> +    /* Set codec parameters */
> +    codec = st->codec;
> +    codec->codec_type = AVMEDIA_TYPE_AUDIO;
> +    codec->sample_rate = ad->sample_rate;
> +    codec->channels = al_format_channels ( ad->sample_format );
> +    codec->codec_id = al_format_codecid ( ad->sample_format );
> +
> +    /* These are needed to read the audio data */
> +    ad->sample_size = al_format_sample_size ( ad->sample_format );
> +    ad->buffer_capacity = sizeof ( ad->buffer ) /ad->sample_size;
> +
> +    /* Finally, start the capture process */
> +    alcCaptureStart ( ad->device );
> +
> +    return 0;
> +
> +    /* Handle faliure */
> +fail2:
> +    alcCaptureCloseDevice ( ad->device );
> +fail1:
> +    if ( error_message )
> +        av_log ( ctx, AV_LOG_ERROR, "OpenAL Error: %s\n", error_message );
> +    return error;
> +}
> +
> +/**
> + * Read audio data and store it in a packet.
> + */
> +static int read_packet ( AVFormatContext* ctx, AVPacket *pkt )
> +{
> +    al_data *ad = ctx->priv_data;
> +    int error=0;
> +    const char *error_message=0;
> +    ALCint num_samples;
> +
> +    /* Read samples from device... */
> +    alcGetIntegerv ( ad->device, ALC_CAPTURE_SAMPLES, ( ALCsizei ) sizeof ( ALCint ), &num_samples );
> +    if ( al_get_error ( ad->device, &error, &error_message ) ) goto fail;
> +
> +    if ( num_samples > ad->buffer_capacity )
> +        num_samples = ad->buffer_capacity;
> +    alcCaptureSamples ( ad->device, ad->buffer, num_samples );
> +    if ( al_get_error ( ad->device, &error, &error_message ) ) goto fail;
> +
> +    /* ...and use them to fill a packet */
> +    av_init_packet ( pkt );
> +    pkt->data = ad->buffer;
> +    pkt->size = num_samples*ad->sample_size;
> +    pkt->pts = av_gettime();
> +
> +    return pkt->size;
> +fail:
> +    /* Handle failure */
> +    if ( error_message )
> +        av_log ( ctx, AV_LOG_ERROR, "OpenAL Error: %s\n", error_message );
> +    return error;
> +}
> +

> +/**
> + * Shuts down the audio device and related data.
> + */

Nit: third person, also no need to document standard callback imo.

> +static int read_close ( AVFormatContext* ctx )
> +{
> +    al_data *ad = ctx->priv_data;
> +
> +    if ( ad->device )
> +    {
> +        alcCaptureStop ( ad->device );
> +        alcCaptureCloseDevice ( ad->device );
> +    }
> +    return 0;
> +}
> +
> +static const AVOption options[] =
> +{
> +    {"channels", "", offsetof ( al_data, channels ), FF_OPT_TYPE_INT, {.dbl=2}, 1, 2, AV_OPT_FLAG_DECODING_PARAM },
> +    {"sample_rate", "", offsetof ( al_data, sample_rate ), FF_OPT_TYPE_INT, {.dbl=44100}, 0, INT_MAX, AV_OPT_FLAG_DECODING_PARAM },
> +    {NULL},
> +};
> +
> +static const AVClass class =
> +{
> +    .class_name = "openal",
> +    .item_name = av_default_item_name,
> +    .option = options,
> +    .version = LIBAVUTIL_VERSION_INT
> +};
> +
> +AVInputFormat ff_openal_demuxer =
> +{
> +    "openal",
> +    .long_name =
> +    NULL_IF_CONFIG_SMALL ( "Capture using OpenAL" ),
> +    .priv_data_size = sizeof ( al_data ),
> +    .read_probe = NULL,
> +    .read_header = read_header,
> +    .read_packet = read_packet,
> +    .read_close = read_close,
> +    .flags = AVFMT_NOFILE,
> +    .priv_class = &class
> +};

Thanks for the patch.
-- 
FFmpeg = Friendly & Fostering Most Ponderous Exciting Gargoyle