[FFmpeg-devel] [PATCH] AVHWAccel infrastructure v2 (take 6) [ping^2]

[Gwenole Beauchesne]

On Sun, 24 Oct 2010, Gwenole Beauchesne wrote:

> Hi,
>
> Ping.
>
> v7 also attached to replace {alloc,free}_surface() hooks with the generic 
> {get,release}_buffer(). If the user implements his own get_buffer() function, 
> he will have to call the hwaccel's one too to initialize data[3] properly.
>
> Regards,
> Gwenole.
>
> On Mon, 20 Sep 2010, Gwenole Beauchesne wrote:
>
>> Hi,
>> 
>> It seems I didn't reply to this one or the message got lost because I had a 
>> v5 lying around for some time. Here is a v6 anyway, that intersects with 
>> the tidsp needs.
>> 
>> On Mon, 22 Feb 2010, Michael Niedermayer wrote:
>> 
>>> On Mon, Jan 04, 2010 at 04:39:02PM +0100, Gwenole Beauchesne wrote:
>>>> Hi,
>>>> 
>>>> On Mon, 4 Jan 2010, Michael Niedermayer wrote:
>>>> 
>>>>>> In the new model I wanted to use, hwaccel_context was to be solely
>>>>>> maintained by libavcodec. However, it still needed a way to get some
>>>>>> hwaccel context initialized by the user-application. e.g. a VADisplay
>>>>>> (void
>>>>>> *), an LPDIRECTXVIDEODECODER, etc.
>>>>>> 
>>>>>> I felt interesting to pass this information through hwaccel attrs along
>>>>>> with other int attributes.
>>>>> 
>>>>> We have existing means to pass information, that is through 
>>>>> AVCodecContext
>>>>> and using AVOption where appropriate
>>>>> As well as using AVFrame instead of AVCodecContext where its per frame
>>>>> data
>>>> 
>>>> OK, I nuked the hwaccel attrs and replaced it with
>>>> AVCodecContext.hwaccel_{id,flags}. hwaccel_flags is handled by AVOption. 
>>>> I
>>>> left hwaccel_id as is to match existing practise (pix_fmt et al.).
>>>> 
>>>> I haven't tested with MPlayer yet but this new approach still works with 
>>>> my
>>>> modified ffplay.
>>>> 
>>>> So, HW accelerator selection works as is:
>>>> - Set AVCodecContext.hwaccel_id to the desired accelerator
>>>> - Set AVCodecContext.hwaccel_flags to HWACCEL_CAP_GET_PIXELS if the user
>>>> wants AVFrame.data[0-2] populated with YV12 or NV12 pixels
>>>> 
>>>> This means we can handle multiple accelerators automatically at once.
>>>> Anyway, this lived in an ideal world without practical use since the
>>>> user-application would still have to handle the HW accelerator
>>>> specifically.
>>>> 
>>>> Regards,
>>>> Gwenole.
>>> 
>>> ]...]
>>>> @@ -2497,7 +2498,7 @@ typedef struct AVCodecContext {
>>>>       * provided by the user. In that case, this holds display-dependent
>>>>       * data FFmpeg cannot instantiate itself. Please refer to the
>>>>       * FFmpeg HW accelerator documentation to know how to fill this
>>>> -     * is. e.g. for VA API, this is a struct vaapi_context.
>>>> +     * is. e.g. for VA API, this is a VADisplay.
>>>>       * - encoding: unused
>>>>       * - decoding: Set by user
>>>>       */
>>> 
>>> hunk ok probably
>>> 
>>> [...]
>>>> +    /**
>>>> +     * Called in codec initialization.
>>>> +     *
>>>> +     * This is used to initialize the AVCodecContext.hwaccel_context
>>>> +     * hardware accelerator context.
>>>> +     *
>>>> +     * @param avctx the codec context
>>>> +     * @return zero if successful, a negative value otherwise
>>>> +     */
>>>> +    int (*init)(AVCodecContext *avctx);
>>>> +
>>>> +    /**
>>>> +     * Called in codec finalization.
>>>> +     *
>>>> +     * This is used to clean-up any data from the hardware accelerator
>>>> +     * context. Should this function be implemented, it shall reset
>>>> +     * AVCodecContext.hwaccel_context to NULL.
>>>> +     *
>>>> +     * @param avctx the codec context
>>>> +     * @return zero if successful, a negative value otherwise
>>>> +     */
>>>> +    int (*close)(AVCodecContext *avctx);
>>>> +
>>>> +    /**
>>>> +     * Called at the beginning of each frame to allocate a HW surface.
>>>> +     *
>>>> +     * The returned surface will be stored in Picture.data[3].
>>>> +     *
>>>> +     * @param avctx the codec context
>>>> +     * @param surface the pointer to the HW accelerator surface
>>>> +     * @return zero if successful, a negative value otherwise
>>>> +     */
>>>> +    int (*alloc_surface)(AVCodecContext *avctx, uint8_t **surface);
>>>> +
>>>> +    /**
>>>> +     * Called to free the HW surface that was allocated with 
>>>> alloc_surface().
>>>> +     *
>>>> +     * @param avctx the codec context
>>>> +     * @param surface the HW accelerator surface
>>>> +     * @return zero if successful, a negative value otherwise
>>>> +     */
>>>> +    void (*free_surface)(AVCodecContext *avctx, uint8_t *surface);
>>>>  } AVHWAccel;
>>> 
>>> iam not in favor of these. What are these good for?
>>> Why are they needed?
>> 
>> This is used to allocate the HW surfaces (VdpSurface, VASurface, etc.) from 
>> within FFmpeg. i.e. the AVFrame.data[3] part, hence the uint8_t * instead 
>> of the void *.
>> 
>>>> +/** Identifies the hardware accelerator */
>>>> +enum HWAccelID {
>>>> +    HWACCEL_ID_NONE,
>>>> +    HWACCEL_ID_VAAPI,           ///< Video Acceleration (VA) API
>>>> +    HWACCEL_ID_NB
>>>> +};
>>> 
>>> this needs a any or auto for autodetection (which should be 0)
>> 
>> I finally forgot about autodetection because it's too complex to handle at 
>> the FFmpeg level and delegated that to the player level. In particular, 
>> HWAccel context may depend on upper level info, like an X display...
>> 
>>>> +
>>>> +/** Identifies the hardware accelerator entry-point */
>>>> +enum HWAccelEntrypoint {
>>>> +    HWACCEL_ENTRYPOINT_VLD = 1, ///< Variable-length decoding
>>>> +    HWACCEL_ENTRYPOINT_IDCT,    ///< Inverse discrete cosine transform
>>>> +    HWACCEL_ENTRYPOINT_MOCO,    ///< Motion compensation
>>>> +    HWACCEL_ENTRYPOINT_NB
>>>> +};
>>> 
>>> as said previously this is ambigous
>>> 
>>> HWACCEL_ENTRYPOINT_VLD could mean the hw does just VLD and leaves IDCT/MC
>>> to software
>> 
>> Actually, I didn't find the exact meaning. But from a pipeline point of 
>> view, this would mean XXX and above. i.e. VLD and above, motion 
>> compensation and above. Flagging those would lead to more player work 
>> (well, just OR'ing more flags).
>> 
>> I tried to express that in the comment.
>> 
>>>> +/**
>>>> + * The hardware accelerator can fetch the pixels from the decoded
>>>> + * frames. If the user requested it, avcodec_default_get_buffer()
>>>> + * will then allocate the data buffers.
>>>> + */
>>>> +#define HWACCEL_CAP_GET_PIXELS 0x00000001
>>> 
>>> might need a AV prefix
>> 
>> OK.
>> 
>> Regards,
>> Gwenole.