[FFmpeg-devel] [PATCH] avcodec/mediacodecenc: Extract configOBUs from AV1CodecConfigurationRecord

[James Almer]

On 10/4/2024 2:54 PM, Zhao Zhili wrote:
> From: Zhao Zhili <zhilizhao at tencent.com>
> 
> MediaCodec can generate AV1CodecConfigurationRecord, which shouldn't
> be put into packet->data. Skip four bytes and extract configOBUs
> if it exist.

But why are you removing them from extradata if you only want to prevent 
them to make it to a packet?
You could instead make sure they are not dumped into the packet in the 
memcpy(pkt->data... below this chunk.

> ---
> I did some test on Pixel 8 Pro. AV1 hardware encoding works with a lot
> of bugs:
> 
> 1. It's broken for width non-aligned to 16. For width 1080 and pixel
> format YUV420P, MediaCodec use 1080 as stride. For pixel format NV12,
> MediaCodec use 1088 as stride. There is no API to get the stride info.
> AMEDIAFORMAT_KEY_STRIDE doesn't work. And set stride to MediaCodec has
> no effect from my test, at least on that device. We know the buffer
> size provided by MediaCodec, but we still cannot get stride by
> buf_size / height:
> 
>    1) For YUV420P, buf_size = 1080 * height
>    2) For NV12, buf_size = 1080 + 1088 * (height - 1). Yes, buf_size doesn't
> count last line's padding :(
> 
> 2. After flush (which means reset for MediaCodec), it doesn't reset GOP
> info, so it output a few non-key frames before generate a key frame.
> This breaks what I did for AV_CODEC_FLAG_GLOBAL_HEADER: send a dummy
> frame, send EOF, get packet and extract extra data, then reset
> MediaCodec state by AMediaCodec_flush.
> 
>   libavcodec/mediacodecenc.c | 10 ++++++++++
>   1 file changed, 10 insertions(+)
> 
> diff --git a/libavcodec/mediacodecenc.c b/libavcodec/mediacodecenc.c
> index e76ea81236..3880ea2fe9 100644
> --- a/libavcodec/mediacodecenc.c
> +++ b/libavcodec/mediacodecenc.c
> @@ -429,6 +429,16 @@ static int mediacodec_receive(AVCodecContext *avctx, AVPacket *pkt)
>       }
>   
>       if (out_info.flags & ff_AMediaCodec_getBufferFlagCodecConfig(codec)) {
> +        if (avctx->codec_id == AV_CODEC_ID_AV1) {
> +            // Skip AV1CodecConfigurationRecord without configOBUs
> +            if (out_info.size <= 4) {
> +                ff_AMediaCodec_releaseOutputBuffer(codec, index, false);
> +                return mediacodec_receive(avctx, pkt);
> +            }
> +            out_info.size -= 4;
> +            out_info.offset += 4;
> +        }
> +
>           ret = av_reallocp(&s->extradata, out_info.size);
>           if (ret)
>               goto bailout;

-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature.asc
Type: application/pgp-signature
Size: 495 bytes
Desc: OpenPGP digital signature
URL: <https://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20241004/43ba3b02/attachment.sig>