[FFmpeg-devel] [PATCH v3] avformat: add AV1 RTP depacketizer and packetizer

[Chris Hodges]

Hi Tristan,

wow, that was a timing coincidence, I just started to write the mail 
with the new patch...

On 12/13/24 14:44, Tristan Matthews via ffmpeg-devel wrote:

>> Do the other implementations you mentioned have that kind of information
>> by parsing OBUs or from the AV1 encoder?
> 
> In both they do it by parsing OBUs:
> 
> - This boolean is later used to build the aggregation header: https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/blob/main/net/rtp/src/av1/pay/imp.rs?ref_type=heads#L285
> 
> - Here they are storing the OBUs in memory and have access to that info directly:
> https://chromium.googlesource.com/external/webrtc/+/HEAD/modules/rtp_rtcp/source/rtp_packetizer_av1.cc#346

I see. I've put the code in an #if <somedefine> block, so you can try if 
it makes a difference to anything you may encounter.

The spec says: https://aomediacodec.github.io/av1-spec/#ordering-of-obus

"Sequence header OBUs may appear in any order within a coded video 
sequence. Within a particular coded video sequence, the contents of 
sequence_header_obu must be bit-identical each time the sequence header 
appears except for the contents of operating_parameters_info. A new 
coded video sequence is required if the sequence header parameters change."

So if an encoder decides to submit Sequence Header OBUs in any 
non-keyframe to maybe change the operating_parameters_info, this will 
break both GStreamer and WebRTC?

> Nice, looking forward to testing the next iteration.

New version of patch attached.

I also found and fixed one more "indirect" bug that would cause some 
other part in ffmpeg to add an additional TD between OBUs to the 
bitstream. I did not know that I was not supposed to issue the 
depacketized RTP data upstream if the temporal unit was not complete yet 
(although the RTP packet was not fragmented and the output could be 
considered "complete" in the term of data acquired).

-- 
Best regards, Chris
-------------- next part --------------
An embedded message was scrubbed...
From: Chris Hodges <chrishod at axis.com>
Subject: [PATCH] avformat: add AV1 RTP depacketizer and packetizer
Date: Mon, 26 Aug 2024 11:29:29 +0200
Size: 47692
URL: <https://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20241213/c5c35de5/attachment.eml>