[FFmpeg-devel] [PATCH] avformat: add AV1 RTP depacketizer and packetizer

[Chris Hodges]

Hi Tristan,

I have a draft reply ongoing the last couple of days, but I got 
sidetracked with other work, sorry. Will send it out without a new patch 
so we can discuss the changes and it gives me more time to look into the 
issue below.

On 12/11/24 23:53, Tristan Matthews via ffmpeg-devel wrote:

> Aside from my minor comments on the patch, one thing that came up in terms of testing is that while the packetization seems to be working well, I'm unable to depacketize a stream being sent by ffmpeg *with* ffmpeg as the receiver.

There was a bug that triggered on fragmented OBUs >= 16384 bytes if it 
was not the last OBU in the TU:

                  // move existing data for OBU back a bit
                  memmove(lebptr + extra_bytes, lebptr,
                          pkt->size - extra_bytes - data->frag_pkt_leb_pos);
+                // move pktpos further down for following OBUs in same 
packet.
+                pktpos += extra_bytes;

This might have been one of the problems.

> This is fairly easy to reproduce locally, start an RTSP server:
> $ docker run --rm -it --network=host bluenviron/mediamtx:latest
> 
> Push an RTP stream:
> $ ./ffmpeg -re -i Chimera-AV1-8bit-1280x720-3363kbps.ivf -c copy -strict experimental -f rtsp rtsp://127.0.0.1:8554/foobar
> 
> Pull the same stream:
> $ ./ffmpeg -i rtsp://127.0.0.1:8554/foobar -copy -f matroska foobar.mkv

Thanks for these examples, helps a lot!

> This fails with a bunch of errors on trying decode e.g.:
> [libdav1d @ 0x5c50bb8d1580] No sequence header available: unable to decode frame header.
> [libdav1d @ 0x5c50bb8d1580] obu_forbidden_bit out of range: 1, but must be in [0,0].
> etc.

I don't seem to be able to replay the Chimera video directly with my 
local build of ffplay, get's all choppy and skips big deals of the 
video. Might be an outdated libdav1d?

Anyway, the AV1 RTP decoder currently does not wait for a keyframe 
before sending down frames. Should it do that? Also, should it wait for 
another keyframe if any frame or packet had been dropped (the GOP is 
going to be useless for the decoder anyway)?

> The good news is that I *can* pull that same stream and save it to disk with e.g. gstreamer so it's clearly not that far off. It's not clear to me yet if this is an issue on the packetization side, the intermediate RTSP server side, the depacketization side or some combination therein.

I haven't been working with such complex frame hierarchies, but I will 
try to get the Chimera working.

-- 
Best regards, Chris