[FFmpeg-devel] [PATCH] avformat: add AV1 RTP depacketizer and packetizer

[Chris Hodges]

Hi there,

On 12/13/24 15:09, Tristan Matthews via ffmpeg-devel wrote:

>> I don't seem to be able to replay the Chimera video directly with my
>> local build of ffplay, get's all choppy and skips big deals of the
>> video. Might be an outdated libdav1d?

I still have problems with the media server and the Chimera video. I 
downloaded the IVF file from here:

http://download.opencontent.netflix.com/?prefix=AV1/Chimera/Old/

It seems like the PTS are all over the place for each cut-scene and 
ffmpeg will thus increase the output video rate to infinity which causes 
a slight UDP overflow and drop on the receiving end.

Maybe I'm doing something wrong here, but I wasn't able to fix the PTS 
in the file with ffmpeg so that it was monotonically increasing.

>> Anyway, the AV1 RTP decoder currently does not wait for a keyframe
>> before sending down frames. Should it do that? Also, should it wait for
>> another keyframe if any frame or packet had been dropped (the GOP is
>> going to be useless for the decoder anyway)?
> 
> That's a good question, I don't believe any of the RTP depacketizers in libavformat do this currently so maybe best left as a future enhancement (behind an option/flag).

Sounds reasonable. The implementation in some browsers* seem to refuse 
to playback the AV1 file if the tile data corrupted due to dropped 
frames and will not skip till the next keyframe. This is rather 
unexpected. There is a AV_PKT_FLAG_CORRUPT flag, maybe I should be 
setting this instead?

* for me it works in Firefox but fails in Chrome

> On a related note, one feature that only the VP8 depacketizer supports now is signalling via RTCP that it needs a keyframe (in the event of loss or the start of a stream), to do this you just have to implement the .need_keyframe callback (see https://git.ffmpeg.org/gitweb/ffmpeg.git/blob/eb79c316c7b205f45e7003e91f77c253818e6685:/libavformat/rtpdec_vp8.c#l274)

That's a great feature, can't see why I shouldn't be implementing it. 
Will provide a new patch next year.

>> I haven't been working with such complex frame hierarchies, but I will
>> try to get the Chimera working.
> 
> FWIW I was seeing similar behaviour with smaller/simpler files as well.

Except for the UDP packet drops, causing loss of most of the latter 
scenes past the keyframes (with the current change of dropping all data 
past the point of corruption), I didn't see any stream corruption anymore.

Best regards,

Chris

PS: Will be unavailable over xmas. God jul!

-- 
Chris