[FFmpeg-devel] [RFC] H.264: Supporting Recovery Point SEIs

Jason Garrett-Glaser darkshikari
Sat Dec 26 05:28:37 CET 2009

On Fri, Dec 25, 2009 at 4:24 PM, Michael Niedermayer <michaelni at gmx.at> wrote:
> On Fri, Dec 25, 2009 at 02:56:57PM -0500, Jason Garrett-Glaser wrote:
>> x264 will soon be committing Periodic Intra Refresh, an extremely
>> powerful feature for low-latency, high-resilience streaming. ?This
>> will involve using SEI Recovery Points as seek points in H.264 video
>> files. ?This already works partially with container formats like MKV
>> and MP4 that flag keyframes. ?However, there is the following, massive
>> gotcha:
> I see no reason why it would work better with mp4/mkv than others.
> But i might be unaware of some changes ...

If using, for example, raw Annex B, keyframes are not flagged as such
in the bitstream, and unless the decoder explicitly looks for SEI
Recovery Points, it won't know where it can and can't seek, since the
recovery points are not on IDR frames.

>> The standard specifies that, upon seeking to an SEI Recovery Point,
>> that the decoder shall not display the video until recovery_frames_cnt
>> frames have been decoded.
>> This is relied upon heavily in Periodic Intra Refresh.
>> How should this be implemented in ffmpeg? ?Should the decoder return 0
>> for every frame decoded after seeking until recovery_frames_cnt is
>> reached?
> with the current API
> it should set got_picture_ptr=0 and return the numer of bytes consumed,
> which is what you meant i suspect
> But there might be a more flexible and powerful solution.
> Add a int flags to FF_COMMON_FRAME (thats AVFrame)
> then add a AVFRAME_FLAG_DO_NOT_DISPLAY and set that for such frames.
> That way an application can for debuging or otherwise display such frames
> at the users request. These flags would also be extreemly usefull to
> inform the user application if there where concealed decoding errors in a
> frame (a thing some people requested ...)

On a related note, how should we make ffmpeg not print errors for the
frames consumed before display?  These will have tons of "error
concealment" due to missing reference frames, but inherently those
errors will be gone by the time one reaches the end of the
recovery_frame_cnt period (well, as long as exact_match_flag == 1).
It seems like bad form to spam the user with errors despite the fact
that they are a for-sure part of the decoding process that will always
happen in this case.

Dark Shikari

More information about the ffmpeg-devel mailing list