[FFmpeg-devel] [Bounty] Impl. check if input h264 P/B frame ref's frame prior to last I

[Andrey Utkin]

On Mon, 22 Feb 2016 22:11:37 +0100
Andrea Lazzarotto <andrea.lazzarotto at gmail.com> wrote:

> 2016-02-22 22:02 GMT+01:00 Andrey Utkin <andrey_utkin at fastmail.com>:
> 
> > In yet other words, it is about defining bounds on which the
> > temporal piece of H.264 video can be cut out and correctly
> > decoded/played without any additional data. With ultimate
> > correctness. 
> 
> Ah, I see. Coincidentally, during these days I made a Python script
> for smart rendering (using ffmpeg, and re-rendering only GOPs that I
> was splitting) and I was wondering why it was working sometimes ok and
> sometimes completely wrong.
> 
> I am not an expert, but I deduce this might be the reason.
> 
> I read somewhere that ffmpeg can use the "select" filter (if it is
> called like that) that can distinguish between I frames and "SI"
> frames and that only I frames shall be considered "safe". Is that the
> problem or is it a different one?
> 
> Best regards
> 

"select" filter is needed if what you really want to do is to filter
and process the filtered frames.
What both me an probably also you are trying to do is to handle frames
conveniently without resorting to reencoding. So only demuxing, parsing
and decoding is what we would want to use.
I cannot describe the problem better than the helpful #ffmpeg-devel
inhabitants. I hope the following quote of our IRC conversations helps.

‎[13:25:55] ‎andrey_utkin‎: so is it safe to assume that GOPs (I-frame + all following non-I-frames sequence) are self-contained and can be replaced? Or this is true only in terms of "IDR-frame + all following non-IDR-frames sequence"?
‎[13:26:50] ‎Mavrik‎: Only true in terms of IDR frames.
‎[13:27:05] ‎Mavrik‎: Frames that follow an IDR frame may not reference frame before an IDR frame.
‎[13:27:31] ‎jkqxz‎: No.  There still might be something after another I frame which references something in the section you've removed.
‎[13:27:56] ‎jkqxz‎: (Unless the end of the section is also at an IDR frame.)
‎[13:29:06] ‎jkqxz‎: That is, you can have exactly the same problem as you initially described at the end as well as at the beginning.
‎[13:29:14] ‎Mavrik‎: Indeed.
‎[13:29:24] ‎Mavrik‎: So essentially you slice on IDR boundaries.
‎[13:30:00] ‎jkqxz‎: (And you aren't saved by decode order, because the problematic frame is after the next I frame in decode order but before in display order.)
[13:34:08] ‎andrey_utkin‎: ok so the problem is actually about P/B frames which may reference previous GOPs?
‎[13:34:56] ‎Mavrik‎: "GOP" is a useless distinction when dealing with H.264 streams.
‎[13:35:12] ‎jkqxz‎: If you have a well-defined GOP structure, yes.  (The problem is really that H.264 allows arbitrary referencing and there isn't any external information to work out what's going on.)
‎[13:35:17] ‎Mavrik‎: All the references care about is frame count and existence of IDR frames.
‎[13:35:44] ‎Mavrik‎: Usually a GOP is bounded by a pair of IDR frames, but static GOPs aren't really a thing.

Another helpful person shared another bit of knowledge on #x264:

‎[22:51:25] ‎Gramner‎: being able to seek to a frame doesn't imply you can cut on it. there's stuff like SEI recovery points as well, not just IDR-frames
‎[22:54:33] ‎Gramner‎: see https://ffmpeg.org/pipermail/ffmpeg-devel/2009-December/062957.html and https://git.videolan.org/?p=x264.git;a=commit;h=cde39046222b112261179144033e7a51430783d0