[FFmpeg-devel] Maintainership question

[Måns Rullgård]

Nicolas George <nicolas.george at normalesup.org> writes:

> Now, the first question: should this timestamp guessing logic exist at all?
>
> After a client application calls avcodec_decode_videoX, it has no less than
> four potential timestamps for this frame:
> - the packet PTS,
> - the packet DTS,
> - the reordered PTS,
> - the codec timestamp.

There is no such thing as a "codec timestamp".

> The packet PTS is always wrong, of course, the reordered PTS supersedes it
> completely. We know it because we have a good understanding of the problem.
>
> But the three others are all valid candidates to be "the" frame timestamp.
> And depending on the format, on the codec, on the actual file, etc., any
> subset of them can be undefined. Plus, the codec timestamp is sometime in a
> different time base than the PTS and DTS. And do not forget that the real
> meaning of the DTS is hardly explained anywhere.
>
> Do you expect all client applications to do the right thing with such a
> complex mess? I do not: most will do things wrong and get jerky playback
> when dealing with B-frames or VFR content.

There is no mess if implemented properly.  Libavformat is not and is
therefor a mess.

>> - the logic used to "guess" pts values is highly questionable and
>> should probably only be used for certain input formats, and possibly
>> only some codecs in those input formats, and that logic is completely
>> missing right now (and also from the original implementation in
>> ffmpeg/ffplay.c)
>
> Could you elaborate on this "highly questionable".
>
> The logic of the current code is this: given the DTS and reordered PTS, if
> only one of them is defined, use it; if both of them are defined, use the
> one that has shown the less discontinuities in the past.

Formats like MPEG with sparse timestamps cause trouble for such heuristics.

> If I understand things correctly, when both the rPTS and DTS are available,
> they should be equal. If they are, the "guessing" logic guesses nothing, it
> just uses them. And if they are not, something is already wrong before the
> guessing logic kicks in.
>
> If the wrong timestamps are somehow generated/mangled by lavf, of course,
> lavf should be fixed. M?ns suggested that this happens, but never answered
> to my request of a sample (2011-02-01 21:09:21 +0100).

Pick any mkv file from samples.ffmpeg.org

> OTOH, if the wrong timestamps are in the file itself, because of a broken
> muxer, we can do nothing about it, and only try to make the better of them.
> Thus the "guessing" logic.

There are two classes of bad timestamps: wrong and impossible.  Wrong
timestamps simply cause bad A/V sync or similar, while impossible could
be for example non-increasing or otherwise inconsistent.  The former
cannot be detected automatically.  In the latter case, the proper
treatment depends on the reason for the timestamps being bad in the
first place.  If random data corruption is the cause, simply discarding
outliers tends to work well.  In other cases, a reversal of the error
may be possible if there is a known pattern to the error.  In all cases,
recovery of accurate timestamps is impeded by a dumb heuristic tampering
with them first.

-- 
M?ns Rullg?rd
mans at mansr.com