[FFmpeg-devel] Fixing av_seek
donmoir at comcast.net
Mon Jan 23 21:47:21 CET 2012
>>> Finnally getting around to looking more closely at seek problems in
>>> This is a follow up to ticket #504.
>>> My goal is to write a higher level seek function that depends on the
>>> lower level ffmpeg seek functions. In some cases, patches to ffmpeg may
>>> need to be applied.
>>> The higher level function will only deal with a timestamp that might
>>> correspond to a slider or similiar, and will seek very close to that
>>> timestamp without image distortion.
>>> There are several categories of success or failure when seeking. This
>>> first group (Group A), deals with things that work or are easy to deal
>>> o - everything works and works well. There are plently of index_entries
>>> and the format in question does the right thing. CODEC_ID_FLV1 is an
>>> example that works well but some other formats work just as well.
>>> o - there are no index_entries or none of the index_entries are marked
>>> as AVINDEX_KEYFRAME. This will cause avformat_seek_file or av_seek_frame
>>> to fail. If there are index_entries and the first entry represents the
>>> first frame, then I have found that just marking this first index_entry
>>> with AVINDEX_KEYFRAME fixes the ffmpeg seek failure problem. If there
>>> are no index_entries then you can add one using av_add_index_entry using
>>> the correct parameters for the first frame and marking it as
>>> AVINDEX_KEYFRAME. This assumes the first frame must be a keyframe and
>>> easier to deal with this at startup rather then trying to deal with it
>>> when doing a seek.
>>> o - lower level seek functions mostly deal with keyframes and thats all
>>> good, but thats not good enough for an end user trying to use a slider
>>> to seek. So get as close as possible using one of the ffmpeg seek
>>> functions and then proceed forward to the timestamp of interest. This
>>> works pretty good and is fast enough. Mostly, good files will have
>>> enough AVINDEX_KEYFRAME index_entries to make this behave quite well.
>>> Even when there are not enough entries, it works good. You probably want
>>> to do this in a separate thread. Make sure you first seek to a timestamp
>>> that is less than or equal to the timestamp of interest so you can
>>> proceed forward. There are probably ways to improve on this.
>>> The second group (Group B) falls into the class of failures or bad
>>> behavior that is not easy to deal with. e.g. an inability to make it
>>> work as expected for various reasons.
>>> I am still dealing with trying to find out Group B problems. I have had
>>> issues with MPEG1, THEORA, and a few others, and I will know more about
>>> these problems as I get into it further.
>>> I use VLC and SMPlayer to cross check and both of these have their share
>>> of seek problems. I use my own player to test with.
>> Using this h264 file, the index_entries are built on the fly and appear
>> to be incorrect.
>> http://sms.pangolin.com/temp/bad_seek_index_h264.zip (2.5mb)
>> codebase: git-67f5650 - after fixes with h264 seeking but I don't see any
>> difference with older codebase versions than this one.
>> Just after av_open_input_file, there are 2 index_entries. The first one
>> has timestamp of 0 and 2nd has timestamp of 0x48f. The odd thing is they
>> both have the same pos which 0x122F. So seems strange from the start. As
>> playback progresses, the number of index_entries increases and this is
>> When you try to seek on this file to a supposed keyframe, you don't get a
>> finished frame until much later (~4 seconds). What should happen is if
>> you have seeked to a keyframe, you should be getting a finished frame
>> immediately. This is the way I see it at any rate. So trying to seek into
>> this file breaks the playback, meaning you will get a long pause before
>> normal playback resumes because you are not getting a finsihed frame when
>> you should be. If I just seek to zero on this file, then all is good and
>> I get the finished frames as expected but of course it's slower to reach
>> the requested timestamp.
>> I bring this up because I think this represents a key problem with some
>> of the codec types. That is index_entries are added that don't click with
>> actual keyframes. I asked a question about this relationship on the
>> original ticket but there was no response.
>> I think getting this fixed will go a long way in fixing several seek
>> problems. But I need information from you if you have any on this. If you
>> don't have a answer, maybe "I or we don't know" would be better than no
>> answer. Also not to just focus on h264 here because I believe there are
>> other formats with this very same problem.
> This file identifies some problems with the ogg decoder when seeking:
> http://sms.pangolin.com/temp/bad_seek_ogg_BuckBunny.zip (46 mb)
> I am not making any attempt to cut some of these samples down to make sure
> all is relevant.
> This file plays fine in my own player without any problems. SMPlayer
> chokes on it abit, then a reload and it plays ok. ffplay plays it fine
> with maybe a few artifacts.
> The problems come in when you try to seek on it. The ogg decoder has just
> a basic seek that calls ff_seek_frame_binary. This can be found in
> libavformat\oggdec.c in the function ogg_read_seek. Seems seek was kind of
> an after thought and patched in to the orignial code.
> o - unlike many of the other decoders, the ogg decoder makes no attempt to
> build it's index_entries when you do a seek. This is the first cause of
> failure. Since it does not have these entries, it will fail in
> ff_seek_frame_binary if the file has not been read sufficiently to create
> it's index. Some ogg files work ok in this regard but BuckBunny will fail.
> I currently go ahead and build these entries if needbe before I call any
> seek for an ogg decoder file and this is a start to get this working.
> BuckBunny is THEORA but ogg decoder handles several CODEC id's.
> o - The ogg decoder keeps a private structure in AVFormatContext.priv_data
> (struct ogg) and within this it keeps some private stream information
> (struct ogg_stream). In the ogg_stream it keeps track of lastdts and
> lastpts. When you do a seek everything seems to be ok like position
> information etc. But these 2 variables are not reset to AV_NOTPTS_VALUE
> and they contain the values that were in effect before you did the seek.
> So on the first read after a seek, you will get the right packet but the
> pts and dts values will be wrong for the packet. No amount of flushing etc
> seems to fix it and this is just a bug. I believe the ogg_read_seek
> functions just needs to set these 2 vairaibles to the proper values. With
> the second read after the seek the dts and pts values seem ok.
> For my own code I fixed the above issues and now seeking for this file
> works fine. I still have some painting artifacts but this is more of a
> general problem and I will deal with that later.
> I develop under windows because thats what most of our users use. Some mac
> too. My experinece with linux is somewhat limited but I used to run that
> using vmware. The application I use to test ffmpeg is designed for windows
> and so it's a bit of a pain to debug and trace thru ffmpeg under windows
> but this is just busy work.
> I can idenify and pinpoint these various problems but I am just not setup
> to provide proper patches. I have people yelling at me already about
> spending too much time on this and I have to earn a living. So while I
> will continue to do this and idenify the problems, I will have to leave it
> up to you to do the patches. Finding the problems is the most work anyway.
> So the 2 things for the ogg decoder is to make sure the index_entries are
> built before you do a seek. Most of the other decoders do this correctly.
> Make sure the lastpts and lastdts variables (struct ogg_stream) are set to
> proper values in the ogg_read_seek function. ogg_read_seek should also
> make sure the index_entries are built up to the timestamp of interest at
> least before calling ff_seek_frame_binary. Note: just reading the packets
> with no decoding is quick and sufficient to build the index_entries. I
> don't do it if sufficient entries are built alreadly and stop when I have
> built up to the timestamp of interest. These entries are normally built on
> the fly as you read packets when processing the file. I think some
> decoders mght have more elegant ways to do it but this is a good fallback.
> ffplay complains about first frame not being key frame when you try to
> seek with ffplay but that doesn't seem to be a problem and may be related
> to the above bugs. Seeking fails badly in SMPlayer for this file and
> generally seeking does not work well at all in ffplay for this file. For
> my own app with the above fixes, it now works perfect outside of some
> artifacts but thats a different issue.
The attached file contains the code fragments along with comments that I am
using to fix seeking with the ogg decoder. Not a patch but details the
basics on fixing seeking for ogg related files. I tested it with a couple
THEORA files and so far so good. This would need to be applied to
libavformat\oggdec.c. The attached file contains a function for creating the
index_entries if needbe and sets the lastpts and lastdts to AV_NOPTS_VALUE
after a successful seek.
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
More information about the ffmpeg-devel