[FFmpeg-devel] Google Summer of Code participation

Michael Niedermayer michaelni
Mon Mar 30 19:46:31 CEST 2009

On Mon, Mar 30, 2009 at 04:23:02PM +0200, Thilo Borgmann wrote:
> Michael Niedermayer schrieb:
>> On Mon, Mar 30, 2009 at 12:33:01AM +0200, Thilo Borgmann wrote:
>>> Vitor Sessak schrieb:
>>>> Thilo Borgmann wrote:
>>>>> Kostya schrieb:
>>>>>> On Sat, Mar 28, 2009 at 12:20:32PM +0100, Thilo Borgmann wrote:
>>>>>> [...]
>>>>>>> no. 12: "16-bit Interplay Video Decoder"
>>>>>>> Sounds interesting and as there is a working 8-bit decoder, wich 
>>>>>>> throws some errors if operating on the 16-bit demo file, there seems 
>>>>>>> to be a good starting point. Unfortunately, the section about 16-bit 
>>>>>>> opcodes is far from useful, if opcodes would have to be changed, this 
>>>>>>> task becomes very difficult...
>>>>>>> http://wiki.multimedia.cx/index.php?title=Interplay_Video
>>>>>>> -> The suggestion in the wiki is to ask the development community 
>>>>>>> (you!) about a good start-off with that problem. So I do: Anyone who 
>>>>>>> can help on this? Anyone with more information about the 16-bit mode?
>>>>>>> (Already inspected the code here... sounds like altering the 
>>>>>>> #define'd functions but how to know if it is 8-bit palette mode or 
>>>>>>> 16-bit whatsoever mode?)
>>>>>>  I suppose, it's only Google who knows more about this format.
>>>>>> Maybe some container parameters should give demuxer a hint on whether 
>>>>>> this
>>>>>> is 16-bit mode or not.
>>>>>>> no. 23: "CorePNG Decoder"
>>>>>>> Well, the descriptions sound quite easy, BUT the current svn version 
>>>>>>> of ffmpeg (as well as my very old one) say that it "could not find 
>>>>>>> codec parameters". Thus, I suppose there is no existing PNG1 decoder 
>>>>>>> wich decodes RGB I-Frame video? Is there a PNG1 coded RGB I-Frame 
>>>>>>> demo video? So this task seems to me like implementing a whole 
>>>>>>> decoder on top of the png image decoder?
>>>>>>> http://samples.mplayerhq.hu/V-codecs/PNG1/
>>>>>>> -> Seems to be a not that difficult task but to have a complex 
>>>>>>> start-off.
>>>>>>> Anyone has an idea about these?
>>>>>> I've looked at it once. If you use PNG decoder for that frames you'll 
>>>>>> see
>>>>>> good picture for the first frame and many colourful dots on the 
>>>>>> latter. That's
>>>>>> because its developer(s) decided to store latter frames as a 
>>>>>> difference to
>>>>>> the key frame and you have to use demuxer keyframe flag to distinguish 
>>>>>> between
>>>>>> both.
>>>>>> Download http://samples.mplayerhq.hu/V-codecs/PNG1/corepng.avi
>>>>>> run
>>>>>> ffmpeg -i corepng.avi -f image2 -vcodec copy %04d.png
>>>>>> and see that for yourself.
>>>>>> P.S. We of FFmpeg welcome creativity since it's not that rare when the 
>>>>>> only
>>>>>> information about codec is located in libavcodec/*.c
>>>>> Ok next problem arises. I'm currently trying to distinguish the I and P 
>>>>> frames to be decoded.
>>>>> As you mentioned, I tried to check the keyframe flag, that comes from 
>>>>> 'outside'.
>>>>> For that, I "av_log"ed the "key_frame" and "pict_type" attributes:
>>>>>  >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> static int decode_frame(AVCodecContext *avctx,
>>>>>                        void *data, int *data_size,
>>>>>                        const uint8_t *buf, int buf_size)
>>>>> {
>>>>>    PNGDecContext * const s = avctx->priv_data;
>>>>>    AVFrame *picture = data;
>>>>>    AVFrame * const p= &s->picture;
>>>>> #ifdef DEBUG
>>>>> av_log(avctx, AV_LOG_DEBUG, "data->key_frame: %i, data->pict_type: 
>>>>> %d\n",
>>>>>       picture->key_frame, picture->pict_type);
>>>>> av_log(avctx, AV_LOG_DEBUG, "s->picture->key_frame: %i, 
>>>>> s->picture->pict_type: %d\n",
>>>>>       p->key_frame, p->pict_type);
>>>>> #endif
>>>>>  >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> (source listing is highly mailing list dependent - tell me how you like 
>>>>> it if this is a pain in your eyes)
>>>>> As these debug outputs are telling that every frame is a key frame, 
>>>>> which most of them are not, the question is if I'm looking at the wrong 
>>>>> attributes or if this information is not parsed at all?
>>>> There are three possibilities:
>>>> 1- This information is coded somewhere in the AVI format (and thus it is 
>>>> the work of the AVI demuxer to read it and set the parameters).
>>>> 2- This information is in some reserved bits of the PNG stream (and so 
>>>> it is in pngdec.c that it should be read).
>>>> 3- This information is nowhere and the codec follow some stupid 
>>>> convention like one frame every X are key-frames.
>>>> I suggest you to look in the available documentation/source code to know 
>>>> where this information is supposed to be. I find (1) unlikely since 
>>>> ffmpeg AVI demuxer is not so buggy, and it would set picture->key_frame 
>>>> correctly...
>>> Found something here. I think, the avi dumexer does its job and reads the 
>>> keyframe flags. I tested this by logging the PKT_FLAG_KEY or'ed into the 
>>> 'flags' attribute of the 'AVPacket' structure (libavcodec/avidec.c: 
>>> avi_read_packet)
>>> This detected a keyframe at the 1st, 7th, 8th, 21st and 22nd frames of 
>>> the stream, which makes sense as far as I can tell.
>>> Ok, but unfortunately, the AVPacket structure is not passed completely to 
>>> the decoder (libavcodec/pngdec.c: decode_frame()), just the attributes 
>>> 'data' and 'size'. (attribute 'flags' hold the keyframe information, as I 
>>> said above). These attributes are passed at the top level - ffplay.c: 
>>> video_thread().
>>> Next to this, an 'AVCodecContext' is passed to the decoder function. I've 
>>> already tried to get the keyframe information out of them but they 
>>> declare al frames to be keyframes. (see above for that)
>>> As far as I understand the code now, this AVCodecContext is created once 
>>> for each stream and therefore can not carry the information per frame. 
>>> (This ist right, isn't it?)
>>> Well, the keyframe flag is not coded in the PNG buffer either. So I do 
>>> not see a chance to get the keyframe flag from the packages read by the 
>>> demuxer into the decoder function. At least, not without changes at top 
>>> level, which means at ffplay.c which is not an option in my eyes.
>> [...]
>>> So, I wonder how other codecs get told about a keyframe they have to 
>>> decode...
>> other codecs store the keyframe info in the frames passed to the decoder.
>> btw, where is the source for corepng? i mean not a win32 self extracting
>> thing
> It can be selected during the codec installation. Comes as a rar-archive 
> afterwards. Attached it for convenience. The part that selects keyframes in 


> that can be found in the vfw/VFWhandler.cpp in the functions: 
> DecompressXXXFrame. Stored in the flags as ICDECOMPRESS_NOTKEYFRAME. As far 
> as my search revealed, this is supported by the AVI file functions of video 
> for windows.
>>> As you mentioned, the demuxer detects the keyframes but the 
>>> picture->key_frame is not set. I checked this by my first attempt in 
>>> reading ist value (was always set to 'keyframe') and by looking into the 
>>> avidec.c (which is the avi demuxer, or am I completely wrong about 
>>> that?), where the key_frame attribute is never touched.
>>> Though, what would be the preferred way to carry this information from 
>>> the package-struct to the decoder function? (I hope I just missed a clue 
>>> somewhere but I looked through the code the whole day...)
>> If it really is not stored in the frames there are only 3 ways
>> 1. change every muxer and demuxer that allows corepng to add&remove a
>> byte specifying the keyframe and fix pngdec/enc to handle that byte
>> 2. add a avcodec_decode_video2() that takes a AVPacket and update
>> ffmpeg.c and ffplay.c (this was planned since a long time and is welcome)
>> 3. try I & P and compare which is closer to the previous frame
>> this if course is not 100% but easy
> 1) Well for the decoding it might not make a difference in my eyes, but 
> when it would come to encoding, this would not comply to the original 
> CorePNG anymore, would it?

well, the avi muxer would have to remove the byte again at least, this of
course is becomes ugly quickly if more (de)muxers use this format.

The real problem is that these things are designed too quickly and without
the video/audio expert community being aware, so that when such problems
are found, its too late to fix.

Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

Why not whip the teacher when the pupil misbehaves? -- Diogenes of Sinope
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
URL: <http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/attachments/20090330/18dc257a/attachment.pgp>

More information about the ffmpeg-devel mailing list