[FFmpeg-devel] H264: mp4toannexb BSF and streaming

[Alexandre Ferrieux]

On 18/08/2010 15:14, Luca Abeni wrote:
> On 08/18/2010 02:53 PM, Alexandre Ferrieux wrote:
>> On 18/08/2010 14:47, Luca Abeni wrote:
>>> On 08/18/2010 02:42 PM, Alexandre Ferrieux wrote:
>>>> Hi,
>>>>
>>>> I'm in a (n admittedly suboptimal) situation where I'm receving a
>>>> 3GP/H264 video file in streaming (over a TCP socket, yes...).
>>>>
>>>> I would like to transcode it on-the-fly without waiting for the full
>>>> 3GP
>>>> file to be received. I know that the 3GP format is not AT ALL designed
>>>> for that, and that even the first chunks will be invalid since the
>>>> 3GP-writer application does a seek() back to the beginning to fill the
>>>> header, etc. In short, 3GP is utterly non-streamable, but I need to do
>>>> something with it nevertheless :/
>>>>
>>>> What I've come up with so far, is to detect the beginning of the 'mdat'
>>>> chunk, which comes quickly in the stream, since it contains the bulk of
>>>> the bitstream (in AVC format).
>>>>
>>>> Now, I know that this AVC bitstream is not suitable for input by
>>>> ffmpeg,
>>>> who is expecting annex B. I also know of the mp4toannexb BSF, but I can
>>>> see in its code that it is relying on extradata coming from the
>>>> container (in other chunks, coming at the end), which it will convert
>>>> and interleave between the annex B NALUs it will spit out.
>>>>
>>>> Hence, I conclude that mp4toannexb cannot work on a "pure" AVC
>>>> bitstream
>>>> without extradata.
>>>
>>> As far as I know "AVC" always has extradata, and is not decodable
>>> without
>>> it (I think the extradata contain the SPS and PPS NALs, which are needed
>>> for decoding the H.264 stream). So, I think "pure AVC" without extradata
>>> does not exist (or is not usable).
>>
>> OK, so the contents of the 'mdat' chunk of a 3GP/H264 are not in
>> themselves sufficient for decoding, right ?
>
> Well, I do not know the details of the container (and I do not know what
> an "mdat" chunk is :).
> What I was trying to say is that (AFAIK) to decode an "AVC" H.264 stream
> you need the extradata (since - AFAIK - the "AVC" H.264 syntax stores
> PPS and SPS in the extradata).

OK :)

What I'm trying to do is understand where these "important" extradata are in the 3GP chunk hierarchy:

  - either they already in a separate chunk that is non-causal (comes at the end of the stream), and I'm doomed.

  - either (as you suggest) they are in the "AVC" bitstream present in the linearly-growing main chunk (mdat), and what 
I need to do is write an "external bitstream filter" inspired from mp4toannexb, because there is no -f value to tell 
ffmpeg to read stdin as "raw AVC".

Do you at least agree with the dichotomy ?

-Alex