[FFmpeg-devel] [PATCH] Add Apple HTTP Live Streaming protocol handler

[Martin Storsjö]

On Tue, 27 Jul 2010, Martin Storsj? wrote:

> On Mon, 26 Jul 2010, Ronald S. Bultje wrote:
> 
> > On Mon, Jul 26, 2010 at 1:36 PM, Martin Storsj? <martin at martin.st> wrote:
> > > On Mon, 26 Jul 2010, Ronald S. Bultje wrote:
> > >>
> > >> I totally agree. Both demuxers and protocols should stay off of this
> > >> stuff. It's the application's job to do this. So we need to expose
> > >> that this is possible to the application. Realmedia/RDT (or
> > >> .rm/.avi/.matroska files that have multiple tracks, or DVDs with
> > >> multiple soundtracks) show what API we use for that. I'd like you to
> > >> use this API also, even if only for consistency.
> > >
> > > Ok, I can give it a try. Doing it that way makes some things prettier, but
> > > some other things get a bit uglier.
> > >
> > > In this setup, we'd need to convey info about which streams belong to
> > > which bandwidth variant, since (especially in low bandwidth setups), you
> > > wouldn't want to pull video from one stream and audio from another one,
> > > forcing you to receive both variants.
> > 
> > "Coupling", I guess. Yes, it makes sense to add some mechanism. *If*
> > the application is silly enough to request that, it should work, but
> > it makes sense to not make it the default, so a coupling ID of some
> > sort would make sense to me.
> 
> Here's the initial attempt. While this has the upside that it gives you 
> much more flexibility, the code is also vastly more complicated than the 
> earlier attempt and a bit longer (>500 lines compared to 300).
> 
> Another downside is that it is much harder to implement automatic quality 
> switching on the libavformat level (if that ever was desired) in this 
> setup - but of course allows much better manual control.
> 
> I expose the variant number that each stream belongs to in AVStream->id - 
> I hope that is ok, it's quite useful for mapping the streams to the 
> internal variants internally, too.
> 
> Unfortunately enough, ffplay picks the stream to use using the 
> AVStream->codec_info_nb_frames value, which makes it more often than not 
> pick audio and video streams from different variants. If starting with 
> -vst 0 -ast 0, it uses the first of each, which should belong to the same 
> variant, and if switching by pressing 'a' and 'v', pressing both at the 
> same time to keep taking both variants from the same streams is 
> recommended.
> 
> Also, with the sample stream at 
> http://devimages.apple.com/iphone/samples/bipbop/bipbopall.m3u8 at least, 
> ffplay seems to be buffering way too much data, so when switching streams, 
> we jump very far ahead. For testing with this stream, it can be fixed with 
> a modification like this:
> 
> --- a/ffplay.c
> +++ b/ffplay.c
> @@ -2546,7 +2546,7 @@ static int decode_thread(void *arg)
>  
>          /* if the queue are full, no need to read more */
>          if (   is->audioq.size + is->videoq.size + is->subtitleq.size > MAX_QUEUE_SIZE
> -            || (   (is->audioq   .size  > MIN_AUDIOQ_SIZE || is->audio_stream<0)
> +            || (   (is->audioq   .nb_packets > MIN_FRAMES || is->audio_stream<0)
>                  && (is->videoq   .nb_packets > MIN_FRAMES || is->video_stream<0)
>                  && (is->subtitleq.nb_packets > MIN_FRAMES || is->subtitle_stream<0))) {
>              /* wait 10 ms */
> 
> Note, I'm not really proposing this ffplay patch in general, but I had to 
> do this modification for stream switching to work well.

Ping - Ronald, any comments on this?

// Martin