Request support for decoding / demuxing Adobe HDS dynamic http streaming

[ottomatic]

Adobe HDS is an adaptive streaming format used primarily to deliver video streams through content delivery networks.

It uses a manifest file (F4F) to describe the segments of a file, and then it adaptively delivers segments and sequences in the "optimal" bitrate depending on the client's bandwith and the total server load.

Certain major content providers are moving to this format (at least in Sweden) and it would be really great if ffmpeg could support it.

A good summary of HDS is found at:
 http://rdkls.blogspot.se/2011/11/what-i-know-about-http-adaptive.html

It seems that most projects that do decode such streams use a PHP script (!) found at:
 https://github.com/K-S-V/Scripts/blob/master/AdobeHDS.php

The Open Source Media Player project has action script code which probably does the same thing:
 http://sourceforge.net/adobe/osmf/

The format is quite similar to applehttp/hls, so it should be possible to borrow some patterns from the support for that format, which is already in libavformat.

I am a C# / java developer and could probably make this happen in C, but it would take a lot of effort. I am not yet familiar with the ffmpeg source code. So I'm hoping that someone might already be working on this? Or that someone well versed on avformat development can take this on. I'd be happy to contribute with my own efforts.

Best regards!

[cehoyos]

Does FFmpeg support playing the segments / the first transmitted segment?

[ottomatic]

No.

Here is the output from when I attempt to play a stream, using the recent static build for 64 bit windows:

>ffmpeg.exe -i "http://svtplay6s-f.akamaihd.net/z/se
/secure/20121118/1232796-008A/EARTHLY_TREASUR-008A-7a4e5968ffc1c170_,900,320,420
,620,1660,2760,.mp4.csmil/manifest.f4m?hdcore=2.8.0&g=LLIOFRVAXRWA" D:\tmp\fmt-t
est.ts
ffmpeg version N-46469-gc995644 Copyright (c) 2000-2012 the FFmpeg developers
  built on Nov  5 2012 17:58:15 with gcc 4.7.2 (GCC)
  configuration: --enable-gpl --enable-version3 --disable-pthreads --enable-runt
ime-cpudetect --enable-avisynth --enable-bzlib --enable-frei0r --enable-libass -
-enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libfreetype --enab
le-libgsm --enable-libmp3lame --enable-libnut --enable-libopenjpeg --enable-libo
pus --enable-librtmp --enable-libschroedinger --enable-libspeex --enable-libtheo
ra --enable-libutvideo --enable-libvo-aacenc --enable-libvo-amrwbenc --enable-li
bvorbis --enable-libvpx --enable-libx264 --enable-libxavs --enable-libxvid --ena
ble-zlib
  libavutil      52.  5.100 / 52.  5.100
  libavcodec     54. 71.100 / 54. 71.100
  libavformat    54. 36.100 / 54. 36.100
  libavdevice    54.  3.100 / 54.  3.100
  libavfilter     3. 21.106 /  3. 21.106
  libswscale      2.  1.102 /  2.  1.102
  libswresample   0. 16.100 /  0. 16.100
  libpostproc    52.  1.100 / 52.  1.100
http://svtplay6s-f.akamaihd.net/z/se/secure/20121118/1232796-008A/EARTHLY_TREASU
R-008A-7a4e5968ffc1c170_,900,320,420,620,1660,2760,.mp4.csmil/manifest.f4m?hdcor
e=2.8.0&g=LLIOFRVAXRWA: Invalid data found when processing input

If I inspect the contents of that URL, I get the F4M manifest:

<?xml version="1.0" encoding="UTF-8"?>
<manifest xmlns="http://ns.adobe.com/f4m/1.0" xmlns:akamai="uri:akamai.com/f4m/1.0">
  <akamai:version>2.0</akamai:version>
  <akamai:bw>5000</akamai:bw>
  <id>/se/secure/20121118/1232796-008A/EARTHLY_TREASUR-008A-7a4e5968ffc1c170_,900,320,420,620,1660,2760,.mp4.csmil_0</id>
  <streamType>recorded</streamType>
  <akamai:streamType>vod</akamai:streamType>
  <duration>300.501</duration>
  <streamBaseTime>0.000</streamBaseTime>
  <bootstrapInfo profile="named" id="bootstrap_1">AAAAi2Fic3QAAAAAAAAAAQAAAAPoAAAAAAAEldUAAAAAAAAAAAAAAAAAAQAAABlhc3J0AAAAAAAAAAABAAAAAQAAADIBAAAARmFmcnQAAAAAAAAD6AAAAAADAAAAAQAAAAAAAAAAAAAXcAAAADIAAAAAAAR8cAAAGWUAAAAAAAAAAAAAAAAAAAAAAA==</bootstrapInfo>
  <bootstrapInfo profile="named" id="bootstrap_2">AAAAi2Fic3QAAAAAAAAAAQAAAAPoAAAAAAAEldUAAAAAAAAAAAAAAAAAAQAAABlhc3J0AAAAAAAAAAABAAAAAQAAADIBAAAARmFmcnQAAAAAAAAD6AAAAAADAAAAAQAAAAAAAAAAAAAXcAAAADIAAAAAAAR8cAAAGWUAAAAAAAAAAAAAAAAAAAAAAA==</bootstrapInfo>
  <bootstrapInfo profile="named" id="bootstrap_3">AAAAi2Fic3QAAAAAAAAAAQAAAAPoAAAAAAAEldUAAAAAAAAAAAAAAAAAAQAAABlhc3J0AAAAAAAAAAABAAAAAQAAADIBAAAARmFmcnQAAAAAAAAD6AAAAAADAAAAAQAAAAAAAAAAAAAXcAAAADIAAAAAAAR8cAAAGWUAAAAAAAAAAAAAAAAAAAAAAA==</bootstrapInfo>
  <bootstrapInfo profile="named" id="bootstrap_0">AAAAi2Fic3QAAAAAAAAAAQAAAAPoAAAAAAAEldUAAAAAAAAAAAAAAAAAAQAAABlhc3J0AAAAAAAAAAABAAAAAQAAADIBAAAARmFmcnQAAAAAAAAD6AAAAAADAAAAAQAAAAAAAAAAAAAXcAAAADIAAAAAAAR8cAAAGWUAAAAAAAAAAAAAAAAAAAAAAA==</bootstrapInfo>
  <bootstrapInfo profile="named" id="bootstrap_4">AAAAi2Fic3QAAAAAAAAAAQAAAAPoAAAAAAAEldUAAAAAAAAAAAAAAAAAAQAAABlhc3J0AAAAAAAAAAABAAAAAQAAADIBAAAARmFmcnQAAAAAAAAD6AAAAAADAAAAAQAAAAAAAAAAAAAXcAAAADIAAAAAAAR8cAAAGWUAAAAAAAAAAAAAAAAAAAAAAA==</bootstrapInfo>
  <bootstrapInfo profile="named" id="bootstrap_5">AAAAi2Fic3QAAAAAAAAAAQAAAAPoAAAAAAAElcAAAAAAAAAAAAAAAAAAAQAAABlhc3J0AAAAAAAAAAABAAAAAQAAADIBAAAARmFmcnQAAAAAAAAD6AAAAAADAAAAAQAAAAAAAAAAAAAXcAAAADIAAAAAAAR8cAAAGVAAAAAAAAAAAAAAAAAAAAAAAA==</bootstrapInfo>
  <media bitrate="315" url="1_ee8b5f6c472833a3_" bootstrapInfoId="bootstrap_1">
    <metadata>AgAKb25NZXRhRGF0YQgAAAAMAAhkdXJhdGlvbgBAcsgEGJN0vAAFd2lkdGgAQHQAAAAAAAAABmhlaWdodABAZoAAAAAAAAANdmlkZW9kYXRhcmF0ZQBAY/imU7h9mQAJZnJhbWVyYXRlAEA4/42A0ljSAAx2aWRlb2NvZGVjaWQAQBwAAAAAAAAADWF1ZGlvZGF0YXJhdGUAQGOUr7/JUGwAD2F1ZGlvc2FtcGxlcmF0ZQBA53AAAAAAAAAPYXVkaW9zYW1wbGVzaXplAEAwAAAAAAAAAAZzdGVyZW8BAQAMYXVkaW9jb2RlY2lkAEAkAAAAAAAAAAhmaWxlc2l6ZQBBZqtwgAAAAAAACQ==</metadata>
  </media>
  <media bitrate="415" url="2_ee8b5f6c472833a3_" bootstrapInfoId="bootstrap_2">
    <metadata>AgAKb25NZXRhRGF0YQgAAAAMAAhkdXJhdGlvbgBAcsgEGJN0vAAFd2lkdGgAQH4AAAAAAAAABmhlaWdodABAcOAAAAAAAAANdmlkZW9kYXRhcmF0ZQBAcDrhxmp71QAJZnJhbWVyYXRlAEA4/42A0ljSAAx2aWRlb2NvZGVjaWQAQBwAAAAAAAAADWF1ZGlvZGF0YXJhdGUAQGOUr7/JUGwAD2F1ZGlvc2FtcGxlcmF0ZQBA53AAAAAAAAAPYXVkaW9zYW1wbGVzaXplAEAwAAAAAAAAAAZzdGVyZW8BAQAMYXVkaW9jb2RlY2lkAEAkAAAAAAAAAAhmaWxlc2l6ZQBBbdPl4AAAAAAACQ==</metadata>
  </media>
  <media bitrate="615" url="3_ee8b5f6c472833a3_" bootstrapInfoId="bootstrap_3">
    <metadata>AgAKb25NZXRhRGF0YQgAAAAMAAhkdXJhdGlvbgBAcsgEGJN0vAAFd2lkdGgAQIIAAAAAAAAABmhlaWdodABAdEAAAAAAAAANdmlkZW9kYXRhcmF0ZQBAfLoOxhoCQwAJZnJhbWVyYXRlAEA4/42A0ljSAAx2aWRlb2NvZGVjaWQAQBwAAAAAAAAADWF1ZGlvZGF0YXJhdGUAQGOUr7/JUGwAD2F1ZGlvc2FtcGxlcmF0ZQBA53AAAAAAAAAPYXVkaW9zYW1wbGVzaXplAEAwAAAAAAAAAAZzdGVyZW8BAQAMYXVkaW9jb2RlY2lkAEAkAAAAAAAAAAhmaWxlc2l6ZQBBdhOW0AAAAAAACQ==</metadata>
  </media>
  <media bitrate="955" url="0_ee8b5f6c472833a3_" bootstrapInfoId="bootstrap_0">
    <metadata>AgAKb25NZXRhRGF0YQgAAAAMAAhkdXJhdGlvbgBAcsgEGJN0vAAFd2lkdGgAQIYAAAAAAAAABmhlaWdodABAeMAAAAAAAAANdmlkZW9kYXRhcmF0ZQBAiPy3if5LYwAJZnJhbWVyYXRlAEA4/42A0ljSAAx2aWRlb2NvZGVjaWQAQBwAAAAAAAAADWF1ZGlvZGF0YXJhdGUAQGOUr7/JUGwAD2F1ZGlvc2FtcGxlcmF0ZQBA53AAAAAAAAAPYXVkaW9zYW1wbGVzaXplAEAwAAAAAAAAAAZzdGVyZW8BAQAMYXVkaW9jb2RlY2lkAEAkAAAAAAAAAAhmaWxlc2l6ZQBBgSCciAAAAAAACQ==</metadata>
  </media>
  <media bitrate="1655" url="4_ee8b5f6c472833a3_" bootstrapInfoId="bootstrap_4">
    <metadata>AgAKb25NZXRhRGF0YQgAAAAMAAhkdXJhdGlvbgBAcsgEGJN0vAAFd2lkdGgAQJAAAAAAAAAABmhlaWdodABAggAAAAAAAAANdmlkZW9kYXRhcmF0ZQBAl27+UGbQvgAJZnJhbWVyYXRlAEA4/42A0ljSAAx2aWRlb2NvZGVjaWQAQBwAAAAAAAAADWF1ZGlvZGF0YXJhdGUAQGOUr7/JUGwAD2F1ZGlvc2FtcGxlcmF0ZQBA53AAAAAAAAAPYXVkaW9zYW1wbGVzaXplAEAwAAAAAAAAAAZzdGVyZW8BAQAMYXVkaW9jb2RlY2lkAEAkAAAAAAAAAAhmaWxlc2l6ZQBBjasJSAAAAAAACQ==</metadata>
  </media>
  <media bitrate="2755" url="5_ee8b5f6c472833a3_" bootstrapInfoId="bootstrap_5">
    <metadata>AgAKb25NZXRhRGF0YQgAAAAMAAhkdXJhdGlvbgBAcseuFHrhSAAFd2lkdGgAQJQAAAAAAAAABmhlaWdodABAhoAAAAAAAAANdmlkZW9kYXRhcmF0ZQBApE4NtpueEgAJZnJhbWVyYXRlAEA5AAAAAAAAAAx2aWRlb2NvZGVjaWQAQBwAAAAAAAAADWF1ZGlvZGF0YXJhdGUAQGOUsjD25OcAD2F1ZGlvc2FtcGxlcmF0ZQBA53AAAAAAAAAPYXVkaW9zYW1wbGVzaXplAEAwAAAAAAAAAAZzdGVyZW8BAQAMYXVkaW9jb2RlY2lkAEAkAAAAAAAAAAhmaWxlc2l6ZQBBmK1T6AAAAAAACQ==</metadata>
  </media>
</manifest>

Unfortunately, I think the contents is protected by geographic restrictions, so you may only be able to test this if you are located in Sweden.

[ottomatic]

By the way, the "(F4F)" in the original ticket description should really read "(F4M)".

[ottomatic]

Replying to cehoyos:

Does FFmpeg support playing the segments / the first transmitted segment?

Oh, I think I misunderstood the question.

I'm not sure if ffmpeg supports playing the first segment / sequence, but I suppose it would. As far as I can understand, the HDS technology is just a way of splitting up the stream and transferring it adaptively in different chunks / bitrates. I think MP4 or flv is the standard container within that stream, but DRM is an optional feature according to the specs which I have ogled.

[ubitux]

A work in progress is available at  https://github.com/SmartJog/ffmpeg/compare/master...hds if someone is interested in continuing this work. I won't work on this anymore and thus this code is stalled. The code here basically just parses the XML bootstrap, the demuxing is left to be done.

[ottomatic]

Replying to ubitux:

A work in progress is available at  https://github.com/SmartJog/ffmpeg/compare/master...hds if someone is interested in continuing this work. I won't work on this anymore and thus this code is stalled. The code here basically just parses the XML bootstrap, the demuxing is left to be done.

Interesting.

Would you say that the manifest parsing is complete?

I notice your TODO:

// TODO: make links between bootstraps and medias

What do you mean by this?

[ubitux]

Replying to ottomatic:

Replying to ubitux:

A work in progress is available at  https://github.com/SmartJog/ffmpeg/compare/master...hds if someone is interested in continuing this work. I won't work on this anymore and thus this code is stalled. The code here basically just parses the XML bootstrap, the demuxing is left to be done.

Interesting.

Would you say that the manifest parsing is complete?

It should, mostly. The base64 data blob are not parsed, but they are decoded.

I notice your TODO:

// TODO: make links between bootstraps and medias

What do you mean by this?

It just means the bootstrap struct and media struct need to be linked (boostrap->media_id and media->stream_id).

[ottomatic]

Replying to ubitux:

Replying to ottomatic:

Replying to ubitux:

A work in progress is available at  https://github.com/SmartJog/ffmpeg/compare/master...hds if someone is interested in continuing this work. I won't work on this anymore and thus this code is stalled. The code here basically just parses the XML bootstrap, the demuxing is left to be done.

Interesting.

Would you say that the manifest parsing is complete?

It should, mostly. The base64 data blob are not parsed, but they are decoded.

I notice your TODO:

// TODO: make links between bootstraps and medias

What do you mean by this?

It just means the bootstrap struct and media struct need to be linked (boostrap->media_id and media->stream_id).

OK.

So, I'm being new to Git and Github. What is the correct way for me to go about if I want to continue this work and still preserve the awareness of the changes made from the original master code (as well as being able to merge updates from the master into this clone)? Do I just clone your repository, ubitux? And then if I manage to produce some proper code, make a pull request straight to the master?

[cehoyos]

Replying to ottomatic:

So, I'm being new to Git and Github. What is the correct way for me to go about if I want to continue this work and still preserve the awareness of the changes made from the original master code (as well as being able to merge updates from the master into this clone)? Do I just clone your repository, ubitux? And then if I manage to produce some proper code, make a pull request straight to the master?

If you clone it, all options are still available:
The changes could be merged in the future (that would keep the complete history), or the changes could still be sent as separate patches to ffmpeg-devel.

[ottomatic]

I attempted to download the stream at the URL stated in comment 2, using the adobeHDS.php script. That script could not parse the stream as it was encrypted using some sort of Akamai encryption. It would be preferable if that kind of decryption would be supported as well. I would look in the OSMF source for pointers.

[ottomatic]

Replying to ubitux:

A work in progress is available at  https://github.com/SmartJog/ffmpeg/compare/master...hds if someone is interested in continuing this work. I won't work on this anymore and thus this code is stalled. The code here basically just parses the XML bootstrap, the demuxing is left to be done.

Out of curiosity, why did you stall working on this, ubitux?

[ubitux]

Replying to ottomatic:

Replying to ubitux:

A work in progress is available at  https://github.com/SmartJog/ffmpeg/compare/master...hds if someone is interested in continuing this work. I won't work on this anymore and thus this code is stalled. The code here basically just parses the XML bootstrap, the demuxing is left to be done.

Out of curiosity, why did you stall working on this, ubitux?

Priorities changed at my company, that technology didn't look appealing, and I don't work for this company anymore so I lost interest.

[ottomatic]

Since there has been no further response to this, I have forked ubitux's repository and started working on the hds branch at:
 https://github.com/ottomatic/ffmpeg/tree/hds

I have reached a state where the entire bootstrap structure, with its segment run tables and fragment run tables, is properly parsed.

The progress is a bit slow, but it's progress all the same.

Next, I shall start working on the actual demuxing - if no one else would like to take over?

[cdunford]

Has this been abandoned or is work still ongoing? I'm very interested in this feature and would like to help in some way.

[ottomatic]

@cdunford: work us still ongoing. Progress is slow and erratic due to reasons given above, plus I work full time and have two kids to attend to. Any help would be appreciated. First off, properly freeing allocated memory would be good. I'm used to working with managed languages and soviet have postponed the whole cleanup bit. Are you used to working with github projects?