[FFmpeg-trac] #9979(undetermined:new): webvtt subtitles not properly detected in DASH manifest
FFmpeg
trac at avcodec.org
Wed Oct 19 00:36:26 EEST 2022
#9979: webvtt subtitles not properly detected in DASH manifest
-------------------------------------+-------------------------------------
Reporter: cookieguru | Type: defect
Status: new | Priority: normal
Component: | Version: git-
undetermined | master
Keywords: DASH dash | Blocked By:
dashdec mp webvtt |
Blocking: | Reproduced by developer: 0
Analyzed by developer: 0 |
-------------------------------------+-------------------------------------
Summary of the bug:
If an MPD manifest contains WebVTT subtitles FFmpeg will not properly
recognize or decode them.
How to reproduce:
Given a MPD with webvtt files, ffmpeg does not recognize it as a subtitle
stream and instead labels it as an unknown data stream:
`ffprobe -hide_banner "https://www.example.com/master.mpd"`
{{{
Input #0, dash, from 'https://www.example.com/master.mpd':
Duration: 00:42:00.00, start: 0.000000, bitrate: 0 kb/s
Program 0
Stream #0:0: Video: h264 (Main) (avc1 / 0x31637661), yuv420p, 416x234
[SAR 1:1 DAR 16:9], 162 kb/s, 29.97 fps, 29.97 tbr, 60k tbn (default)
Metadata:
variant_bitrate : 139000
id : video=139000
(snip: more video streams)
Stream #0:8(en): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo,
fltp, 126 kb/s (default)
Metadata:
variant_bitrate : 125000
id : audio_eng=125000
Stream #0:9(en): Data: none (wvtt / 0x74747677), 0 kb/s (default)
Metadata:
id : 1
Unsupported codec with id 0 for input stream 9
}}}
In this example, stream 9 contains the subtitles but is marked as Data.
Here is the contents of the MPD file:
{{{
<?xml version="1.0"?>
<MPD xmlns="urn:mpeg:dash:schema:mpd:2011" profiles="urn:mpeg:dash:profile
:isoff-live:2011,urn:com:dashif:dash264" type="static"
mediaPresentationDuration="PT42M0.818300S" minBufferTime="PT10S">
<Period id="1" duration="PT42M0.8183S">
<!-- audio and video AdaptationSet removed for brevity -->
<AdaptationSet mimeType="application/mp4" codecs="wvtt"
startWithSAP="1" segmentAlignment="true" lang="en" contentType="text">
<Role schemeIdUri="urn:mpeg:dash:role:2011"
value="subtitle"/>
<SegmentTemplate
initialization="caption_uuid_init.mp4" media="caption_uuid_$Time$.m4s"
startNumber="1" timescale="1000">
<SegmentTimeline>
<S t="0" d="30000" r="15"/>
<S d="27273"/>
<S d="30000" r="16"/>
<S d="21498"/>
<S d="30000" r="9"/>
<S d="7941"/>
<S d="30000" r="9"/>
<S d="967"/>
<S d="30000" r="17"/>
<S d="15688"/>
<S d="30000" r="9"/>
<S d="9733"/>
</SegmentTimeline>
</SegmentTemplate>
<Representation mimeType="application/mp4"
bandwidth="256" codecs="wvtt" id="1"/>
</AdaptationSet>
<EventStream schemeIdUri="urn:scte:scte35:2014:xml+bin"
timescale="30000">
<Event id="1" presentationTime="15218203"
duration="0"/>
<Event id="2" presentationTime="31163132"
duration="0"/>
<Event id="3" presentationTime="40401361"
duration="0"/>
<Event id="4" presentationTime="49430381"
duration="0"/>
<Event id="5" presentationTime="66101035"
duration="0"/>
</EventStream>
</Period>
</MPD>
}}}
Note `codecs="wvtt"` located twice in this `AdaptationSet`.
If we manually download the contents of this stream:
`ffmpeg -hide_banner -i "https://www.example.com/master.mpd" -map 0:d -c
copy -copy_unknown -f data raw.bin`
We can see that it contains the subs, but note it is not a valid webVTT
file (first byte is a NUL byte):
{{{
xxd raw.bin | head
00000000: 0000 005f 7674 7463 0000 0009 6964 656e ..._vttc....iden
00000010: 3800 0000 4e70 6179 6c3c 693e 5b6e 6172 8...Npayl<i>[nar
00000020: 7261 746f 725d 2041 6e64 2061 2066 616e rator] And a fan
00000030: 7461 7374 6963 616c 3c2f 693e 0a3c 693e tastical</i>.<i>
00000040: 776f 726c 6420 6869 6464 656e 2069 6e20 world hidden in
00000050: 7468 6520 6a75 6e67 6c65 2e3c 2f69 3e00 the jungle.</i>.
00000060: 0000 5f76 7474 6300 0000 0969 6465 6e38 .._vttc....iden8
00000070: 0000 004e 7061 796c 3c69 3e5b 6e61 7272 ...Npayl<i>[narr
00000080: 6174 6f72 5d20 416e 6420 6120 6661 6e74 ator] And a fant
00000090: 6173 7469 6361 6c3c 2f69 3e0a 3c69 3e77 astical</i>.<i>w
}}}
The URL of the stream in question is time-limited (it's a signed URL with
an expiration parameter) and thus is not included here. US-based users
can probably find similar MPD files on https://go.discovery.com/ but note
that the site uses both HLS and DASH streaming (newer videos tend to be
DASH; older videos never are).
ffmpeg version:
{{{
ffmpeg version N-108711-g3141dbb7ad-20221018 Copyright (c) 2000-2022 the
FFmpeg developers
built with gcc 12.1.0 (crosstool-NG 1.25.0.55_3defb7b)
configuration: --prefix=/ffbuild/prefix --pkg-config-flags=--static --pkg-
config=pkg-config --cross-prefix=x86_64-w64-mingw32- --arch=x86_64
--target-os=mingw32 --enable-gpl --enable-version3 --disable-debug
--disable-w32threads --enable-pthreads --enable-iconv --enable-libxml2
--enable-zlib --enable-libfreetype --enable-libfribidi --enable-gmp
--enable-lzma --enable-fontconfig --enable-libvorbis --enable-opencl
--disable-libpulse --enable-libvmaf --disable-libxcb --disable-xlib
--enable-amf --enable-libaom --enable-libaribb24 --enable-avisynth
--enable-libdav1d --enable-libdavs2 --disable-libfdk-aac --enable-
ffnvcodec --enable-cuda-llvm --enable-frei0r --enable-libgme --enable-
libkvazaar --enable-libass --enable-libbluray --enable-libjxl --enable-
libmp3lame --enable-libopus --enable-librist --enable-libssh --enable-
libtheora --enable-libvpx --enable-libwebp --enable-lv2 --enable-libmfx
--enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenh264
--enable-libopenjpeg --enable-libopenmpt --enable-librav1e --enable-
librubberband --enable-schannel --enable-sdl2 --enable-libsoxr --enable-
libsrt --enable-libsvtav1 --enable-libtwolame --enable-libuavs3d
--disable-libdrm --disable-vaapi --enable-libvidstab --enable-vulkan
--enable-libshaderc --enable-libplacebo --enable-libx264 --enable-libx265
--enable-libxavs2 --enable-libxvid --enable-libzimg --enable-libzvbi
--extra-cflags=-DLIBTWOLAME_STATIC --extra-cxxflags= --extra-
ldflags=-pthread --extra-ldexeflags= --extra-libs=-lgomp --extra-
version=20221018
libavutil 57. 39.101 / 57. 39.101
libavcodec 59. 51.100 / 59. 51.100
libavformat 59. 34.101 / 59. 34.101
libavdevice 59. 8.101 / 59. 8.101
libavfilter 8. 49.101 / 8. 49.101
libswscale 6. 8.112 / 6. 8.112
libswresample 4. 9.100 / 4. 9.100
libpostproc 56. 7.100 / 56. 7.100
}}}
--
Ticket URL: <https://trac.ffmpeg.org/ticket/9979>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker
More information about the FFmpeg-trac
mailing list