[FFmpeg-trac] #10772(avformat:new): codec_whitelist does not influence codec picking
FFmpeg
trac at avcodec.org
Tue Jan 2 14:30:50 EET 2024
#10772: codec_whitelist does not influence codec picking
-------------------------------------+-------------------------------------
Reporter: Rudolf | Owner: (none)
Polzer |
Type: defect | Status: new
Priority: important | Component: avformat
Version: git-master | Resolution:
Keywords: | Blocked By:
Blocking: | Reproduced by developer: 0
Analyzed by developer: 0 |
-------------------------------------+-------------------------------------
Description changed by Rudolf Polzer:
Old description:
> Summary of the bug:
>
> The codec_whitelist feature, as introduced in
> 5c8eb16769e581f828ce420373c558c190185cc1, appears to block codecs from
> decoding, but it does not prevent automatic codec selection from picking
> a blocked codec.
>
> Thus, when the feature is used and one wants to play/decode a file whose
> preferred codec is not on the whitelist, but a different codec is on the
> whitelist that can decode the file, decoding will fail unless explicitly
> selecting the decoder.
>
> This currently breaks Opus decoding (and thus 100% of youtube.com) in
> Chromium on Alpine Linux (and probably any other system that compiles
> Chromium to link against a system-wide installed ffmpeg library, and
> configures that system-wide library to contain both the opus and libopus
> decoders): https://gitlab.alpinelinux.org/alpine/aports/-/issues/15585
>
> Also, as the below log shows, even when explicitly setting the codec,
> there still appear to be issues during analysis of the file; these do not
> appear to break playback though.
>
> Component set to avformat as I believe `find_probe_decoder` in
> `libavformat/demux.c` should take the whitelist into account and skip
> codecs not listed on it.
>
> Priority set to important, as Chromium being able to play YouTube videos
> is pretty important.
>
> Not sure if Type should really be defect or enhancement - it depends on
> what the intent of the codec whitelist feature was (and thus whether
> Chrome is abusing it). The documentation of the feature does not go into
> much detail.
>
> How to reproduce:
>
> (NOTE: this repro requires _any_ opus file, and a ffmpeg binary that has
> both the opus and libopus decoders enabled in it. For my tests I was
> using http://rm.cloudns.org/~xonotic/temp/b-opus.mp4)
>
> {{{
> % ffplay -codec_whitelist h264,libopus /tmp/b-opus.mp4
> ffplay version N-113171-g85b8d59ec7 Copyright (c) 2003-2024 the FFmpeg
> developers
> built with gcc 13 (Debian 13.2.0-5)
> configuration: --prefix=/home/rpolzer/src/update-ffmpeg/out --cc=gcc
> --cxx=g++ --objcc= --dep-cc=gcc --disable-static --enable-shared
> --enable-nonfree --enable-gpl --enable-version3 --enable-libaom --enable-
> libvpx --enable-libwebp --enable-libx264 --enable-libfribidi --enable-
> libass --enable-libfontconfig --enable-libfreetype --enable-libmp3lame
> --enable-libvorbis --enable-libopus --enable-libpulse --enable-vaapi
> --enable-vdpau --enable-demuxer=dash --enable-gnutls --enable-libxml2
> libavutil 58. 36.100 / 58. 36.100
> libavcodec 60. 36.100 / 60. 36.100
> libavformat 60. 20.100 / 60. 20.100
> libavdevice 60. 4.100 / 60. 4.100
> libavfilter 9. 16.100 / 9. 16.100
> libswscale 7. 6.100 / 7. 6.100
> libswresample 4. 13.100 / 4. 13.100
> libpostproc 57. 4.100 / 57. 4.100
> [opus @ 0x7ff164007a80] Codec (opus) not on whitelist 'h264,libopus'
> [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7ff164000c80] Failed to open codec in
> avformat_find_stream_info
> [opus @ 0x7ff164007a80] Codec (opus) not on whitelist 'h264,libopus'
> [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7ff164000c80] Could not find codec
> parameters for stream 1 (Audio: opus (Opus / 0x7375704F), 48000 Hz, 0
> channels, 104 kb/s): unspecified number of channels
> Consider increasing the value for the 'analyzeduration' (0) and
> 'probesize' (5000000) options
> Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/tmp/b-opus.mp4':
> Metadata:
> major_brand : isom
> minor_version : 512
> compatible_brands: isomiso2avc1mp41
> encoder : Lavf60.16.100
> Duration: 00:00:25.48, start: 0.000000, bitrate: 884 kb/s
> Stream #0:0[0x1](und): Video: h264 (High) (avc1 / 0x31637661),
> yuv420p(tv, bt470bg/unknown/unknown, progressive), 640x360, 768 kb/s, 60
> fps, 60 tbr, 15360 tbn (default)
> Metadata:
> handler_name : VideoHandler
> vendor_id : [0][0][0][0]
> encoder : Lavc60.31.102 libx264
> Stream #0:1[0x2](und): Audio: opus (Opus / 0x7375704F), 48000 Hz, 2
> channels, 104 kb/s (default)
> Metadata:
> handler_name : SoundHandler
> vendor_id : [0][0][0][0]
> [opus @ 0x7ff164022d80] Codec (opus) not on whitelist 'h264,libopus'
> Failed to open file '/tmp/b-opus.mp4' or configure filtergraph
>
> % ffplay -codec_whitelist h264,libopus -vn -codec:a libopus
> /tmp/b-opus.mp4
> ffplay version N-113171-g85b8d59ec7 Copyright (c) 2003-2024 the FFmpeg
> developers
> built with gcc 13 (Debian 13.2.0-5)
> configuration: --prefix=/home/rpolzer/src/update-ffmpeg/out --cc=gcc
> --cxx=g++ --objcc= --dep-cc=gcc --disable-static --enable-shared
> --enable-nonfree --enable-gpl --enable-version3 --enable-libaom --enable-
> libvpx --enable-libwebp --enable-libx264 --enable-libfribidi --enable-
> libass --enable-libfontconfig --enable-libfreetype --enable-libmp3lame
> --enable-libvorbis --enable-libopus --enable-libpulse --enable-vaapi
> --enable-vdpau --enable-demuxer=dash --enable-gnutls --enable-libxml2
> libavutil 58. 36.100 / 58. 36.100
> libavcodec 60. 36.100 / 60. 36.100
> libavformat 60. 20.100 / 60. 20.100
> libavdevice 60. 4.100 / 60. 4.100
> libavfilter 9. 16.100 / 9. 16.100
> libswscale 7. 6.100 / 7. 6.100
> libswresample 4. 13.100 / 4. 13.100
> libpostproc 57. 4.100 / 57. 4.100
> [opus @ 0x7fd82c011d40] Codec (opus) not on whitelist 'h264,libopus'
> [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7fd82c000c80] Failed to open codec in
> avformat_find_stream_info
> [opus @ 0x7fd82c011d40] Codec (opus) not on whitelist 'h264,libopus'
> [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7fd82c000c80] Could not find codec
> parameters for stream 1 (Audio: opus (Opus / 0x7375704F), 48000 Hz, 0
> channels, 104 kb/s): unspecified number of channels
> Consider increasing the value for the 'analyzeduration' (0) and
> 'probesize' (5000000) options
> Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/tmp/b-opus.mp4':
> Metadata:
> major_brand : isom
> minor_version : 512
> compatible_brands: isomiso2avc1mp41
> encoder : Lavf60.16.100
> Duration: 00:00:25.48, start: 0.000000, bitrate: 884 kb/s
> Stream #0:0[0x1](und): Video: h264 (High) (avc1 / 0x31637661),
> yuv420p(tv, bt470bg/unknown/unknown, progressive), 640x360, 768 kb/s, 60
> fps, 60 tbr, 15360 tbn (default)
> Metadata:
> handler_name : VideoHandler
> vendor_id : [0][0][0][0]
> encoder : Lavc60.31.102 libx264
> Stream #0:1[0x2](und): Audio: opus (Opus / 0x7375704F), 48000 Hz, 2
> channels, 104 kb/s (default)
> Metadata:
> handler_name : SoundHandler
> vendor_id : [0][0][0][0]
> 0.58 M-A: 0.000 fd= 0 aq= 10KB vq= 0KB sq= 0B f=0/0
> }}}
New description:
Summary of the bug:
The codec_whitelist feature, as introduced in
5c8eb16769e581f828ce420373c558c190185cc1, appears to block codecs from
decoding, but it does not prevent automatic codec selection from picking a
blocked codec (and thus it does not make ffmpeg behave as if only the
listed codecs were compiled in, which I presume was the intent - it is at
least what the Chromium developers believe the intent was in
https://source.chromium.org/chromium/chromium/src/+/02508a7461cde79277a7e5e55f1704d3fb6a5f7a).
Thus, when the feature is used and one wants to play/decode a file whose
preferred codec is not on the whitelist, but a different codec is on the
whitelist that can decode the file, decoding will fail unless explicitly
selecting the decoder.
This currently breaks Opus decoding (and thus 100% of youtube.com) in
Chromium on Alpine Linux (and probably any other system that compiles
Chromium to link against a system-wide installed ffmpeg library, and
configures that system-wide library to contain both the opus and libopus
decoders): https://gitlab.alpinelinux.org/alpine/aports/-/issues/15585
Also, as the below log shows, even when explicitly setting the codec,
there still appear to be issues during analysis of the file; these do not
appear to break playback though.
Component set to avformat as I believe `find_probe_decoder` in
`libavformat/demux.c` should take the whitelist into account and skip
codecs not listed on it.
Priority set to important, as Chromium being able to play YouTube videos
is pretty important.
Not sure if Type should really be defect or enhancement - it depends on
what the intent of the codec whitelist feature was (and thus whether
Chrome is abusing it). The documentation of the feature does not go into
much detail.
How to reproduce:
(NOTE: this repro requires _any_ opus file, and a ffmpeg binary that has
both the opus and libopus decoders enabled in it. For my tests I was using
http://rm.cloudns.org/~xonotic/temp/b-opus.mp4)
{{{
% ffplay -codec_whitelist h264,libopus /tmp/b-opus.mp4
ffplay version N-113171-g85b8d59ec7 Copyright (c) 2003-2024 the FFmpeg
developers
built with gcc 13 (Debian 13.2.0-5)
configuration: --prefix=/home/rpolzer/src/update-ffmpeg/out --cc=gcc
--cxx=g++ --objcc= --dep-cc=gcc --disable-static --enable-shared --enable-
nonfree --enable-gpl --enable-version3 --enable-libaom --enable-libvpx
--enable-libwebp --enable-libx264 --enable-libfribidi --enable-libass
--enable-libfontconfig --enable-libfreetype --enable-libmp3lame --enable-
libvorbis --enable-libopus --enable-libpulse --enable-vaapi --enable-vdpau
--enable-demuxer=dash --enable-gnutls --enable-libxml2
libavutil 58. 36.100 / 58. 36.100
libavcodec 60. 36.100 / 60. 36.100
libavformat 60. 20.100 / 60. 20.100
libavdevice 60. 4.100 / 60. 4.100
libavfilter 9. 16.100 / 9. 16.100
libswscale 7. 6.100 / 7. 6.100
libswresample 4. 13.100 / 4. 13.100
libpostproc 57. 4.100 / 57. 4.100
[opus @ 0x7ff164007a80] Codec (opus) not on whitelist 'h264,libopus'
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x7ff164000c80] Failed to open codec in
avformat_find_stream_info
[opus @ 0x7ff164007a80] Codec (opus) not on whitelist 'h264,libopus'
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x7ff164000c80] Could not find codec parameters
for stream 1 (Audio: opus (Opus / 0x7375704F), 48000 Hz, 0 channels, 104
kb/s): unspecified number of channels
Consider increasing the value for the 'analyzeduration' (0) and
'probesize' (5000000) options
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/tmp/b-opus.mp4':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
encoder : Lavf60.16.100
Duration: 00:00:25.48, start: 0.000000, bitrate: 884 kb/s
Stream #0:0[0x1](und): Video: h264 (High) (avc1 / 0x31637661),
yuv420p(tv, bt470bg/unknown/unknown, progressive), 640x360, 768 kb/s, 60
fps, 60 tbr, 15360 tbn (default)
Metadata:
handler_name : VideoHandler
vendor_id : [0][0][0][0]
encoder : Lavc60.31.102 libx264
Stream #0:1[0x2](und): Audio: opus (Opus / 0x7375704F), 48000 Hz, 2
channels, 104 kb/s (default)
Metadata:
handler_name : SoundHandler
vendor_id : [0][0][0][0]
[opus @ 0x7ff164022d80] Codec (opus) not on whitelist 'h264,libopus'
Failed to open file '/tmp/b-opus.mp4' or configure filtergraph
% ffplay -codec_whitelist h264,libopus -vn -codec:a libopus
/tmp/b-opus.mp4
ffplay version N-113171-g85b8d59ec7 Copyright (c) 2003-2024 the FFmpeg
developers
built with gcc 13 (Debian 13.2.0-5)
configuration: --prefix=/home/rpolzer/src/update-ffmpeg/out --cc=gcc
--cxx=g++ --objcc= --dep-cc=gcc --disable-static --enable-shared --enable-
nonfree --enable-gpl --enable-version3 --enable-libaom --enable-libvpx
--enable-libwebp --enable-libx264 --enable-libfribidi --enable-libass
--enable-libfontconfig --enable-libfreetype --enable-libmp3lame --enable-
libvorbis --enable-libopus --enable-libpulse --enable-vaapi --enable-vdpau
--enable-demuxer=dash --enable-gnutls --enable-libxml2
libavutil 58. 36.100 / 58. 36.100
libavcodec 60. 36.100 / 60. 36.100
libavformat 60. 20.100 / 60. 20.100
libavdevice 60. 4.100 / 60. 4.100
libavfilter 9. 16.100 / 9. 16.100
libswscale 7. 6.100 / 7. 6.100
libswresample 4. 13.100 / 4. 13.100
libpostproc 57. 4.100 / 57. 4.100
[opus @ 0x7fd82c011d40] Codec (opus) not on whitelist 'h264,libopus'
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x7fd82c000c80] Failed to open codec in
avformat_find_stream_info
[opus @ 0x7fd82c011d40] Codec (opus) not on whitelist 'h264,libopus'
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x7fd82c000c80] Could not find codec parameters
for stream 1 (Audio: opus (Opus / 0x7375704F), 48000 Hz, 0 channels, 104
kb/s): unspecified number of channels
Consider increasing the value for the 'analyzeduration' (0) and
'probesize' (5000000) options
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/tmp/b-opus.mp4':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
encoder : Lavf60.16.100
Duration: 00:00:25.48, start: 0.000000, bitrate: 884 kb/s
Stream #0:0[0x1](und): Video: h264 (High) (avc1 / 0x31637661),
yuv420p(tv, bt470bg/unknown/unknown, progressive), 640x360, 768 kb/s, 60
fps, 60 tbr, 15360 tbn (default)
Metadata:
handler_name : VideoHandler
vendor_id : [0][0][0][0]
encoder : Lavc60.31.102 libx264
Stream #0:1[0x2](und): Audio: opus (Opus / 0x7375704F), 48000 Hz, 2
channels, 104 kb/s (default)
Metadata:
handler_name : SoundHandler
vendor_id : [0][0][0][0]
0.58 M-A: 0.000 fd= 0 aq= 10KB vq= 0KB sq= 0B f=0/0
}}}
--
--
Ticket URL: <https://trac.ffmpeg.org/ticket/10772#comment:1>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker
More information about the FFmpeg-trac
mailing list