[FFmpeg-trac] #8465(undetermined:new): avformat_find_stream_info does not fill MPEG2-TS/HEVC resolution with HEVC decoder disabled

[FFmpeg]

#8465: avformat_find_stream_info does not fill MPEG2-TS/HEVC resolution with HEVC
decoder disabled
-------------------------------------+-------------------------------------
             Reporter:  ddyndo       |                     Type:  defect
               Status:  new          |                 Priority:  normal
            Component:               |                  Version:  git-
  undetermined                       |  master
             Keywords:  hevc         |               Blocked By:
  mpeg2ts mpegts metadata            |
             Blocking:               |  Reproduced by developer:  0
Analyzed by developer:  0            |
-------------------------------------+-------------------------------------
 Summary of the bug:

 I've found that if FFmpeg is built with HEVC decoder disabled (or does not
 have "hevc" decoder on whitelist in CLI), it will fail to acquire basic
 metadata (e.g. resolution, color characteristics, etc.) even though
 internal parsers are enabled and do parse NAL units like SPS or VUI from
 it (and during the call too).

 It seems that in most cases it works because at the end of this logic if
 FFmpeg detects that the parameters are not set, it tries to decode the
 first sample which then correctly sets resolution and other stream
 metadata.


 How to reproduce:

 Either:
 1. Build FFmpeg without HEVC decoder
 2. Use FFmpeg API (even example code is fine) -
 avformat_find_stream_info() function on opened MPEG2-TS/HEVC file
 3. Watch errors logged and check out that video stream in AVFormatContext
 does not have basic metadata like resolution, color space, etc. filled
 correctly.
 4. If you enable logs on very high level, you will actually see that it
 does parse SPS NAL unit correctly.

 or:
 {{{
 % ffprobe -v 50 -codec_whitelist h265 h1.ts
 %
 % ...
 % [NULL @ 0x3ed7a528ca00] Could not find codec parameters for stream 0
 (Video: hevc, 1 reference frame (HEVC / 0x43564548), none): unspecified
 size
 % ...
 }}}

 I've used h1.ts file from #3487 (http://trac.ffmpeg.org/ticket/3487) and
 it reproduces just as well.

 ------------------

 Some analysis

 I've found out that SPS NAL unit (and others) are parsed on such stack
 trace:

 - `ff_hevc_decode_nal_sps()`
 - `hevc_parse()`
 - `av_parser_parse2()`
 - `parse_packet()`
 - `read_frame_internal()`
 - `avformat_find_stream_info()`

 The problem is that this information parsed there is not propagated higher
 (outside of the parser (AVCodecParserContext) to the codec
 (AVCodecContext)). This can be seen in `av_parser_parse2()` where there is
 code like:

 {{{
 %    if (avctx->codec_type == AVMEDIA_TYPE_VIDEO) {
 %        FILL(field_order);
 %    }
 }}}

 This is one place in code I've found that does not depend on specific
 parser/internal data but can rewrite metadata from the parser to
 AVCodecContext. Adding there something like:

 {{{
 %    FILL(width);
 %    FILL(height);
 }}}

 correctly propagates video resolution further. Unfortunately, this seems
 to be only part of missing metadata because  format/pix_fmt is not
 correctly set/propagated either, and neither are other fields there like
 AVCodecContext's colorspace, color_trc or color_primaries members.

 I wasn't able to find a good place to do this in a clean way. For HEVC I
 see that such metadata is accessible and partially set in `hevc_parser.c`
 file in `hevc_parse_slice_header()` function which looks currently like:

 {{{
 %    s->coded_width  = ps->sps->width;
 %    s->coded_height = ps->sps->height;
 %    s->width        = ps->sps->width  - ow->left_offset -
 ow->right_offset;
 %    s->height       = ps->sps->height - ow->top_offset  -
 ow->bottom_offset;
 %    s->format       = ps->sps->pix_fmt;
 %    avctx->profile  = ps->sps->ptl.general_ptl.profile_idc;
 %    avctx->level    = ps->sps->ptl.general_ptl.level_idc;
 }}}

 while it sets the parser's metadata, it is not propagated forward (only
 profile and level here seems to be propagated forward).

 ----

 In the end, I think that this can affect negatively performance (fallback
 to decoding just to get metadata from the stream) and actually makes it
 impossible to obtain stream information (metadata such as resolution,
 color space, etc.) through parsers and `avformat_find_stream_info()`
 function and as such should have high priority for fixing it.

--
Ticket URL: <https://trac.ffmpeg.org/ticket/8465>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker