[FFmpeg-devel] [PATCH v2] movenc: Use first H264/HEVC frame as extradata, if it is missing

[Martin Storsjö]

On Thu, 21 May 2020, Michael Niedermayer wrote:

> On Wed, May 20, 2020 at 09:10:21AM +0300, Martin Storsjö wrote:
>> Sticking a full frame in the extradata works, as the code for writing
>> the avcC/hvcC extracts the relevant parameter set NAL units - provided
>> that they actually exist in the frame.
>>
>> Some encoders don't provide split out extradata directly on init (or
>> at all). In particular, the MediaFoundation encoder wrapper doesn't
>> always (depending on the actual encoder device) - this is the case for
>> Qualcomm's HEVC encoder on SD835, and also on some QSV H264 encoders).
>>
>> This only works for cases where the moov hasn't already been written
>> (e.g. when not writing fragmented mp4 with empty_moov, unless using
>> the delay_moov option).
>>
>> Signed-off-by: Martin Storsjö <martin at martin.st>
>> ---
>>  libavformat/movenc.c | 4 +++-
>>  1 file changed, 3 insertions(+), 1 deletion(-)
>>
>> diff --git a/libavformat/movenc.c b/libavformat/movenc.c
>> index 27d7621e27..6a85440a3f 100644
>> --- a/libavformat/movenc.c
>> +++ b/libavformat/movenc.c
>> @@ -5584,7 +5584,9 @@ int ff_mov_write_packet(AVFormatContext *s, AVPacket *pkt)
>>
>>      if ((par->codec_id == AV_CODEC_ID_DNXHD ||
>>           par->codec_id == AV_CODEC_ID_TRUEHD ||
>> -         par->codec_id == AV_CODEC_ID_AC3) && !trk->vos_len) {
>> +         par->codec_id == AV_CODEC_ID_AC3 ||
>> +         par->codec_id == AV_CODEC_ID_H264 ||
>> +         par->codec_id == AV_CODEC_ID_HEVC) && !trk->vos_len) {
>>          /* copy frame to create needed atoms */
>>          trk->vos_len  = size;
>>          trk->vos_data = av_malloc(size + AV_INPUT_BUFFER_PADDING_SIZE);
>
> This changes avcintra output
>
> example testcase:
> ./ffmpeg  -i ~/videos/mm-short.mpg -avcintra-class 100 -tune psnr -flags +ildct-global_header -t 0.5 -pix_fmt yuv422p10 -vf scale=1920:1080 -an file.mov

Right, if explicitly disabling global headers on the encoder, this will 
end up putting them back.

Any ideas on what the best path forward would be?

// Martin