[FFmpeg-trac] #9650(avcodec:new): h264_qsv DECODER eventually freezes after many seeks [WINDOWS]
FFmpeg
trac at avcodec.org
Wed Feb 16 12:30:50 EET 2022
#9650: h264_qsv DECODER eventually freezes after many seeks [WINDOWS]
-------------------------------------+-------------------------------------
Reporter: teslan | Owner: (none)
Type: defect | Status: new
Priority: normal | Component: avcodec
Version: git-master | Resolution:
Keywords: qsv, intel | Blocked By:
media sdk, |
Blocking: | Reproduced by developer: 0
Analyzed by developer: 0 |
-------------------------------------+-------------------------------------
Description changed by teslan:
Old description:
> Summary of the bug:
> The h264_qsv decoder freezes after random number of seeks with flushes
> (most of the times you need max 100 seeks). Does not happen when standard
> h264 or h264_cuvid decoder is used.
>
> Note that this might be a bug not in FFMPEG, but in Intel Media SDK.
> I am not sure on which grave should I cry, but because there are
> developers from Intel actively enhancing the QSV support, i'll describe
> the issue both here and at Intel Media SDK github.
>
> Tested with both Windows-provided libmfxhw64.dll (File Version
> 11.21.4.14) and from the latest driver (File Version 21.11.3.320)
>
> This is the debug call stack on my decoding thread when the freeze
> happens:
>
> 1 ZwWaitForAlertByThreadId
> ntdll 0x7fff7a530764
> 2 RtlSleepConditionVariableSRW
> ntdll 0x7fff7a4f4021
> 3 SleepConditionVariableSRW
> KERNELBASE 0x7fff77ecce89
> 4 MFXVideoVPP_GetVPPStat
> libmfxhw64 0x7fff027ef8cc
> 5 MFXVideoVPP_GetVPPStat
> libmfxhw64 0x7fff027ec7f4
> 6 MFXVideoVPP_GetVPPStat
> libmfxhw64 0x7fff027ec69f
> 7 MFXVideoVPP_GetVPPStat
> libmfxhw64 0x7fff023575b0
> 8 MFXVideoCORE_SyncOperation
> libmfxhw64 0x7fff02337a4d
> 9 av_qsv_alloc_context
> avcodec_59 0x7fff10eb4c06
> 10 av_qsv_alloc_context
> avcodec_59 0x7fff10eb573d
> 11 av_qsv_alloc_context
> avcodec_59 0x7fff10eb59c7
> 12 avcodec_default_get_buffer2
> avcodec_59 0x7fff10a9f9b1
> 13 avcodec_send_packet
> avcodec_59 0x7fff10aa0590
> 14 libavDecoder::playLoopPrivate
> libavdecoder.cpp 724 0x7ff78094f6da
>
> I am using FFMPEG builds from BtbN - confirmed with the latest master
> update (2022-02-15 12:35)
>
> How to reproduce:
> You need to flush the decoder and seek the video many times. Then after
> some decoded frames the decoder freezes. These are important excerpts
> from my code to hint how am I using it:
>
> Seek:
> {{{
> ...........................
> err = av_seek_frame(m.fmtctx, m.videoStream, [random-pts-in-media], 0);
> if(err != 0) {
> av_strerror(err, errbuff, 256);
> std::cerr << "Error Seeking. AVERROR: " << errbuff <<
> std::endl;
> }
>
> ptsToFind = referencePts;
> seekMode = true;
> seekModeKeyframeFound = false;
> seekModeFlushed = false;
>
> .........................
> }}}
> Read&decode:
>
> {{{
> ...........................
> //Reads the right amount of data from stream
> err = av_read_frame(m.fmtctx, pPacket);
> if(err < 0) {
> av_strerror(err, errbuff, 256);
> std::cerr << "AV READ FRAME error or EOF " << errbuff << " "
> << m.lastPts[m.videoStream] << std::endl;
> if(err == AVERROR_EOF) {
> emit endOfFile(m.lastPts[m.videoStream]);
> }
> av_packet_unref(pPacket);
> break;
> }
>
> //If we are seeking, flush the codec and reset the GOP counter
> (as we automatically found nearest I frame)
> if(seekMode) {
>
> if(!seekModeFlushed) {
> if(!(pPacket->flags & AV_PKT_FLAG_KEY)){
> av_packet_unref(pPacket);
> continue;
> }
> avcodec_send_packet(avcc, NULL); //flush
> avcodec_flush_buffers(avcc);
> actual_gop_counter = 0;
> seekModeFlushed = true;
> }
> }
>
> //Send the "right amount of data" to the decoder
> err = avcodec_send_packet(avcc, pPacket);
> if(err != 0) {
> av_strerror(err, errbuff, 256);
> std::cerr << "Error sending packet. AVERROR: " << errbuff <<
> std::endl;
> av_packet_unref(pPacket);
> continue;
> }
>
> //Gather first ready frame from the internal queue
> err = avcodec_receive_frame(avcc, pFrame);
> if(err != 0) {
> if(err != AVERROR(EAGAIN)) { //resource can be unavailable
> pretty often, do not write out anything in such case
> av_strerror(err, errbuff, 256);
> std::cerr << "Error receiving frame. AVERROR: " <<
> errbuff << std::endl;
> }
> av_packet_unref(pPacket);
> av_frame_unref(pFrame);
> continue;
> }
>
> //Save data for average GOP computation
> if(pFrame->pict_type == AV_PICTURE_TYPE_I && actual_gop_counter
> != 0) {
> if(actual_gop_counter > max_gop) {
> max_gop = actual_gop_counter;
> }
> actual_gop_counter = 0;
> }
> ++actual_gop_counter;
>
> if(seekMode) {
> if(pFrame->pts >= ptsToFind && pFrame->pts <= ptsToFind +
> avg_pts*MAX_ALLOWED_SKIP_FRAMES) {
> seekMode = false;
> //Because QSV is a fucked-up decoder, it always returns
> exactly TWO frames from the previous
> //decoding chunk. And you can not flush it, it will only
> return them after you provide
> //two fresh packets. Really fucked-up. Really. Therefore
> we can not stay with
> //the check that if the pts is already the same or
> greater that the one we need to find.
> //We also have to introduce if it is not too far away.
> That is defined in MAX_ALLOWED_SKIP_FRAMES
> }
> else {
> av_frame_unref(pFrame);
> av_packet_unref(pPacket);
> continue; //do not proceed if still in seek
> }
> }
>
> .............................
> }}}
>
>
>
> Patches should be submitted to the ffmpeg-devel mailing list and not this
> bug tracker.
New description:
Summary of the bug:
The h264_qsv decoder freezes after random number of seeks with flushes
(most of the times you need max 100 seeks). Does not happen when standard
h264 or h264_cuvid decoder is used.
Note that this might be a bug not in FFMPEG, but in Intel Media SDK.
I am not sure on which grave should I cry, but because there are
developers from Intel actively enhancing the QSV support, i'll describe
the issue both here and at Intel Media SDK github.
Tested with both Windows-provided libmfxhw64.dll (File Version 11.21.4.14)
and from the latest driver (File Version 21.11.3.320)
This is the debug call stack on my decoding thread when the freeze
happens:
1 ZwWaitForAlertByThreadId
ntdll 0x7fff7a530764
2 RtlSleepConditionVariableSRW
ntdll 0x7fff7a4f4021
3 SleepConditionVariableSRW
KERNELBASE 0x7fff77ecce89
4 MFXVideoVPP_GetVPPStat
libmfxhw64 0x7fff027ef8cc
5 MFXVideoVPP_GetVPPStat
libmfxhw64 0x7fff027ec7f4
6 MFXVideoVPP_GetVPPStat
libmfxhw64 0x7fff027ec69f
7 MFXVideoVPP_GetVPPStat
libmfxhw64 0x7fff023575b0
8 MFXVideoCORE_SyncOperation
libmfxhw64 0x7fff02337a4d
9 av_qsv_alloc_context
avcodec_59 0x7fff10eb4c06
10 av_qsv_alloc_context
avcodec_59 0x7fff10eb573d
11 av_qsv_alloc_context
avcodec_59 0x7fff10eb59c7
12 avcodec_default_get_buffer2
avcodec_59 0x7fff10a9f9b1
13 avcodec_send_packet
avcodec_59 0x7fff10aa0590
14 libavDecoder::playLoopPrivate
libavdecoder.cpp 724 0x7ff78094f6da
I am using FFMPEG builds from BtbN - confirmed with the latest master
update (2022-02-15 12:35)
How to reproduce:
You need to flush the decoder and seek the video many times. Then after
some decoded frames the decoder freezes. These are important excerpts from
my code to hint how am I using it - in my use case I need to send an exact
amount of frames from a given PTS with the need to consider the GOP
interval:
Seek:
{{{
...........................
err = av_seek_frame(m.fmtctx, m.videoStream, [random-pts-in-media minus
GOP interval], 0);
if(err != 0) {
av_strerror(err, errbuff, 256);
std::cerr << "Error Seeking. AVERROR: " << errbuff <<
std::endl;
}
ptsToFind = referencePts;
seekMode = true;
seekModeKeyframeFound = false;
seekModeFlushed = false;
.........................
}}}
Read&decode:
{{{
...........................
//Reads the right amount of data from stream
err = av_read_frame(m.fmtctx, pPacket);
if(err < 0) {
av_strerror(err, errbuff, 256);
std::cerr << "AV READ FRAME error or EOF " << errbuff << " "
<< m.lastPts[m.videoStream] << std::endl;
if(err == AVERROR_EOF) {
emit endOfFile(m.lastPts[m.videoStream]);
}
av_packet_unref(pPacket);
break;
}
//If we are seeking, flush the codec and reset the GOP counter (as
we automatically found nearest I frame)
if(seekMode) {
if(!seekModeFlushed) {
if(!(pPacket->flags & AV_PKT_FLAG_KEY)){
av_packet_unref(pPacket);
continue;
}
avcodec_send_packet(avcc, NULL); //flush
avcodec_flush_buffers(avcc);
actual_gop_counter = 0;
seekModeFlushed = true;
}
}
//Send the "right amount of data" to the decoder
err = avcodec_send_packet(avcc, pPacket);
if(err != 0) {
av_strerror(err, errbuff, 256);
std::cerr << "Error sending packet. AVERROR: " << errbuff <<
std::endl;
av_packet_unref(pPacket);
continue;
}
//Gather first ready frame from the internal queue
err = avcodec_receive_frame(avcc, pFrame);
if(err != 0) {
if(err != AVERROR(EAGAIN)) { //resource can be unavailable
pretty often, do not write out anything in such case
av_strerror(err, errbuff, 256);
std::cerr << "Error receiving frame. AVERROR: " << errbuff
<< std::endl;
}
av_packet_unref(pPacket);
av_frame_unref(pFrame);
continue;
}
//Save data for average GOP computation
if(pFrame->pict_type == AV_PICTURE_TYPE_I && actual_gop_counter !=
0) {
if(actual_gop_counter > max_gop) {
max_gop = actual_gop_counter;
}
actual_gop_counter = 0;
}
++actual_gop_counter;
if(seekMode) {
if(pFrame->pts >= ptsToFind && pFrame->pts <= ptsToFind +
avg_pts*MAX_ALLOWED_SKIP_FRAMES) {
seekMode = false;
//Because QSV is a fucked-up decoder, it always returns
exactly TWO frames from the previous
//decoding chunk. And you can not flush it, it will only
return them after you provide
//two fresh packets. Really fucked-up. Really. Therefore
we can not stay with
//the check that if the pts is already the same or greater
that the one we need to find.
//We also have to introduce if it is not too far away.
That is defined in MAX_ALLOWED_SKIP_FRAMES
}
else {
av_frame_unref(pFrame);
av_packet_unref(pPacket);
continue; //do not proceed if still in seek
}
}
.............................
}}}
Patches should be submitted to the ffmpeg-devel mailing list and not this
bug tracker.
--
--
Ticket URL: <https://trac.ffmpeg.org/ticket/9650#comment:1>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker
More information about the FFmpeg-trac
mailing list