[FFmpeg-devel] Flushing while decoding , but need already decoded frames

[Michael Henrik Bodenhoff]

Hi ,

my team recently had to abandon switching to using FFmpeg from specific decoder implementations (NvDEC, Intel Media SDK , IPP and quite a few codec specific decoders) because of big performance issues because of the way FFmpeg works….. or at least we think it is (we’re FFmpeg noobs 😃  )

It's actually an issue we also had with Intel Media SDK,  leading us to pay Intel to extend Media SDK to do what we needed.

Our product is a video surveillance system, and that means we have to decode a LOT of video streams simultaneously.

For motion detection we want to only decode keyframes, and skip P and B frames , and that works fine with FFmpeg most of the time, except for when the video stream contains B frames.
Without B-Frames it’s really simple (simplified pseudocode) :

while(true)
{
  receiveStreamCompleteFrame();
  If(KeyFrame)
  {
    avcodec_send_packet();
    if(avcodec_receive_frame()==0)
    {
       // do motion detection
    }
  }
}

But! with B Frames FFmpeg doesn’t return keyframes when they are decoded, they are kept, and we can’t seem to flush them out. avcodec_flush_buffers allow us to continue to next keyframe, but it doesn’t seem to give us the keyframe we just gave to FFmpeg with avcodec_send_packet.

while(true)
{
  receiveStreamCompleteFrame();
  If(KeyFrame)
  {
    avcodec_send_packet();
    if(avcodec_receive_frame()==0)
    {
       // do motion detection
    }
    Else
    {
      avcodec_flush_buffers();
      if(avcodec_receive_frame()==0)
      {
         // do motion detection
      }
    }
  }
}

Calling avcodec_receive_frame after calling avcodec_flush_buffer results in -11 and no frame

is there anyway around this ? And if not, could FFmpeg be made to have this functionality ?

I tried contacting one of the FFmpeg consultants from https://ffmpeg.org/consulting.html but never got a response

Michael Bodenhoff
Principal Software Engineer
FT01
Email: mibo at milestone.dk

[Linkedin Logo]<https://linkedin.com/company/milestone-systems>[Twitter Logo]<https://twitter.com/milestonesys>[Youtube Logo]<https://www.youtube.com/user/Milestonesys>[Facebook Logo]<https://www.facebook.com/milestonesys/>[Instagram Logo]<https://www.instagram.com/milestonesystems/>

[Milestone Logo]<https://www.milestonesys.com/>

-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 183 bytes
Desc: image001.png
URL: <https://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20240524/f3c02102/attachment.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image002.png
Type: image/png
Size: 3730 bytes
Desc: image002.png
URL: <https://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20240524/f3c02102/attachment-0001.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image003.png
Type: image/png
Size: 6222 bytes
Desc: image003.png
URL: <https://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20240524/f3c02102/attachment-0002.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image004.png
Type: image/png
Size: 4532 bytes
Desc: image004.png
URL: <https://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20240524/f3c02102/attachment-0003.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image005.png
Type: image/png
Size: 2547 bytes
Desc: image005.png
URL: <https://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20240524/f3c02102/attachment-0004.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image006.png
Type: image/png
Size: 12009 bytes
Desc: image006.png
URL: <https://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20240524/f3c02102/attachment-0005.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image007.png
Type: image/png
Size: 8722 bytes
Desc: image007.png
URL: <https://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20240524/f3c02102/attachment-0006.png>