[FFmpeg-devel] Flushing while decoding , but need already decoded frames

[Andreas Rheinhardt]

Michael Henrik Bodenhoff via ffmpeg-devel:
> Hi ,
> 
> my team recently had to abandon switching to using FFmpeg from specific decoder implementations (NvDEC, Intel Media SDK , IPP and quite a few codec specific decoders) because of big performance issues because of the way FFmpeg works….. or at least we think it is (we’re FFmpeg noobs 😃  )
> 
> It's actually an issue we also had with Intel Media SDK,  leading us to pay Intel to extend Media SDK to do what we needed.
> 
> Our product is a video surveillance system, and that means we have to decode a LOT of video streams simultaneously.
> 
> For motion detection we want to only decode keyframes, and skip P and B frames , and that works fine with FFmpeg most of the time, except for when the video stream contains B frames.
> Without B-Frames it’s really simple (simplified pseudocode) :
> 
> while(true)
> {
>   receiveStreamCompleteFrame();
>   If(KeyFrame)
>   {
>     avcodec_send_packet();
>     if(avcodec_receive_frame()==0)
>     {
>        // do motion detection
>     }
>   }
> }
> 
> But! with B Frames FFmpeg doesn’t return keyframes when they are decoded, they are kept, and we can’t seem to flush them out. avcodec_flush_buffers allow us to continue to next keyframe, but it doesn’t seem to give us the keyframe we just gave to FFmpeg with avcodec_send_packet.
> 
> while(true)
> {
>   receiveStreamCompleteFrame();
>   If(KeyFrame)
>   {
>     avcodec_send_packet();
>     if(avcodec_receive_frame()==0)
>     {
>        // do motion detection
>     }
>     Else
>     {
>       avcodec_flush_buffers();
>       if(avcodec_receive_frame()==0)
>       {
>          // do motion detection
>       }
>     }
>   }
> }
> 
> Calling avcodec_receive_frame after calling avcodec_flush_buffer results in -11 and no frame
> 
> is there anyway around this ? And if not, could FFmpeg be made to have this functionality ?
> 
> I tried contacting one of the FFmpeg consultants from https://ffmpeg.org/consulting.html but never got a response
> 

Send your packet with the keyframe, send a NULL packet (to signal EOF),
then the internally stored frames should be output by
avcodec_receive_frame(). Then flush the decoder (to be able to send new
packets to it).

- Andreas