[FFmpeg-user] Creating filter calling external library

[Jonathan Viney]

Hi everyone,

We have a filter that sends video frames to another C library for
processing, then collects the result and applies some transformations to
the frames.

This filter runs as part of a longer filter chain that may include things
like rotate and scale, as well as then producing several h.264 outputs.

Currently the library runs synchronously. We send it 1 frame, and it gives
us one back.

Would there be any potential performance benefit from making the calls from
the ffmpeg filter into the library non-blocking? We would most likely run
the library in another thread.

Eg, instead of:

  library.send_frame(frame);
  result = library.get_result();
  ... process result and send frame to output

Something like:

if (ff_outlink_frame_wanted(outlink) && library.has_results()) {
  ... process results, sending frame(s) to output
}
if (ff_inlink_check_available_frame(inlink) &&
library.ready_for_next_frame()) {
  library.send_frame(frame);
}
return FFERROR_NOT_READY;

I would expect that for a reasonable number of calls to activate(), there
will be no forward steps that can be taken, so it will return
FFERROR_NOT_READY. Could this be a problem for the filter system?

Would the redesign potentially allow ffmpeg to do some useful work instead
of being blocked on calls into the external library?

Cheers,
-Jonathan.