[FFmpeg-user] Programmatically detecting 'busiest' parts of a video

[Rob Hallam]

> > On 7 Aug 2024, at 11:04, Phil Rhodes via ffmpeg-user <ffmpeg-user at ffmpeg.org> wrote:
> >
> > Might this be achieved simply by detecting the differences between adjacent frames?
>
> <int>
> Sleeping cat
> Cut to:
> Plant on windowsill
> Bouke

Heh.

Interesting thought. Question is- how do you detect differences?

As Bouke and inhahe point out, a minor change in angle or a scene
change will show up as a huge change, though I think the latter is
less of a problem: a change of scene will result in let's say for sake
of argument 100% change between frames- but if the change is averaged
out over 1 second or 5 seconds it's going to be pretty minimal
compared to a busy scene with more or less each frame changing.

I had a thought about handling minor camera changes which would result
in (seemingly) all pixels changing, which was to sort all the pixels
in a frame and then compare -- ordering a frame's pixels is a pretty
nonsensical thing to do in almost every other context -- as a scene
with a slight translation of camera view will have very similar pixels
overall.

The 'check bitrate over time' is probably the simplest, and the
cheapest computationally- I'd love another approach for CBR videos
which would also be IO bound.

Cheers,
Rob