[FFmpeg-devel] [PATCH] area changed: scdet filter

Sun Jun 2 23:17:29 EEST 2024

> -----Original Message-----
> From: ffmpeg-devel <ffmpeg-devel-bounces at ffmpeg.org> On Behalf Of
> Michael Niedermayer
> Sent: vineri, 31 mai 2024 00:32
> To: FFmpeg development discussions and patches <ffmpeg-devel at ffmpeg.org>
> Subject: Re: [FFmpeg-devel] [PATCH] area changed: scdet filter
> 
> On Mon, May 13, 2024 at 06:52:19PM +0300, radu.taraibuta at gmail.com
> wrote:
> > Previous observations:
> >
> >  - Inconsistent code style with other filters. (Mostly using
> > AVFilterLink* link instead of AVFilterLink *link).
> > I hope it's fine now.
> >
> >  - Unrelated changes, please split trivial unrelated changes into
> > separate patches.
> > Removed trivial changes from this patch.
> >
> >  - Can't tables be generated at .init/.config_props time? No point in
> > storing them into binary.
> > Done.
> >
> >  - Adding extra delay is not backward compatible change, it should be
> > implemented properly by adding option for users to select mode: next &
> > prev frame or just next or prev frame.
> > Added legacy option to the mode parameter.
> >
> >  - Could split frame clone change into earlier separate patch.
> > Cannot be done. It's either frame clone or 1 frame delay.
> >
> >  - Where are results of improvements with accuracy so it can be
confirmed?
> > Here are my test results with manual labeling of scene changes:
> > 2379	Full length movie
> >
> > Method	Threshold	TP	FP	FN		Precision
> > Recall	F
> > Cubic	7	2357	423	22		0.847841727
0.990752417
> > 0.913742973
> > Cubic	10	2297	200	82		0.919903885
0.965531736
> > 0.94216571
> > Cubic	12	2217	146	162		0.938214135
0.931904161
> > 0.935048503
> > Cubic	15	2049	101	330		0.953023256
0.861286255
> > 0.904835505
> > Linear	2.8	2357	1060	22		0.689786362
0.990752417
> > 0.813319531
> > Linear	8	2099	236	280		0.898929336
0.882303489
> > 0.890538821
> > Linear	10	1886	173	493		0.91597863
0.792770071
> > 0.849932402
> > Legacy	5	2235	1260	144		0.639484979
> 	0.939470366
> > 0.760980592
> > Legacy	8	1998	414	381		0.828358209
> 	0.839848676
> > 0.83406387
> > Legacy	10	1743	193	636		0.900309917
> 	0.732660782
> > 0.80787949
> >
> > 15	HDR10Plus_PB_EAC3JOC
> > https://mega.nz/file/nehDka6Z#C5_OPbSZkONdOp1jRmc09C9-
> viDc3zMj8ZHruHcW
> > KyA
> >
> > Method	Threshold	TP	FP	FN		Precision
> > Recall	F
> > Cubic	10	15	0	0		1	1	1
> > Linear	5	13	1	2		0.928571429
0.866666667
> > 0.896551724
> > Legacy	5	12	2	3		0.857142857	0.8
> > 0.827586207
> >
> > 21	(HDR HEVC 10-bit BT.2020 24fps) Exodus Sample
> >
> https://mega.nz/file/Sfw1hDpK#ErxCOpQDVjcI1gq6ZbX3vIfdtXZompkFe0jq47E
> h
> > R2o
> >
> > Method	Threshold	TP	FP	FN		Precision
> > Recall	F
> > Cubic	10	21	0	0		1	1	1
> > Linear	4	20	0	1		1	0.952380952
> > 0.975609756
> > Legacy	4	19	0	2		1	0.904761905
> 	0.95
> >
> > 94	Bieber Grammys
> > https://mega.nz/#!c9dhAaKA!MG5Yi-
> MJNATE2_KqcnNJZCRKtTWvdjJP1NwG8Ggdw3E
> >
> > Method	Threshold	TP	FP	FN		Precision
> > Recall	F
> > Cubic	15	91	23	3		0.798245614
0.968085106
> > 0.875
> > Cubic	18	85	9	9		0.904255319
0.904255319
> > 0.904255319
> > Linear	7	79	49	15		0.6171875
0.840425532
> > 0.711711712
> > Linear	8	74	28	20		0.725490196
0.787234043
> > 0.755102041
> > Legacy	7	74	40	20		0.649122807
> 	0.787234043
> > 0.711538462
> > Legacy	8	71	26	23		0.731958763
> 	0.755319149
> > 0.743455497
> >
> >
> > Improve scene detection accuracy by comparing frame with both previous
> > and next frame (creates one frame delay).
> > Add new mode parameter and new method to compute the frame difference
> > using cubic square to increase the weight of small changes and new mean
> formula.
> > This improves accuracy significantly. Slightly improve performance by
> > not using frame clone.
> > Add legacy mode for backward compatibility.
> >
> > Signed-off-by: raduct <radu.taraibuta at gmail.com>
> > ---
> >  doc/filters.texi            |  16 ++++
> >  libavfilter/scene_sad.c     | 151 ++++++++++++++++++++++++++++++++++
> >  libavfilter/scene_sad.h     |   6 ++
> >  libavfilter/vf_scdet.c      | 156 +++++++++++++++++++++++++-----------
> >  tests/fate/filter-video.mak |   3 +
> >  5 files changed, 284 insertions(+), 48 deletions(-)
> >
> > diff --git a/doc/filters.texi b/doc/filters.texi index
> > bfa8ccec8b..53814e003b 100644
> > --- a/doc/filters.texi
> > +++ b/doc/filters.texi
> > @@ -21797,6 +21797,22 @@ Default value is @code{10.}.
> >  @item sc_pass, s
> >  Set the flag to pass scene change frames to the next filter. Default
> > value is @code{0}
> 
> The patch is corrupted by linebreaks:
> 
> Applying: area changed: scdet filter
> error: corrupt patch at line 16
> Patch failed at 0001 area changed: scdet filter
> 
> please check the linebreak settings or attach the patch or use git
send-email
> 
> thx
> 
> [...]
> --
> Michael     GnuPG fingerprint:
> 9FF2128B147EF6730BADF133611EC787040B0FAB
> 
> Homeopathy is like voting while filling the ballot out with transparent
ink.
> Sometimes the outcome one wanted occurs. Rarely its worse than filling out
a
> ballot properly.

Please find attached the patch.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0001-area-changed-scdet-filter.patch
Type: application/octet-stream
Size: 16106 bytes
Desc: not available
URL: <https://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20240602/e85e72af/attachment.obj>