[FFmpeg-devel] [PATCH v3] fate: add tests for psnr and ssim filter
Michael Niedermayer
michael at niedermayer.cc
Wed Sep 20 02:38:43 EEST 2017
On Tue, Sep 19, 2017 at 09:48:45AM +0200, Tobias Rapp wrote:
> Metadata filter output is passed through an Awk script comparing floats
> against reference values with specified "fuzz" tolerance to account for
> architectural differences (e.g. x86-32 vs. x86-64).
>
> Signed-off-by: Tobias Rapp <t.rapp at noa-archive.com>
> ---
> tested on Linux x86-32/64 and Mips (Qemu)
>
> v3:
> - avoid precision loss due to rounding of float values
>
> v2:
> - removed CPUFLAGS work-around for ssim filter issue
> - added metadata float value post-processing script
>
> tests/fate-run.sh | 9 ++++++
> tests/fate/filter-video.mak | 14 ++++++++
> tests/ref/fate/filter-refcmp-psnr-rgb | 45 ++++++++++++++++++++++++++
> tests/ref/fate/filter-refcmp-psnr-yuv | 45 ++++++++++++++++++++++++++
> tests/ref/fate/filter-refcmp-ssim-rgb | 30 +++++++++++++++++
> tests/ref/fate/filter-refcmp-ssim-yuv | 30 +++++++++++++++++
> tests/refcmp-metadata.awk | 61 +++++++++++++++++++++++++++++++++++
> 7 files changed, 234 insertions(+)
> create mode 100644 tests/ref/fate/filter-refcmp-psnr-rgb
> create mode 100644 tests/ref/fate/filter-refcmp-psnr-yuv
> create mode 100644 tests/ref/fate/filter-refcmp-ssim-rgb
> create mode 100644 tests/ref/fate/filter-refcmp-ssim-yuv
> create mode 100644 tests/refcmp-metadata.awk
This seems not to work (normal x86-64 linux) but maybe ive done something silly
--- ./tests/ref/fate/filter-refcmp-psnr-rgb 2017-09-20 00:58:04.063337500 +0200
+++ tests/data/fate/filter-refcmp-psnr-rgb 2017-09-20 00:59:26.399339234 +0200
@@ -1,12 +1,3 @@
-frame:0 pts:0 pts_time:0
-lavfi.psnr.mse.r=1381.80
-lavfi.psnr.psnr.r=16.73
-lavfi.psnr.mse.g=896.00
-lavfi.psnr.psnr.g=18.61
-lavfi.psnr.mse.b=277.38
-lavfi.psnr.psnr.b=23.70
-lavfi.psnr.mse_avg=851.73
-lavfi.psnr.psnr_avg=18.83
frame:1 pts:1 pts_time:1
lavfi.psnr.mse.r=1380.37
lavfi.psnr.psnr.r=16.73
@@ -14,22 +5,12 @@
lavfi.psnr.psnr.g=18.24
lavfi.psnr.mse.b=435.72
lavfi.psnr.psnr.b=21.74
-lavfi.psnr.mse_avg=930.67
-lavfi.psnr.psnr_avg=18.44
-frame:2 pts:2 pts_time:2
-lavfi.psnr.mse.r=1403.20
-lavfi.psnr.psnr.r=16.66
-lavfi.psnr.mse.g=954.05
-lavfi.psnr.psnr.g=18.34
-lavfi.psnr.mse.b=494.22
-lavfi.psnr.psnr.b=21.19
-lavfi.psnr.mse_avg=950.49
-lavfi.psnr.psnr_avg=18.35
-frame:3 pts:3 pts_time:3
-lavfi.psnr.mse.r=1452.80
lavfi.psnr.psnr.r=16.51
+lavfi.psnr.mse_avg=930.67
lavfi.psnr.mse.g=1001.02
+lavfi.psnr.psnr_avg=18.44
lavfi.psnr.psnr.g=18.13
+frame:2 pts:2 pts_time:2
lavfi.psnr.mse.b=557.39
lavfi.psnr.psnr.b=20.67
lavfi.psnr.mse_avg=1003.74
@@ -37,9 +18,28 @@
frame:4 pts:4 pts_time:4
lavfi.psnr.mse.r=1401.25
lavfi.psnr.psnr.r=16.67
+frame:0 pts:0 pts_time:0
+lavfi.psnr.mse.r=1381.80
+lavfi.psnr.psnr.r=16.73
+lavfi.psnr.mse.g=896.00
+lavfi.psnr.psnr.g=18.61
+lavfi.psnr.mse.b=277.38
+lavfi.psnr.psnr.b=23.70
+lavfi.psnr.mse_avg=851.73
+lavfi.psnr.psnr_avg=18.83
+lavfi.psnr.mse.r=1403.20
+lavfi.psnr.psnr.r=16.66
+lavfi.psnr.mse.g=954.05
+lavfi.psnr.psnr.g=18.34
+lavfi.psnr.mse.b=494.22
+lavfi.psnr.psnr.b=21.19
+lavfi.psnr.mse_avg=950.49
lavfi.psnr.mse.g=1009.80
+lavfi.psnr.psnr_avg=18.35
lavfi.psnr.psnr.g=18.09
+frame:3 pts:3 pts_time:3
lavfi.psnr.mse.b=602.42
+lavfi.psnr.mse.r=1452.80
lavfi.psnr.psnr.b=20.33
lavfi.psnr.mse_avg=1004.49
lavfi.psnr.psnr_avg=18.11
Test filter-refcmp-psnr-rgb failed. Look at tests/data/fate/filter-refcmp-psnr-rgb.err for details.
make: *** [fate-filter-refcmp-psnr-rgb] Error 1
TEST fitsdec-bitpix-32
--- ./tests/ref/fate/filter-refcmp-ssim-rgb 2017-09-20 00:58:04.063337500 +0200
+++ tests/data/fate/filter-refcmp-ssim-rgb 2017-09-20 00:59:26.415339235 +0200
@@ -1,30 +1,30 @@
-frame:0 pts:0 pts_time:0
-lavfi.ssim.R=0.72
-lavfi.ssim.G=0.76
+lavfi.ssim.R=0.71
+lavfi.ssim.All=0.76
lavfi.ssim.B=0.89
+lavfi.ssim.G=0.74
+lavfi.ssim.dB=6.29
lavfi.ssim.All=0.79
+lavfi.ssim.B=0.80
+frame:3 pts:3 pts_time:3
lavfi.ssim.dB=6.74
+lavfi.ssim.All=0.75
frame:1 pts:1 pts_time:1
lavfi.ssim.R=0.70
lavfi.ssim.G=0.74
lavfi.ssim.B=0.85
-lavfi.ssim.All=0.77
-lavfi.ssim.dB=6.31
-frame:2 pts:2 pts_time:2
-lavfi.ssim.R=0.71
-lavfi.ssim.G=0.75
-lavfi.ssim.B=0.84
-lavfi.ssim.All=0.76
-lavfi.ssim.dB=6.29
-frame:3 pts:3 pts_time:3
lavfi.ssim.R=0.70
+lavfi.ssim.All=0.77
+lavfi.ssim.dB=6.05
lavfi.ssim.G=0.73
+lavfi.ssim.dB=6.31
lavfi.ssim.B=0.83
+frame:2 pts:2 pts_time:2
lavfi.ssim.All=0.76
+lavfi.ssim.R=0.71
+frame:0 pts:0 pts_time:0
lavfi.ssim.dB=6.11
+lavfi.ssim.G=0.75
+lavfi.ssim.R=0.72
frame:4 pts:4 pts_time:4
-lavfi.ssim.R=0.71
-lavfi.ssim.G=0.74
-lavfi.ssim.B=0.80
-lavfi.ssim.All=0.75
-lavfi.ssim.dB=6.05
+lavfi.ssim.B=0.84
+lavfi.ssim.G=0.76
Test filter-refcmp-ssim-rgb failed. Look at tests/data/fate/filter-refcmp-ssim-rgb.err for details.
make: *** [fate-filter-refcmp-ssim-rgb] Error 1
TEST fitsdec-bitpix-64
--- ./tests/ref/fate/filter-refcmp-psnr-yuv 2017-09-20 00:58:04.063337500 +0200
+++ tests/data/fate/filter-refcmp-psnr-yuv 2017-09-20 00:59:26.435339235 +0200
@@ -1,12 +1,3 @@
-frame:0 pts:0 pts_time:0
-lavfi.psnr.mse.y=222.06
-lavfi.psnr.psnr.y=24.67
-lavfi.psnr.mse.u=339.38
-lavfi.psnr.psnr.u=22.82
-lavfi.psnr.mse.v=705.41
-lavfi.psnr.psnr.v=19.65
-lavfi.psnr.mse_avg=372.23
-lavfi.psnr.psnr_avg=22.42
frame:1 pts:1 pts_time:1
lavfi.psnr.mse.y=236.74
lavfi.psnr.psnr.y=24.39
@@ -14,22 +5,12 @@
lavfi.psnr.psnr.u=21.94
lavfi.psnr.mse.v=704.98
lavfi.psnr.psnr.v=19.65
-lavfi.psnr.mse_avg=398.66
-lavfi.psnr.psnr_avg=22.12
-frame:2 pts:2 pts_time:2
-lavfi.psnr.mse.y=234.79
-lavfi.psnr.psnr.y=24.42
-lavfi.psnr.mse.u=435.72
-lavfi.psnr.psnr.u=21.74
-lavfi.psnr.mse.v=699.60
-lavfi.psnr.psnr.v=19.68
-lavfi.psnr.mse_avg=401.23
-lavfi.psnr.psnr_avg=22.10
-frame:3 pts:3 pts_time:3
-lavfi.psnr.mse.y=250.88
lavfi.psnr.psnr.y=24.14
+lavfi.psnr.mse_avg=398.66
lavfi.psnr.mse.u=479.73
+lavfi.psnr.psnr_avg=22.12
lavfi.psnr.psnr.u=21.32
+frame:2 pts:2 pts_time:2
lavfi.psnr.mse.v=707.55
lavfi.psnr.psnr.v=19.63
lavfi.psnr.mse_avg=422.26
@@ -37,9 +18,28 @@
frame:4 pts:4 pts_time:4
lavfi.psnr.mse.y=241.05
lavfi.psnr.psnr.y=24.31
+frame:0 pts:0 pts_time:0
+lavfi.psnr.mse.y=222.06
+lavfi.psnr.psnr.y=24.67
+lavfi.psnr.mse.u=339.38
+lavfi.psnr.psnr.u=22.82
+lavfi.psnr.mse.v=705.41
+lavfi.psnr.psnr.v=19.65
+lavfi.psnr.mse_avg=372.23
+lavfi.psnr.psnr_avg=22.42
+lavfi.psnr.mse.y=234.79
+lavfi.psnr.psnr.y=24.42
+lavfi.psnr.mse.u=435.72
+lavfi.psnr.psnr.u=21.74
+lavfi.psnr.mse.v=699.60
+lavfi.psnr.psnr.v=19.68
+lavfi.psnr.mse_avg=401.23
lavfi.psnr.mse.u=505.04
+lavfi.psnr.psnr_avg=22.10
lavfi.psnr.psnr.u=21.10
+frame:3 pts:3 pts_time:3
lavfi.psnr.mse.v=716.00
+lavfi.psnr.mse.y=250.88
lavfi.psnr.psnr.v=19.58
lavfi.psnr.mse_avg=425.79
lavfi.psnr.psnr_avg=21.84
Test filter-refcmp-psnr-yuv failed. Look at tests/data/fate/filter-refcmp-psnr-yuv.err for details.
make: *** [fate-filter-refcmp-psnr-yuv] Error 1
TEST fitsdec-multi
GEN tests/data/lena-gray.fits
GEN tests/data/lena-gbrp.fits
--- ./tests/ref/fate/filter-refcmp-ssim-yuv 2017-09-20 00:58:04.063337500 +0200
+++ tests/data/fate/filter-refcmp-ssim-yuv 2017-09-20 00:59:26.491339236 +0200
@@ -1,30 +1,30 @@
-frame:0 pts:0 pts_time:0
lavfi.ssim.Y=0.80
-lavfi.ssim.U=0.76
+lavfi.ssim.All=0.75
lavfi.ssim.V=0.69
+lavfi.ssim.U=0.72
+lavfi.ssim.dB=6.10
lavfi.ssim.All=0.76
-lavfi.ssim.dB=6.25
-frame:1 pts:1 pts_time:1
-lavfi.ssim.Y=0.80
-lavfi.ssim.U=0.73
lavfi.ssim.V=0.68
+frame:3 pts:3 pts_time:3
+lavfi.ssim.dB=6.25
lavfi.ssim.All=0.75
-lavfi.ssim.dB=6.08
-frame:2 pts:2 pts_time:2
+frame:1 pts:1 pts_time:1
lavfi.ssim.Y=0.80
lavfi.ssim.U=0.73
lavfi.ssim.V=0.68
-lavfi.ssim.All=0.75
-lavfi.ssim.dB=6.10
-frame:3 pts:3 pts_time:3
lavfi.ssim.Y=0.79
+lavfi.ssim.All=0.75
+lavfi.ssim.dB=5.97
lavfi.ssim.U=0.72
+lavfi.ssim.dB=6.08
lavfi.ssim.V=0.68
+frame:2 pts:2 pts_time:2
lavfi.ssim.All=0.75
+lavfi.ssim.Y=0.80
+frame:0 pts:0 pts_time:0
lavfi.ssim.dB=5.94
-frame:4 pts:4 pts_time:4
+lavfi.ssim.U=0.73
lavfi.ssim.Y=0.80
-lavfi.ssim.U=0.72
+frame:4 pts:4 pts_time:4
lavfi.ssim.V=0.68
-lavfi.ssim.All=0.75
-lavfi.ssim.dB=5.97
+lavfi.ssim.U=0.76
Test filter-refcmp-ssim-yuv failed. Look at tests/data/fate/filter-refcmp-ssim-yuv.err for details.
make: *** [fate-filter-refcmp-ssim-yuv] Error 1
[...]
--
Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB
Dictatorship: All citizens are under surveillance, all their steps and
actions recorded, for the politicians to enforce control.
Democracy: All politicians are under surveillance, all their steps and
actions recorded, for the citizens to enforce control.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: Digital signature
URL: <http://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20170920/d34b7572/attachment.sig>
More information about the ffmpeg-devel
mailing list