[FFmpeg-devel] Testing Inexact Data

[Benjamin Larsson]

Mike Melanson wrote:
> Hi,
>
> I am at the point with the FATE server ( http://fate.multimedia.cx/ ) 
> where I want to start testing data that is not defined to be bit exact. 
> Michael recommended that I should use the tiny_psnr utility. Sounds 
> great, except that, well... I'm not very good at math and I don't know 
> what the numbers actually mean. :)
>
> I decoded surge-2-16-B-.mp3.mov from:
>
>    http://samples.mplayerhq.hu/mov/qtaudio/
>
> I decoded the file to WAV on x86_32 and PPC machines and then ran:
>
> $ tiny_psnr mp3-ppc.wav mp3-x86.wav 2 0 0
> stddev:  0.53 PSNR:101.81 bytes:2322432
>
> When I compare one of the files against itself:
> stddev:  0.00 PSNR:99.99 bytes:2322432
>
> Issues:
>
> 1) What are good numbers vs. bad numbers in this case? And for other cases?
>
> 2) I'm assuming that anything on one side of a threshold should be a 
> "PASS" and anything on the other side should be a "FAIL". I need a tool 
> that can easily return this information. One approach would be to supply 
> a threshold number to tiny_psnr and modify the program to return 
> non-zero if that threshold fails.
>
>   

You have to skip the wav header.

./tiny_psnr
tiny_psnr <file1> <file2> [<elem size> [<shift> [<skip bytes>]]]
for wav files use the following:
./tiny_psnr file1.wav file2.wav 2 0 44 to skip the header.

And I would say that anything over 80dB PSNR should be considered a 
PASS. But that's just a guess.

MvH
Benjamin Larsson