[FFmpeg-devel] [PATCH 1/9] dnxhdenc: fix scan used for bitstream generation

[Michael Niedermayer]

On Sun, Oct 11, 2015 at 11:07:25PM +0200, Christophe Gisquet wrote:
> Hi,
> 
> 2015-10-11 18:20 GMT+02:00 Michael Niedermayer <michael at niedermayer.cc>:
> > On Sun, Oct 11, 2015 at 04:06:05PM +0200, Christophe Gisquet wrote:
> >> The functions related to bitstream reading must use the natural zigzag
> >> order, and not the one permuted for use in the iDCT. This currently
> >> results in bitstreams with 2 issues:
> >> - AC coefficients are encoded in an unexpected order;
> >> - Incorrect weights are applied in the (de)quantization.
> >>
> >> This currently can't show up because there's no simple_idct which
> >> requires a permutation.
> >>
> >> With one that does (based on prores'), and using the following source:
> >> -f lavfi -i testsrc=s=1920x1080 -vframes 1 -pix_fmt yuv422p10le
> >> and encoding to CID 1241 (interlaced 1080p YUV422) using:
> >> -vcodec dnxhd -flags +ildct -vb 185M
> >>
> >> before: stddev:   55.57 PSNR: 61.43 MAXDIFF:  960
> >> after:  stddev:    0.35 PSNR:105.45 MAXDIFF:   11
> >> ---
> >>  libavcodec/dnxhdenc.c | 12 +++++++-----
> >>  1 file changed, 7 insertions(+), 5 deletions(-)
> >
> > breaks:
> > ./ffmpeg -f lavfi -i testsrc=s=1920x1080 -vframes 1 -pix_fmt yuv422p -vcodec dnxhd -flags +ildct -vb 185M test.mov
> 
> > and thinking about it a bit more (which i should have done earlier)
> > i suspect it to break every 8bit case
> > the fdct permutes its output if the idct_permutation is set so its
> 
> OK, so that's the issue, and the current dnxhdenc code totally misled me here.
> 
> So, actually, it's the quantize function that applies this, and it's
> missing from the 10bits version.
> 
> In the end, I don't think the code really needs all of this (there's
> one single scantable...), but that cleanup is not for now.
> 
> I decided to reuse the code from mpegvideo_enc in that matter, so see
> the new patch here.
> 
> -- 
> Christophe

>  dnxhdenc.c      |   11 ++++++++---
>  mpegvideo.h     |    3 ++-
>  mpegvideo_enc.c |    6 +++---
>  3 files changed, 13 insertions(+), 7 deletions(-)
> c4ff74009048b80d9e5b7c7639d6c24109edb325  0001-mpegvideo-dnxhdenc-permute-10bits-content.patch
> From b7e33c1bd9c3645b3b242d336e2e728bbfd4b822 Mon Sep 17 00:00:00 2001
> From: Christophe Gisquet <christophe.gisquet at gmail.com>
> Date: Sun, 11 Oct 2015 22:58:21 +0200
> Subject: [PATCH 1/9] mpegvideo: dnxhdenc: permute 10bits content
> 
> Dequant or encoding were trying to reverse a scan that hadn't been
> applied...

this seems to cause artifacts on the red/cyan edge in
./ffmpeg -f lavfi -i testsrc=s=1920x1080 -vframes 1 -pix_fmt yuv422p -vcodec dnxhd -flags +ildct -vb 185M test.mov


[...]

-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

The educated differ from the uneducated as much as the living from the
dead. -- Aristotle 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: Digital signature
URL: <http://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20151012/a53093f3/attachment.sig>