[FFmpeg-devel] [PATCH 1/2] libavcodec/zmbvenc: block scoring improvements/bug fixes

Michael Niedermayer michael at niedermayer.cc
Fri Feb 22 23:53:29 EET 2019

On Fri, Feb 22, 2019 at 01:09:12PM +0100, Tomas Härdin wrote:
> fre 2019-02-22 klockan 00:11 +0100 skrev Michael Niedermayer:
> > On Thu, Feb 21, 2019 at 09:26:44PM +0000, Matthew Fearnley wrote:
> > > - To clear up the effects: the change from 'i=1' to 'i=0' was the only
> > > change that should make any difference in the output.
> > > The rest of the changes were to improve the speed of an inner loop, and to
> > > fix two bugs that happily cancelled each other out, but would have broken
> > > if the code were changed to emit larger blocks.  These should not lead to
> > > changes in the blocks chosen or the final compression size.
> > > 
> > > - Regarding the worse compression: let me start by stating that scoring on
> > > byte frequency alone will not always predict well how Deflate will work,
> > > since Deflate also uses pattern matching as well.
> > > Frequency-based block scoring will only ever be a heuristic.  There may be
> > > scope for improving the scoring, but it would add a lot of complexity, and
> > > I think it would be easy to make things worse rather than better.  (I found
> > > this in my brief experiments with including the frequencies from the
> > > previous block.)
> > 
> > implementing the exact scoring by using Deflate itself would allow us to
> > understand how much there is between the heuristic and what is achievable.
> > If there is a gap that is significant then it may be interresting to
> > look at improving the heuristic if the gap is small than the whole
> > heuristic improvment path can be discarded.
> > The obvious improvment for the heuristic would be to move to higher
> > order entropy estimation. ATM IIUC this uses a 0th order. 1st order could
> > be tried or any fast compression algorithm could also be used to estimate
> > entropy.
> We tried a sliding window order-0 model, which did worse. An order-1
> model might be useful, but would need quite a bit of context for it to
> be meaningful

> > It also would be interresting to see how a higher order heuristic or
> > some simple fast LZ like compressor would perform with the previous block.
> > That is if the issue you saw with the previous block inclusion was because
> > of the very simplistic entropy estimation.
> zlib at level 1 maybe? You could also have a multi-pass thing where it
> first does a coarse pass then tries to jiggle MVs and greedily accept
> any change that results in smaller output.

yes, but i would avoid using external code like zlib for this. Its not
designed for fast entropy estimation of many blocks. 

Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

Opposition brings concord. Out of discord comes the fairest harmony.
-- Heraclitus
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: not available
URL: <http://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20190222/e3b0c4c6/attachment.sig>

More information about the ffmpeg-devel mailing list