[FFmpeg-devel] [PATCH 2/2] libavcodec/ffv1: Support storing LSB raw

[Michael Niedermayer]

On Wed, Oct 16, 2024 at 03:36:55PM +0200, Michael Niedermayer wrote:
> On Wed, Oct 16, 2024 at 02:13:35AM +0200, Lynne via ffmpeg-devel wrote:
> > On 16/10/2024 01:17, Michael Niedermayer wrote:
> > > This makes a 16bit RGB raw sample 25% faster at a 2% loss of compression with rawlsb=4
> > > 
> > > Please test and comment
> > > 
> > > This stores the LSB through non binary range coding, this is simpler than using a
> > > separate coder
> > > For cases where range coding is not wanted its probably best to use golomb rice
> > > for everything.
> > > 
> > > We also pass the LSB through the decorrelation and context stages (which is basically free)
> > > this leads to slightly better compression than separating them earlier.
> > > 
> > > Signed-off-by: Michael Niedermayer <michael at niedermayer.cc>
> > > ---
> > >   libavcodec/ffv1.h             |  2 ++
> > >   libavcodec/ffv1_template.c    | 19 ++++++++++---------
> > >   libavcodec/ffv1dec.c          |  2 ++
> > >   libavcodec/ffv1dec_template.c | 16 +++++++++++++---
> > >   libavcodec/ffv1enc.c          | 15 ++++++++++++++-
> > >   libavcodec/ffv1enc_template.c | 17 +++++++++++++++--
> > >   libavcodec/rangecoder.h       | 20 ++++++++++++++++++++
> > >   libavcodec/tests/rangecoder.c |  9 +++++++++
> > >   8 files changed, 85 insertions(+), 15 deletions(-)
> [...]
> > > diff --git a/libavcodec/rangecoder.h b/libavcodec/rangecoder.h
> > > index 89d178ac314..d02a65fa7da 100644
> > > --- a/libavcodec/rangecoder.h
> > > +++ b/libavcodec/rangecoder.h
> > > @@ -111,6 +111,16 @@ static inline void put_rac(RangeCoder *c, uint8_t *const state, int bit)
> > >       renorm_encoder(c);
> > >   }
> > > +static inline void put_rac_raw(RangeCoder *c, int bits, int len)
> > > +{
> > > +    int r = c->range >> len;
> > > +
> > > +    c->low += r * bits;
> > > +    c->range = r;
> > > +
> > > +    renorm_encoder(c);
> > > +}
> > > +
> > >   static inline void refill(RangeCoder *c)
> > >   {
> > >       if (c->range < 0x100) {
> > > @@ -142,4 +152,14 @@ static inline int get_rac(RangeCoder *c, uint8_t *const state)
> > >       }
> > >   }
> > > +static inline int get_rac_raw(RangeCoder *c, int len)
> > > +{
> > > +    int r = c->range >> len;
> > > +    int bits = c->low / r;
> > > +    c->low -= r * bits;
> > > +    c->range = r;
> > > +    refill(c);
> > > +    return bits;
> > > +}
> > > +
> [...]
> >
> > You're interfering with the rangecoder by asking it to write very random
> > data in between each symbol.
> 
> the data is needed in that order for context modeling and decorrelation
> to work.
> 
> At least with the CPU implementation we have this gives the same speedup
> but better compression and its simpler code

btw, on a related note, whould something like this:

diff --git a/libavcodec/ffv1dec_template.c b/libavcodec/ffv1dec_template.c
index dbdcad7768e..5d4d51cc070 100644
--- a/libavcodec/ffv1dec_template.c
+++ b/libavcodec/ffv1dec_template.c
@@ -39,6 +39,8 @@ RENAME(decode_line)(FFV1Context *f, FFV1SliceContext *sc,
     if (is_input_end(c, gb, ac))
         return AVERROR_INVALIDDATA;

+    c->range = 1<<av_log2(c->range);
+
     if (sc->slice_coding_mode == 1) {
         int i;
         for (x = 0; x < w; x++) {
diff --git a/libavcodec/ffv1enc_template.c b/libavcodec/ffv1enc_template.c
index 848328c70af..0a3cb8f28b9 100644
--- a/libavcodec/ffv1enc_template.c
+++ b/libavcodec/ffv1enc_template.c
@@ -40,6 +40,7 @@ RENAME(encode_line)(FFV1Context *f, FFV1SliceContext *sc,
             av_log(logctx, AV_LOG_ERROR, "encoded frame too large\n");
             return AVERROR_INVALIDDATA;
         }
+        c->range = 1<<av_log2(c->range);
     } else {
         if (put_bytes_left(&sc->pb, 0) < w * 4) {
             av_log(logctx, AV_LOG_ERROR, "encoded frame too large\n");

help a GPU implementation ?
The idea here is to reduce the number of different states the range coder can be in
at the end of each line so the next line has fewer states to consider
but maybe iam totally wrong and misunderstanding the problem

Above change seems to cost maybe 0.1-0.2% compression on vsynth1 and vsynth2 tests

thx

[...]

-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

If you fake or manipulate statistics in a paper in physics you will never
get a job again.
If you fake or manipulate statistics in a paper in medicin you will get
a job for life at the pharma industry.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 195 bytes
Desc: not available
URL: <https://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20241016/7c10adca/attachment.sig>