[FFmpeg-devel] [PATCH] avutil: add API for mb types.

Ronald S. Bultje rsbultje at gmail.com
Sun Nov 12 19:54:02 EET 2017


Hi,

On Sun, Nov 12, 2017 at 11:42 AM, Michael Niedermayer <
michael at niedermayer.cc> wrote:

> On Sun, Nov 12, 2017 at 04:21:15PM +0000, Rostislav Pehlivanov wrote:
> > On 12 November 2017 at 15:59, Michael Niedermayer <michael at niedermayer.cc
> >
> > wrote:
> >
> > > This is based on motion_type.h
> > >
> > > TODO: docs & split into a commit per lib
> > > Signed-off-by: Michael Niedermayer <michael at niedermayer.cc>
> > > ---
> > >  libavcodec/avcodec.h       |   4 ++
> > >  libavcodec/options_table.h |   1 +
> > >  libavutil/block_type.h     | 107 ++++++++++++++++++++++++++++++
> > > +++++++++++++++
> > >  libavutil/frame.h          |  14 ++++++
> > >  4 files changed, 126 insertions(+)
> > >  create mode 100644 libavutil/block_type.h
> > >
> > > diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h
> > > index 15ca871b59..1a49fa0a9a 100644
> > > --- a/libavcodec/avcodec.h
> > > +++ b/libavcodec/avcodec.h
> > > @@ -923,6 +923,10 @@ typedef struct RcOverride{
> > >   * Show all frames before the first keyframe
> > >   */
> > >  #define AV_CODEC_FLAG2_SHOW_ALL       (1 << 22)
> > > +/**
> > > + * Export block types through frame side data
> > > + */
> > > +#define AV_CODEC_FLAG2_EXPORT_BLOCKS  (1 << 27)
> > >  /**
> > >   * Export motion vectors through frame side data
> > >   */
> > > diff --git a/libavcodec/options_table.h b/libavcodec/options_table.h
> > > index d89f58d540..0b86b4d0fb 100644
> > > --- a/libavcodec/options_table.h
> > > +++ b/libavcodec/options_table.h
> > > @@ -74,6 +74,7 @@ static const AVOption avcodec_options[] = {
> > >  {"chunks", "Frame data might be split into multiple chunks", 0,
> > > AV_OPT_TYPE_CONST, {.i64 = AV_CODEC_FLAG2_CHUNKS }, INT_MIN, INT_MAX,
> V|D,
> > > "flags2"},
> > >  {"showall", "Show all frames before the first keyframe", 0,
> > > AV_OPT_TYPE_CONST, {.i64 = AV_CODEC_FLAG2_SHOW_ALL }, INT_MIN, INT_MAX,
> > > V|D, "flags2"},
> > >  {"export_mvs", "export motion vectors through frame side data", 0,
> > > AV_OPT_TYPE_CONST, {.i64 = AV_CODEC_FLAG2_EXPORT_MVS}, INT_MIN,
> INT_MAX,
> > > V|D, "flags2"},
> > > +{"export_blocks", "export block types through frame side data", 0,
> > > AV_OPT_TYPE_CONST, {.i64 = AV_CODEC_FLAG2_EXPORT_BLOCKS}, INT_MIN,
> INT_MAX,
> > > V|D, "flags2"},
> > >  {"skip_manual", "do not skip samples and export skip information as
> frame
> > > side data", 0, AV_OPT_TYPE_CONST, {.i64 = AV_CODEC_FLAG2_SKIP_MANUAL},
> > > INT_MIN, INT_MAX, V|D, "flags2"},
> > >  {"ass_ro_flush_noop", "do not reset ASS ReadOrder field on flush", 0,
> > > AV_OPT_TYPE_CONST, {.i64 = AV_CODEC_FLAG2_RO_FLUSH_NOOP}, INT_MIN,
> INT_MAX,
> > > S|D, "flags2"},
> > >  {"time_base", NULL, OFFSET(time_base), AV_OPT_TYPE_RATIONAL, {.dbl =
> 0},
> > > 0, INT_MAX},
> > > diff --git a/libavutil/block_type.h b/libavutil/block_type.h
> > > new file mode 100644
> > > index 0000000000..3f53f1d082
> > > --- /dev/null
> > > +++ b/libavutil/block_type.h
> > > @@ -0,0 +1,107 @@
> > > +/*
> > > + * This file is part of FFmpeg.
> > > + *
> > > + * FFmpeg is free software; you can redistribute it and/or
> > > + * modify it under the terms of the GNU Lesser General Public
> > > + * License as published by the Free Software Foundation; either
> > > + * version 2.1 of the License, or (at your option) any later version.
> > > + *
> > > + * FFmpeg is distributed in the hope that it will be useful,
> > > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> > > + * Lesser General Public License for more details.
> > > + *
> > > + * You should have received a copy of the GNU Lesser General Public
> > > + * License along with FFmpeg; if not, write to the Free Software
> > > + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA
> > > 02110-1301 USA
> > > + */
> > > +
> > > +#ifndef AVUTIL_BLOCK_TYPE_H
> > > +#define AVUTIL_BLOCK_TYPE_H
> > > +
> > > +#include <stdint.h>
> > > +
> > > +
> > > +typedef struct AVBlockType {
> > > +    /**
> > > +     * Block type.
> > > +     * 1: Prediction (this can be spatial prediction or motion
> > > compensation for example)
> > > +     * 2: Transform
> > > +     * 3: Residual
> > > +     * 4: Filter
> > > +     * 5: Metadata
> > > +     *
> > > +     * Multiple Filter, Transform and prediction blocks are allowed,
> for
> > > example
> > > +     * for bidirectional motion compensation. Multiple residuals are
> > > allowed, for
> > > +     * example DC and AC residual.
> > > +     */
> > > +    uint8_t type;
> > > +    /**
> > > +     * Bitmask that lists which planes (for example:
> > > Y:1,Cb:2,Cr:4,Alpha:8)
> > > +     * this block applies to.
> > > +     */
> > > +    uint8_t plane_mask;
> > > +    /**
> > > +     * The field (top:1, bottom:2) this block applies to.
> > > +     */
> > > +    uint8_t field_mask;
> > > +    uint8_t name[16];
> > > +    /**
> > > +     * Left Top corner position. This can be outside of the vissible
> > > frame.
> > > +     */
> > > +    int32_t x, y;
> > > +    /**
> > > +     * Width and height of the block.
> > > +     */
> > > +    uint16_t w, h;
> > > +    /**
> > > +     * Number identifying the slice the Block is in.
> > > +     */
> > > +    uint16_t slice_num;
> > > +    /**
> > > +     * Location in bits where the block related information is stored.
> > > can be -1
> > > +     */
> > > +    uint32_t block_index, block_bits;
> > > +
> > > +    /**
> > > +     * Main Direction, 0 is horizontal, 48 is vertical,
> > > +     * values 0 to 95 follow clockwise direction.
> > > +     * 255 means that the block has no direction.
> > > +     * This represents the main direction of a transform, prediction
> or
> > > filter
> > > +     */
> > > +    uint8_t direction;
> > > +
> > > +    /**
> > > +     * Quantization parameter, a value of 0x8000 means lossless,
> 0x7FFF
> > > means not applicable
> > > +     * for MC prediction this is the precission of the motion vector.
> > > +     */
> > > +    int16_t qp;
> > > +
> > >
> >
> > Make 0 lossless.
>
> This would conflict with a codec that uses 0 for something else


The fundamental issue with both is that it's derived from codec behaviour.
Michael is trying to merge a special "lossless" bit in h264/hevc into
quantizer. Rostislav is trying to reproduce the av1/vp9 behaviour where 0
*is* lossless.

Neither is correct or translates well beyond itself.

If you add a transform type and have a special for "passthrough" (or vp9:
dwt), then that becomes lossless and the quantizer can be ignored for those
cases.

Ronald


More information about the ffmpeg-devel mailing list