[FFmpeg-devel] [PATCH v3 3/7] avcodec/mediacodecenc: use bsf to handle crop

[Tomas Härdin]

ons 2022-12-14 klockan 18:08 +0100 skrev Tomas Härdin:
> tis 2022-12-13 klockan 11:20 +0800 skrev zhilizhao(赵志立):
> > 
> > 
> > > On Dec 12, 2022, at 23:27, Tomas Härdin <git at haerdin.se> wrote:
> > > 
> > > lör 2022-12-10 klockan 01:22 +0800 skrev Zhao Zhili:
> > > > From: Zhao Zhili <zhilizhao at tencent.com>
> > > > 
> > > > It's well known that mediacodec encoder requires 16x16
> > > > alignment.
> > > > Use our bsf to fix the crop info.
> > > > ---
> > > > v3: don't change the dimension for AV_PIX_FMT_MEDIACODEC. It
> > > > can
> > > > have
> > > > side effect.
> > > 
> > > Looks like this silently crops? Is that really a good idea? We
> > > usually
> > > don't do stuff like that. For example codecs that require even
> > > dimensions complain loudly then fail.
> > 
> > It’s reasonable to require even dimensions. Require dimensions
> > aligned
> > to 16 is uncommon. Everyone will complain why 1080x1920 doesn’t
> > work.
> > 
> > A lot of apps just use aligned dimensions. Users have no control on
> > these apps. It’s not the same with FFmpeg, users (developer or not)
> > can specify the dimension directly.
> 
> Wait a sec, I think I was misunderstanding what the code is doing.
> FFALIGN rounds *up*. Does this mean you insert fake data in the
> border
> that then gets cropped away, meaning the original essence is still
> "there"? That's a different thing and probably perfectly OK.
> 
> I think we might want something for this inside lavf somewhere, so
> that
> encoders can signal dimension alignment requirements. Some containers
> (MXF, MOV) support such cropping in a codec-agnostic manner.
> 
> > 
> > If we don’t fix it, either:
> > 
> > 1. Reject and fail directly. User complain why.
> > 2. Accept and keep going. Sometimes it works, sometimes don’t. It
> > depends on the device and get into a confused situation.
> > 
> > I know there are getWidthAlignment()/getHeightAlignment() to get
> > alignment info of codecs. The results are unreliable. The only
> > reliable way I can find is don’t depends on those API and fix it
> > by ourself.
> 
> Given how temperamental MC seems to be a "belt and braces" approach
> might be appropriate when dealing with it. Tell users (ffmpeg.c is a
> user here) that dimensions must be aligned by 16x16 and then
> automagically doing the required padding and cropping somewhere (lavf
> or ffmpeg.c) feels like a decent solution.

Come to think of it this kind of 16x16 requirement is very common and
is already being handled silently: it's the macroblock size for almost
every DCT codec when using 4:2:0 subsampling.

/Tomas