[FFmpeg-devel] [PATCH] avfilter: add panorama filter

[Paul B Mahol]

On 3/9/18, James Almer <jamrial at gmail.com> wrote:
> On 3/9/2018 12:12 PM, James Almer wrote:
>> On 3/9/2018 12:04 PM, Paul B Mahol wrote:
>>> On 3/9/18, James Almer <jamrial at gmail.com> wrote:
>>>> On 3/9/2018 7:05 AM, Paul B Mahol wrote:
>>>>> On 3/9/18, Paul B Mahol <onemda at gmail.com> wrote:
>>>>>> On 3/9/18, wm4 <nfxjfg at googlemail.com> wrote:
>>>>>>> On Fri, 9 Mar 2018 09:15:13 +0100
>>>>>>> Paul B Mahol <onemda at gmail.com> wrote:
>>>>>>>
>>>>>>>> On 3/9/18, wm4 <nfxjfg at googlemail.com> wrote:
>>>>>>>>> On Thu, 8 Mar 2018 21:53:48 -0300
>>>>>>>>> James Almer <jamrial at gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> On 3/8/2018 9:50 PM, Hazem Ashmawy wrote:
>>>>>>>>>>> [PATCH] avfilter: add panorama filter
>>>>>>>>>>>
>>>>>>>>>>> Sorry about that! I removed them now.
>>>>>>>>>>> For the future, any recommendation for a tool for linting /
>>>>>>>>>>> checking
>>>>>>>>>>> formating
>>>>>>>>>>> rules?
>>>>>>>>>>
>>>>>>>>>> There's tools/patcheck. Feed it a git format-patch style of patch
>>>>>>>>>> to
>>>>>>>>>> find common issues, but keep in mind it can generate a lot of
>>>>>>>>>> false
>>>>>>>>>> positives.
>>>>>>>>>>
>>>>>>>>>> I don't know if we have documentation about actual formatting
>>>>>>>>>> rules
>>>>>>>>>> anywhere.
>>>>>>>>>
>>>>>>>>> Also:
>>>>>>>>>
>>>>>>>>> <_jamrial> shouldn't that panorama filter sent to the ml use the
>>>>>>>>> spherical
>>>>>>>>> frame side data?
>>>>>>>>>
>>>>>>>>> I think so.
>>>>>>>>
>>>>>>>> Are there actual files that have such data?
>>>>>>>
>>>>>>> Is that a trick question? I only know the non-standard, Google
>>>>>>> specific
>>>>>>> metadata in mkv and mp4 that lavf can read (was any of this
>>>>>>> standardized yet?).
>>>>>>>
>>>>>>> But that doesn't change that we can tag AVFrames with this info, and
>>>>>>> for files which don't have the metadata, it makes sense to me to set
>>>>>>> it
>>>>>>> with a new vf_format argument or some sort of vf_setinfo (if we don't
>>>>>>> have anything like this yet).
>>>>>>>
>>>>>>> The part that is annoying is that vf_panorama still seems to require
>>>>>>> setting an output projection, which would make the whole thing more
>>>>>>> annoying instead of less, but even then I'd argue it should default
>>>>>>> to
>>>>>>> taking the AVFrame configuration (AV_FRAME_DATA_SPHERICAL) as input
>>>>>>> by
>>>>>>> default, even if the filter arguments can override it.
>>>>>>
>>>>>> That frame side data is very specific and thus considered barely
>>>>>> useful
>>>>>> here.
>>>>>>
>>>>>> Is it at all updated to the latest improvements, like new equi-angular
>>>>>> cubemap projection?
>>>>>>
>>>>>
>>>>> I guess not at all, I get this:
>>>>>
>>>>> [mov,mp4,m4a,3gp,3g2,mj2 @ 0x21c1740] Unknown projection type
>>>>
>>>> Sample? Also, where is this new projection defined?
>>>
>>> https://www.youtube.com/watch?v=qhLExhpXX0E
>>>
>>> It is defined by Google?
>>
>> Vittorio and I used
>> https://github.com/google/spatial-media/blob/master/docs/spherical-video-v2-rfc.md
>> to write the current Matroska and mov implementations, and by extension
>> the AVSphericalMapping API. Specifically the Equirectangular and Cubemap
>> projections.
>>
>> This sample seems to have a "ytmp" projection box, but it's not defined
>> in the above document. I guess the name hints at it being a very early
>> an internal draft? We can't really do much without a spec...
>
> The h264/mp4 version has this unknown "ytmp" box, but the vp9/webm one
> has a ProjectionType element with a value of 3, plus some binary data in
> the ProjectionPrivate element, which according to the spec means a Mesh
> projection. We don't currently support that one.
>
> Weird, seeing mp4 supposedly has the "mshp" box for this.

Do you know of any video sample that have "standard" cubemap layout?