[FFmpeg-user] Why is format=rgb24 required after maskedmerge?

[Paul B Mahol]

On 8/19/20, Michael Koch <astroelectronic at t-online.de> wrote:
> Am 19.08.2020 um 16:34 schrieb Paul B Mahol:
>> On 8/19/20, Michael Koch <astroelectronic at t-online.de> wrote:
>>> Am 18.08.2020 um 14:35 schrieb Michael Koch:
>>>> Hello all,
>>>>
>>>> I have a question about this script:
>>>>
>>>> rem  Create red and yellow videos 300x300
>>>>
>>>> c:\ffmpeg\ffmpeg -f lavfi -i color=red:size=300x300:duration=5 -y
>>>> red.mp4
>>>> c:\ffmpeg\ffmpeg -f lavfi -i color=yellow:size=300x300:duration=5 -y
>>>> yellow.mp4
>>>>
>>>> rem  Create a mergemap file 600x300
>>>>
>>>> c:\ffmpeg\ffmpeg -f lavfi -i nullsrc=size=300x300 -vf
>>>> "format=gray8,geq='clip(128-128/10*(180-200/150*hypot(X-150,Y-150)),0,255)',v360=input=fisheye:output=e:ih_fov=200:iv_fov=200,format=rgb24"
>>>>
>>>> -frames 1 -y mergemap.png
>>>>
>>>> rem  Stitch two fisheye videos together to an equirectangular video,
>>>> with merging at the border
>>>>
>>>> c:\ffmpeg\ffmpeg -i red.mp4 -i yellow.mp4 -i mergemap.png -lavfi
>>>> "[0]format=rgb24,v360=input=fisheye:output=e:ih_fov=200:iv_fov=200[a];[1]format=rgb24,v360=input=fisheye:output=e:yaw=180:ih_fov=200:iv_fov=200[b];[a][b][2]maskedmerge,format=rgb24,format=yuv422p"
>>>>
>>>> -y out.mp4
>>>>
>>>>
>>>> This works fine and the output video looks as expected. My question is
>>>> why "format=rgb24" is required between "maskedmerge" and
>>>> "format=yuv422p". Normally two consecutive format conversions should
>>>> be unnecessary. But it doesn't work when I omit "format=rgb24". The
>>>> colors become wrong.
>>> It seems that v360 and maskedmerge change the pixel format from rgb24 to
>>> the planar gbrp pixel format.
>>> But why isn't it possible to convert gbrp directly to yuv422p, without
>>> an intermediate conversion to rgb24? Is this a bug?
>>>
>>> I suggest to add to the documentation that v360 and maskedmerge (and
>>> possibly also other filters) change the pixel format to gbrp. As a user
>>> I was assuming that the pixel format remains the same, unless noted
>>> otherwise in the documentation.
>> You are deeply confused about our filters.
>> Any filter can change pixel formats to one that they accepts thus gbrp
>> is picked instead of packed rgb, this is already documented
>> implicitly.
>
> It would be very helpful to find in the documentation for each filter a
> list of the accepted input pixel formats, and it's also a useful
> information if a filter is capable of changing the pixel format. Or can
> all filters change the pixel format?
>
>> Your mergemap is produced in rgb24 pixel format and it only work with
>> rgb pixel formats otherwise you get wrong output from maskedmerge.
>
> OK, understood. I converted the mergemap to gbrp and now it works fine.
> In my opinion the whole thing is badly documented.

If input is in yuv422p that you want to be filtered than mergemap
needs to be in yuv422p too.

> On one hand, the filter does automatically change the first two inputs
> to a format that it accepts, in this case gbrp.
> On the other hand, the filter is unable to automatically change the
> third input to the same pixel format, which is the only pixel format
> that works.
>
> Do you agree that this is the correct workflow?
> 1. Insert a showinfo filter after maskedmerge and check the pixel format.

Just use -v debug, it will show whenever scale/format filter is auto inserted.
It may be to spammy with unrelated messages from other components.

There is patch in preparation to disable all auto conversions in filtergraph,
than you are on your own to fix any issues if you enable such flag.

> 2. Insert a format conversion in the filter chain that converts the
> mergemap to the same pixel format as found in step 1.

That is useful only when you disable auto conversion with flag that
gonna be added (I hope soon) to lavfi.

>
> Michael
>
> _______________________________________________
> ffmpeg-user mailing list
> ffmpeg-user at ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-user
>
> To unsubscribe, visit link above, or email
> ffmpeg-user-request at ffmpeg.org with subject "unsubscribe".