[FFmpeg-devel] [PATCH 3/8] lavu: add a Vulkan hwcontext

[Mark Thompson]

On 22/04/18 17:28, Rostislav Pehlivanov wrote:
> On 22 April 2018 at 17:21, Rostislav Pehlivanov <atomnuker at gmail.com> wrote:
>> On 22 April 2018 at 12:46, Mark Thompson <sw at jkqxz.net> wrote:
>>> On 21/04/18 23:29, Carl Eugen Hoyos wrote:
>>>> 2018-04-21 23:33 GMT+02:00, Rostislav Pehlivanov <atomnuker at gmail.com>:
>>>>> On 21 April 2018 at 21:24, Carl Eugen Hoyos <ceffmpeg at gmail.com>
>>> wrote:
>>>>>
>>>>>> 2018-04-20 6:30 GMT+02:00, Rostislav Pehlivanov <atomnuker at gmail.com
>>>> :
>>>>>>
>>>>>>> +    [AV_PIX_FMT_P010]      =
>>>>>>> VK_FORMAT_G10X6_B10X6R10X6_2PLANE_420_UNORM_3PACK16,
>>>>>>
>>>>>>> +    [AV_PIX_FMT_YUV420P10] =
>>>>>>> VK_FORMAT_G10X6_B10X6_R10X6_3PLANE_420_UNORM_3PACK16,
>>>>>>
>>>>>> I don't think both can be correct (unless "PACK16" has no meaning).
>>>>
>>>>> They're both correct and work.
>>>>
>>>> That's really strange...
>>>> (Could this be a bug in the driver?)
>>>
>>> Sounds like it must be a bug somewhere.
>>>
>>> The Vulkan specification says:
>>>
>>> """
>>> VK_FORMAT_G10X6_B10X6R10X6_2PLANE_420_UNORM_3PACK16 specifies a unsigned
>>> normalized multi-planar format that has a 10-bit G component in the top 10
>>> bits of each 16-bit word of plane 0, and a two-component, 32-bit BR plane 1
>>> consisting of a 10-bit B component in the top 10 bits of the word in bytes
>>> 0..1, and a 10-bit R component in the top 10 bits of the word in bytes
>>> 2..3, the bottom 6 bits of each word set to 0.
>>>
>>> VK_FORMAT_G10X6_B10X6_R10X6_3PLANE_420_UNORM_3PACK16 specifies a
>>> unsigned normalized multi-planar format that has a 10-bit G component in
>>> the top 10 bits of each 16-bit word of plane 0, a 10-bit B component in the
>>> top 10 bits of each 16-bit word of plane 1, and a 10-bit R component in the
>>> top 10 bits of each 16-bit word of plane 2, with the bottom 6 bits of each
>>> word set to 0.
>>> """
>>>
>>> Which I think makes it pretty clear that VK_FORMAT_G10X6_B10X6R10X6_2PLANE_420_UNORM_3PACK16
>>> is indeed P010 but VK_FORMAT_G10X6_B10X6_R10X6_3PLANE_420_UNORM_3PACK16
>>> isn't YUV420P10 because they pack the 10 bits at different ends of the
>>> 16-bit value.  If a driver is getting that wrong then it should be reported
>>> to the vendor.
>>>
>>> I don't see any formats at all in the Vulkan specification which put the
>>> value at the low end of a containing word, but I might not be looking in
>>> the right place?
>>>
>>> - Mark
>>>
>>>
>>> (Vaguely related, because it made me look it up, it appears that the
>>> device will always match host-endianness:
>>>
>>> After talking about the numeric types,
>>> """
>>> The representation and endianness of these types on the host must match
>>> the representation and endianness of the same types on every physical
>>> device supported."
>>> """
>>>
>>> I don't know what that actually means for little-endian graphics cards
>>> (e.g. AMD/Nvidia) in big-endian machines (e.g. POWER) - maybe Vulkan just
>>> doesn't support that, or maybe the driver can fix it up somehow - but we
>>> don't need to think about it at all.)
>>
>> Something's weird:
>>
>>
> Sorry, pushed the wrong button.
> 
> For this filter chain:
> format=<SRC_FORMAT>,hwupload,scale_vulkan=w=1024:h=-1:format=rgba,hwdownload,format=rgba
> 
> This is what happens for each SRC_FORMAT:
> NVIDIA 960M with binary drivers:
> p010 - works fine
> yuv420p10le - mostly green screen with some minor variations, enough to
> make out the original video
> yuv420p10be - works fine
> 
> Intel 530:
> p010 - works fine
> yuv420p10le - works fine
> yuv420p10be - works fine
> 
> I'm not entirely sure what to make of that. How does the intel deal with
> formats with different endianess when there's no way to indicate endianess
> at all? Why does nvidia deal with big endian when you said its little
> endian?

hwupload checks the supported formats with get_constraints and only exposes the supported ones to lavfi query_formats.  Probably some auto-conversion is happening for the big-endian formats?  Maybe on Intel the three-plane format also isn't supported, and so auto-conversion happens there too?

I think the green screen is what we would expect from the above analysis, since all you would be getting is the high 4 bits of each component in the low 4 bits of the output.

- Mark