[FFmpeg-devel] [PATCH 4/7] hwcontext_vulkan: add support for AV_PIX_FMT_RGBA32

[Lynne]

On 12/10/2024 00:11, James Almer wrote:
> On 10/10/2024 2:32 AM, Lynne via ffmpeg-devel wrote:
>> ---
>>   libavutil/hwcontext_vulkan.c |  1 +
>>   libavutil/vulkan.c           | 11 ++++++++++-
>>   2 files changed, 11 insertions(+), 1 deletion(-)
>>
>> diff --git a/libavutil/hwcontext_vulkan.c b/libavutil/hwcontext_vulkan.c
>> index 1fb0481fa7..24aafcba07 100644
>> --- a/libavutil/hwcontext_vulkan.c
>> +++ b/libavutil/hwcontext_vulkan.c
>> @@ -320,6 +320,7 @@ static const struct FFVkFormatEntry {
>>       { VK_FORMAT_R8G8B8A8_UNORM,           AV_PIX_FMT_RGB0,    
>> VK_IMAGE_ASPECT_COLOR_BIT, 1, 1, 1, 
>> { VK_FORMAT_R8G8B8A8_UNORM           } },
>>       { VK_FORMAT_A2R10G10B10_UNORM_PACK32, AV_PIX_FMT_X2RGB10, 
>> VK_IMAGE_ASPECT_COLOR_BIT, 1, 1, 1, 
>> { VK_FORMAT_A2R10G10B10_UNORM_PACK32 } },
>>       { VK_FORMAT_A2B10G10R10_UNORM_PACK32, AV_PIX_FMT_X2BGR10, 
>> VK_IMAGE_ASPECT_COLOR_BIT, 1, 1, 1, 
>> { VK_FORMAT_A2B10G10R10_UNORM_PACK32 } },
>> +    { VK_FORMAT_R32G32B32A32_UINT,        AV_PIX_FMT_RGBA32,  
>> VK_IMAGE_ASPECT_COLOR_BIT, 1, 1, 1, 
>> { VK_FORMAT_R32G32B32A32_UINT        } },
>>       { VK_FORMAT_R32G32B32A32_SFLOAT,      AV_PIX_FMT_RGBAF32, 
>> VK_IMAGE_ASPECT_COLOR_BIT, 1, 1, 1, 
>> { VK_FORMAT_R32G32B32A32_SFLOAT      } },
>>       /* Planar RGB */
>> diff --git a/libavutil/vulkan.c b/libavutil/vulkan.c
>> index 4cd180f54c..1ad3b3feb5 100644
>> --- a/libavutil/vulkan.c
>> +++ b/libavutil/vulkan.c
>> @@ -1282,7 +1282,7 @@ int ff_vk_mt_is_np_rgb(enum AVPixelFormat pix_fmt)
>>           pix_fmt == AV_PIX_FMT_BGR565 || pix_fmt == AV_PIX_FMT_BGR0   ||
>>           pix_fmt == AV_PIX_FMT_0BGR   || pix_fmt == AV_PIX_FMT_RGB0   ||
>>           pix_fmt == AV_PIX_FMT_X2RGB10 || pix_fmt == 
>> AV_PIX_FMT_X2BGR10 ||
>> -        pix_fmt == AV_PIX_FMT_RGBAF32)
>> +        pix_fmt == AV_PIX_FMT_RGBAF32 || pix_fmt == AV_PIX_FMT_RGBA32)
>>           return 1;
>>       return 0;
>>   }
>> @@ -1366,6 +1366,15 @@ const char *ff_vk_shader_rep_fmt(enum 
>> AVPixelFormat pix_fmt,
>>           };
>>           return rep_tab[rep_fmt];
>>       }
>> +    case AV_PIX_FMT_RGBA32: {
>> +        const char *rep_tab[] = {
>> +            [FF_VK_REP_NATIVE] = "rgba32ui",
>> +            [FF_VK_REP_FLOAT] = NULL,
>> +            [FF_VK_REP_INT] = "rgba32i",
>> +            [FF_VK_REP_UINT] = "rgba32ui",
>> +        };
>> +        return rep_tab[rep_fmt];
>> +    };
>>       case AV_PIX_FMT_GRAY8:
>>       case AV_PIX_FMT_GBRAP: {
>>           const char *rep_tab[] = {
> 
> What does this accomplish if you don't add input and/or output support 
> to libsws? Assuming Vulkan driver outputs this, nothing but 
> av_read_image_line() and av_write_image_line() will be able to do 
> anything with it.

Me and the folks working on the VC-2 encoder/decoder need the format so 
we can use the standard AVHWFramesContext interface to allocate internal 
intermediate frames for lossless codecs. Currently we're using buffers,
but they're slower and more error-prone than using images.
Its possible to duplicate all the code from hwcontext_vulkan.c, but 
Vulkan doesn't make it easy, plus the code there is tested.

The plan is to add swscale support with a later patch. The format can be 
useful in other situations too, such as CGI intermediates, since it can 
fully hold 16-bit convolutions (16*16->32), such as what you get when 
tonemapping. Also TIFF supports it (though TIFF supports everything).
-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_0xA2FEA5F03F034464.asc
Type: application/pgp-keys
Size: 624 bytes
Desc: OpenPGP public key
URL: <https://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20241012/6acb8781/attachment.key>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature.asc
Type: application/pgp-signature
Size: 236 bytes
Desc: OpenPGP digital signature
URL: <https://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20241012/6acb8781/attachment.sig>