[FFmpeg-devel] [PATCH v3] libavfilter: add a gblur_vulkan filter
Lynne
dev at lynne.ee
Sat Nov 6 05:05:38 EET 2021
9 Sept 2021, 07:44 by jianhua.wu at intel.com:
> This commit adds a powerful and customizable gblur Vulkan filter,
> which provides a maximum 127x127 kernel size of Gaussian Filter.
> The size could be adjusted by requirements on quality or performance.
>
> The following command is on how to apply gblur_vulkan filter:
>
> ffmpeg -init_hw_device vulkan=vul:0 -filter_hw_device vul -i input.264
> -vf hwupload=extra_hw_frames=16,gblur_vulkan,hwdownload,format=yuv420p
> output.264
>
> Signed-off-by: Wu Jianhua <jianhua.wu at intel.com>
> ---
> configure | 1 +
> libavfilter/Makefile | 1 +
> libavfilter/allfilters.c | 1 +
> libavfilter/vf_gblur_vulkan.c | 511 ++++++++++++++++++++++++++++++++++
> 4 files changed, 514 insertions(+)
> create mode 100644 libavfilter/vf_gblur_vulkan.c
>
> diff --git a/configure b/configure
> index af410a9d11..4b9a0d8e07 100755
> --- a/configure
> +++ b/configure
> @@ -3601,6 +3601,7 @@ freezedetect_filter_select="scene_sad"
> frei0r_filter_deps="frei0r libdl"
> frei0r_src_filter_deps="frei0r libdl"
> fspp_filter_deps="gpl"
> +gblur_vulkan_filter_deps="vulkan_lib libglslang"
> histeq_filter_deps="gpl"
> hqdn3d_filter_deps="gpl"
> interlace_filter_deps="gpl"
> diff --git a/libavfilter/Makefile b/libavfilter/Makefile
> index af957a5ac0..5f74e33599 100644
> --- a/libavfilter/Makefile
> +++ b/libavfilter/Makefile
> @@ -286,6 +286,7 @@ OBJS-$(CONFIG_FREEZEFRAMES_FILTER) += vf_freezeframes.o
> OBJS-$(CONFIG_FREI0R_FILTER) += vf_frei0r.o
> OBJS-$(CONFIG_FSPP_FILTER) += vf_fspp.o qp_table.o
> OBJS-$(CONFIG_GBLUR_FILTER) += vf_gblur.o
> +OBJS-$(CONFIG_GBLUR_VULKAN_FILTER) += vf_gblur_vulkan.o vulkan.o
> OBJS-$(CONFIG_GEQ_FILTER) += vf_geq.o
> OBJS-$(CONFIG_GRADFUN_FILTER) += vf_gradfun.o
> OBJS-$(CONFIG_GRAPHMONITOR_FILTER) += f_graphmonitor.o
> diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c
> index 0c6b2347c8..b5576de3af 100644
> --- a/libavfilter/allfilters.c
> +++ b/libavfilter/allfilters.c
> @@ -271,6 +271,7 @@ extern const AVFilter ff_vf_freezeframes;
> extern const AVFilter ff_vf_frei0r;
> extern const AVFilter ff_vf_fspp;
> extern const AVFilter ff_vf_gblur;
> +extern const AVFilter ff_vf_gblur_vulkan;
> extern const AVFilter ff_vf_geq;
> extern const AVFilter ff_vf_gradfun;
> extern const AVFilter ff_vf_graphmonitor;
> diff --git a/libavfilter/vf_gblur_vulkan.c b/libavfilter/vf_gblur_vulkan.c
> new file mode 100644
> index 0000000000..5c072f8971
> --- /dev/null
> +++ b/libavfilter/vf_gblur_vulkan.c
> @@ -0,0 +1,511 @@
> +/*
> + * copyright (c) 2021 Wu Jianhua <jianhua.wu at intel.com>
> + * This file is part of FFmpeg.
> + *
> + * FFmpeg is free software; you can redistribute it and/or
> + * modify it under the terms of the GNU Lesser General Public
> + * License as published by the Free Software Foundation; either
> + * version 2.1 of the License, or (at your option) any later version.
> + *
> + * FFmpeg is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
> + * Lesser General Public License for more details.
> + *
> + * You should have received a copy of the GNU Lesser General Public
> + * License along with FFmpeg; if not, write to the Free Software
> + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
> + */
> +
> +#include "libavutil/random_seed.h"
> +#include "libavutil/opt.h"
> +#include "vulkan.h"
> +#include "internal.h"
> +
> +#define CGS 32
> +#define GBLUR_MAX_KERNEL_SIZE 127
> +
> +typedef struct GBlurVulkanContext {
> + VulkanFilterContext vkctx;
> + FFVkExecContext *exec;
> + VulkanPipeline *pl_hor;
> + VulkanPipeline *pl_ver;
> + FFVkBuffer params_buf_hor;
> + FFVkBuffer params_buf_ver;
> +
> + VkDescriptorImageInfo input_images[3];
> + VkDescriptorImageInfo tmp_images[3];
> + VkDescriptorImageInfo output_images[3];
> + VkDescriptorBufferInfo params_desc_hor;
> + VkDescriptorBufferInfo params_desc_ver;
> +
> + int initialized;
> + int size;
> + int planes;
> + int kernel_size;
> + float sigma;
> + float sigmaV;
> + AVFrame *tmpframe;
> +} GBlurVulkanContext;
> +
> +static const char gblur_horizontal[] = {
> + C(0, void gblur(const ivec2 pos, const int index) )
> + C(0, { )
> + C(1, vec4 sum = texture(input_image[index], pos) * kernel[0]; )
> + C(0, )
> + C(1, for(int i = 1; i < kernel.length(); i++) { )
> + C(2, sum += texture(input_image[index], pos + vec2(i, 0.0)) * kernel[i]; )
> + C(2, sum += texture(input_image[index], pos - vec2(i, 0.0)) * kernel[i]; )
> + C(1, } )
> + C(0, )
> + C(1, imageStore(output_image[index], pos, sum); )
> + C(0, } )
> +};
> +
> +static const char gblur_vertical[] = {
> + C(0, void gblur(const ivec2 pos, const int index) )
> + C(0, { )
> + C(1, vec4 sum = texture(input_image[index], pos) * kernel[0]; )
> + C(0, )
> + C(1, for(int i = 1; i < kernel.length(); i++) { )
> + C(2, sum += texture(input_image[index], pos + vec2(0.0, i)) * kernel[i]; )
> + C(2, sum += texture(input_image[index], pos - vec2(0.0, i)) * kernel[i]; )
> + C(1, } )
> + C(0, )
> + C(1, imageStore(output_image[index], pos, sum); )
> + C(0, } )
> +};
>
The reason why avgblur_vulkan is split into horizontal and vertical
was because you can change the blur radius in either direction.
This is always going to be square, so to speed it up significantly,
you can just do it all at once in both directions.
By the way, I've written a replacement for the synchronization
system that uses timeline semaphores. The repo is at
https://github.com/cyanreg/FFmpeg/tree/vulkan
I'll be pushing it in a few days once someone reviews it.
You can adapt your patches to that.
I've CC'd Wenbin Chen here too. I think your single_memory
flag patch is reasonable, could you rebase it onto the branch and
resubmit?
More information about the ffmpeg-devel
mailing list