[FFmpeg-devel] [PATCH] avfilter: add v360_vulkan filter

Paul B Mahol onemda at gmail.com
Tue Jun 25 16:27:16 EEST 2024 

On Tue, Jun 25, 2024 at 3:17 PM Dennis Mungai <dmngaie at gmail.com> wrote:

> On Tue, 25 Jun 2024 at 16:07, Paul B Mahol <onemda at gmail.com> wrote:
>
> > On Tue, Jun 25, 2024 at 2:47 PM Dennis Mungai <dmngaie at gmail.com> wrote:
> >
> > > On Sun, 28 Jun 2020 at 16:59, James Almer <jamrial at gmail.com> wrote:
> > >
> > > > On 6/27/2020 7:54 AM, Paul B Mahol wrote:
> > > > > On 6/26/20, James Almer <jamrial at gmail.com> wrote:
> > > > >> On 6/26/2020 4:16 PM, Lynne wrote:
> > > > >>> Jun 26, 2020, 19:05 by jamrial at gmail.com:
> > > > >>>
> > > > >>>> On 6/26/2020 1:58 PM, Paul B Mahol wrote:
> > > > >>>>
> > > > >>>>> On 6/26/20, James Almer <jamrial at gmail.com> wrote:
> > > > >>>>>
> > > > >>>>>> On 6/26/2020 1:13 PM, Paul B Mahol wrote:
> > > > >>>>>>
> > > > >>>>>>> On 6/26/20, Paul B Mahol <onemda at gmail.com> wrote:
> > > > >>>>>>>
> > > > >>>>>>>> On 6/24/20, Paul B Mahol <onemda at gmail.com> wrote:
> > > > >>>>>>>>
> > > > >>>>>>>>> Signed-off-by: Paul B Mahol <onemda at gmail.com>
> > > > >>>>>>>>> ---
> > > > >>>>>>>>> Please test it and reports benchmark results on non toyish
> > > GPUs.
> > > > >>>>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>> Please, I really need it!
> > > > >>>>>>>>
> > > > >>>>>>>
> > > > >>>>>>>
> > > > >>>>>>> Why nobody wants to help?
> > > > >>>>>>>
> > > > >>>>>>> This makes me really really sad.
> > > > >>>>>>>
> > > > >>>>>>
> > > > >>>>>> I can try. Can you give an example command line with a
> publicly
> > > > >>>>>> available sample?
> > > > >>>>>>
> > > > >>>>>
> > > > >>>>> You can use any sample, samples are irrelevant. just report
> > > > resolution
> > > > >>>>> used.
> > > > >>>>> Also compare with CPU solution with just v360 filter.
> > > > >>>>>
> > > > >>>>> For yuv420p inputs:
> > > > >>>>>
> > > > >>>>> ffmpeg -init_hw_device vulkan=vulkan -i INPUT -filter_hw_device
> > > > vulkan
> > > > >>>>> -vf
> > > > >>>>>
> > > >
> > >
> >
> hwupload,v360_vulkan=fisheye:e:ih_fov=180:iv_fov=180:w=2048:h=1024,hwdownload,format=yuv420p
> > > > >>>>> -f null -
> > > > >>>>>
> > > > >>>>> ffmpeg -i INPUT -vf
> > > > v360=fisheye:e:ih_fov=180:iv_fov=180:w=2048:h=1024
> > > > >>>>> -f null -
> > > > >>>>>
> > > > >>>>> Just make sure that you get same output resolution in both
> cases.
> > > If
> > > > >>>>> not, adjust w/h accordingly.
> > > > >>>>>
> > > > >>>>
> > > > >>>> I'm getting a segfault on vkGetInstanceProcAddr, and i don't
> know
> > if
> > > > >>>> it's your filter or some issue in the driver. gdb is not being
> > > > helpful.
> > > > >>>> _______________________________________________
> > > > >>>> ffmpeg-devel mailing list
> > > > >>>> ffmpeg-devel at ffmpeg.org
> > > > >>>> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> > > > >>>>
> > > > >>>> To unsubscribe, visit link above, or email
> > > > >>>> ffmpeg-devel-request at ffmpeg.org>  with subject "unsubscribe".
> > > > >>>>
> > > > >>>
> > > > >>> Can you test again but replace EXT_EXTERNAL_HOST_MEMORY in
> > > > >>> optional_device_exts
> > > > >>> with EXT_NO_FLAG?
> > > > >>
> > > > >> Applying
> > > > >>
> > > > >>> diff --git a/libavutil/hwcontext_vulkan.c
> > > > b/libavutil/hwcontext_vulkan.c
> > > > >>> index 5e51d0390f..49fb7827ba 100644
> > > > >>> --- a/libavutil/hwcontext_vulkan.c
> > > > >>> +++ b/libavutil/hwcontext_vulkan.c
> > > > >>> @@ -228,7 +228,7 @@ static const VulkanOptExtension
> > > > optional_device_exts[]
> > > > >>> = {
> > > > >>>      { VK_EXT_EXTERNAL_MEMORY_DMA_BUF_EXTENSION_NAME,
> > > > >>> EXT_EXTERNAL_DMABUF_MEMORY, },
> > > > >>>      { VK_EXT_IMAGE_DRM_FORMAT_MODIFIER_EXTENSION_NAME,
> > > > >>> EXT_DRM_MODIFIER_FLAGS,     },
> > > > >>>      { VK_KHR_EXTERNAL_SEMAPHORE_FD_EXTENSION_NAME,
> > > > >>> EXT_EXTERNAL_FD_SEM,        },
> > > > >>> -    { VK_EXT_EXTERNAL_MEMORY_HOST_EXTENSION_NAME,
> > > > >>> EXT_EXTERNAL_HOST_MEMORY,   },
> > > > >>> +    { VK_EXT_EXTERNAL_MEMORY_HOST_EXTENSION_NAME,
> > > > >>> EXT_NO_FLAG,   },
> > > > >>>  };
> > > > >>>
> > > > >>>  /* Converts return values to strings */
> > > > >>
> > > > >> I still get a segfault:
> > > > >>
> > > > >>> Thread 1 received signal SIGSEGV, Segmentation fault.
> > > > >>> 0x00007ff8ed9274c0 in vkGetInstanceProcAddr ()
> > > > >>>    from
> > > > >>>
> > > >
> > >
> >
> C:\Windows\System32\DriverStore\FileRepository\nv_dispi.inf_amd64_ccad5caddc3a3d35\nvoglv64.dll
> > > > >>
> > > > >> Debug output shows
> > > > >>
> > > > >>> [AVHWDeviceContext @ 000001532a762800] GPU listing:
> > > > >>> [AVHWDeviceContext @ 000001532a762800]     0: GeForce GTX 1660
> > > > (discrete)
> > > > >>> (0x2184)
> > > > >>> [AVHWDeviceContext @ 000001532a762800] Using queue family 0
> (total
> > > > queues:
> > > > >>> 16) for graphics
> > > > >>> [AVHWDeviceContext @ 000001532a762800]     QF 0 flags: (graphics)
> > > > >>> (compute) (transfers) (sparse)
> > > > >>> [AVHWDeviceContext @ 000001532a762800] Using queue family 2
> (total
> > > > queues:
> > > > >>> 8) for compute
> > > > >>> [AVHWDeviceContext @ 000001532a762800]     QF 2 flags: (compute)
> > > > >>> (transfers) (sparse)
> > > > >>> [AVHWDeviceContext @ 000001532a762800] Using queue family 1
> (total
> > > > queues:
> > > > >>> 2) for transfers
> > > > >>> [AVHWDeviceContext @ 000001532a762800]     QF 1 flags:
> (transfers)
> > > > >>> (sparse)
> > > > >>> [AVHWDeviceContext @ 000001532a762800] Using device extension
> > > > >>> "VK_EXT_external_memory_host"
> > > > >>> [AVHWDeviceContext @ 000001532a762800] Using device extension
> > > > >>> VK_EXT_external_memory_host
> > > > >>> [AVHWDeviceContext @ 000001532a762800] Using device: GeForce GTX
> > 1660
> > > > >>> [AVHWDeviceContext @ 000001532a762800] Alignments:
> > > > >>> [AVHWDeviceContext @ 000001532a762800]
> > > > >>> optimalBufferCopyRowPitchAlignment: 1
> > > > >>> [AVHWDeviceContext @ 000001532a762800]     minMemoryMapAlignment:
> > > > >>>     64
> > > > >>
> > > > >> And crashes right after printing:
> > > > >>
> > > > >>> [Parsed_v360_vulkan_1 @ 000001532abd5c80] Shader v360_compute
> > linked!
> > > > >>> Size: 5508 bytes
> > > > >>
> > > > >> I'm using the latest driver, 451.48, on Windows 10 2004.
> > > > >
> > > > > So other vulkan filters, like vulkan_chromaber works for you?
> > > >
> > > > All fail the same way in vkGetInstanceProcAddr, so the issue is in
> > > > hwcontext.
> > > >
> > > > Lynne should be able to test/bench this filter.
> > > >
> > >
> > >  Hello Paul, et al,
> > >
> > > Any updates on this filter?
> > >
> >
> > I'm no longer working on FFmpeg.
> >
> > The v360_opencl filter got in more useful state but due complete lack of
> > interest in it from users, I forgot about its existence.
> > The v360_vulkan filter is probably in worse shape than v360_opencl filter
> > and 0/negative interest, and I forgot about its existence too.
> >
>
> Is there an FFmpeg tree with v360_opencl (or a similar filter) out there?
>


I can give you patches/files for both v360 vulkan and opencl filter if you
want to hack on it.
But I will not adapt them for free to current library versions, nor provide
any kind of free support for it.



> A GPU-accelerated v360 filter is definitely useful to test and implement.
> Definitely more useful than the DNN stack in FFmpeg (by leaps and bounds)
> as far as applicability is concerned.
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".
>

More information about the ffmpeg-devel mailing list