[FFmpeg-trac] #11371(avutil:new): Document lock ordering for hwcontext_vulkan.h

[FFmpeg]

#11371: Document lock ordering for hwcontext_vulkan.h
-------------------------------------+-------------------------------------
             Reporter:  mstoeckl     |                     Type:
                                     |  enhancement
               Status:  new          |                 Priority:  normal
            Component:  avutil       |                  Version:  git-
             Keywords:               |  master
  documentation, vulkan, deadlock    |               Blocked By:
             Blocking:               |  Reproduced by developer:  0
Analyzed by developer:  0            |
-------------------------------------+-------------------------------------
 There are two types of locks exposed in libavutil/hwcontext_vulkan.h: a
 lock for exclusive access to a Vulkan queue,
 AVVulkanDeviceContext.lock_queue, and a lock for exclusive access to a
 frame, AVVulkanFramesContext.lock_frame. As far as I can tell, the
 documentation does not clearly say in which order such locks should be
 taken, but I believe it should.

 I currently have code (for `waypipe`'s video encoding) that locks an
 AVVkFrame, builds a command buffer to fill its contents and updates the
 AVVkFrame, locks a Vulkan queue, submits the command buffer, and then
 promptly unlocks everything.

 However, if ffmpeg ever has a thread operate in the reverse order (lock a
 queue, and then locks a frame), then a deadlock could be possible if my
 code locks frame A, ffmpeg locks queue B, and then my code tries to lock
 queue B while ffmpeg tries to lock frame A for something.

 Holding both queue and frame locks simultaneously appears necessary. The
 AVVulkanFramesContext.lock_frame documentation, currently implies that at
 command buffer submission time the frame lock should be held: "Users
 SHOULD only ever lock just before command submission in order get accurate
 frame properties, and unlock immediately after command submission without
 waiting for it to finish." On the other hand, Vulkan's documentation marks
 vkQueueSubmit as "externally synchronized", so it is unsafe to submit
 command buffers without holding a queue lock.
-- 
Ticket URL: <https://trac.ffmpeg.org/ticket/11371>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker