[FFmpeg-devel] Extending ffmpeg hwaccel list

[Andriy Gelman]

On Wed, 15. Apr 02:58, Mark Clayton wrote:
> Hello,
> 
> I have an Nvidia Jetson Nano, which while having CUDA does not feature an
> nvenc capable encoder.  Instead, a high level API using L4T (
> https://docs.nvidia.com/jetson/l4t-multimedia/classNvVideoConverter.html)
> is supported.
> 
> It’s been a while and it seems that this device is not popular enough to
> warrant an ffmpeg patch, so I’m inclined to write one myself.  There exists
> a working encode/decode patch (https://github.com/jocover/jetson-ffmpeg)
> not using the hwaccel headers.
> 
> Is writing atop a high level api (first link) appropriate for a hwaccel
> class, and if so where can I find documentation for how existing
> accelerators (vaapi, nvenc) are skeletoned?  I’m a decent C programmer but
> have not contributed to ffmpeg before.

Nvidia's api for the Nano is based on v4l2m2m  which we already support in
libavcodec/v4l2_m2m*   

In the past, I attempted to get the Nano working with FFmpeg but came across
several issues: 

- In addition to usual v4l2m2m workflow, the Nano requires calls to
  NvBufferMemSyncForCpu()/NvBufferMemSyncForDevice(). These are in a proprietary
  header which cannot be part of FFmpeg:    	
  https://docs.nvidia.com/metropolis/deepstream/dev-guide/DeepStream%20Development%20Guide/baggage/nvbuf__utils_8h_source.html

- All ioctl() calls need to be done via libv4l2... There are ways to get around
  this as is done in libavdevice/v4l2.c 

- I often saw segfaults on the Nano coming from their version of libv4l2.  Other users
  experienced this too:
  https://forums.developer.nvidia.com/t/vidioc-dqbuf-blocks/63414/10

In this forum, Nvidia's admin says that FFmpeg will be supported in r32.4...
https://forums.developer.nvidia.com/t/building-ffmpeg-for-jetson-error-cuda-requested-but-not-all-dependencies-are-satisfied-ffnvcodec/118260

No idea whether this support is based on v4l2m2m... But I would be interested to
see their sources :)

-- 
Andriy