[FFmpeg-user] compiling ffmpeg for nvidia gpu under Debian Bullseye

[Steve Newcomb]

I'm hoping to encode h265 streams using my new nvidia glx1650-super.  As 
a Debian devotee, the first thing I learned was that the drivers aren't 
available in the Debian 10 ("stable") repository, so I did a fresh 
install of Debian Bullseye ("testing").  That done, I attempted to 
compile ffmpeg, following the ffmpeg compilation instructions provided 
at 
https://docs.nvidia.com/video-technologies/video-codec-sdk/ffmpeg-with-nvidia-gpu/ 
<https://docs.nvidia.com/video-technologies/video-codec-sdk/ffmpeg-with-nvidia-gpu/>. 
I succeeded eventually, after encountering problems.  Here are the 
instructions provided at the aforementioned link, but with added 
commentary about what happened and how I eventually succeeded.  (I did 
everything as superuser, so you won't see any "sudo" below.)

git clone https://git.videolan.org/git/ffmpeg/nv-codec-headers.git
No problem here.

cd nv-codec-headers && sudo make install && cd –
This worked, but the final invocation of cd was uninterpretable due to 
the last character's being garbled (by my browser? or by Emacs?).  I 
followed this incantation with a "cd ..", which turned out to be the 
right thing to do.

git clone https://git.ffmpeg.org/ffmpeg.git ffmpeg/
No problem here.

apt-get install build-essential yasm cmake libtool libc6 libc6-dev unzip wget libnuma1 libnuma-dev
This turned out not to be enough packages.  I ended up installing the following packages, as well, even if some of them may not be strictly necessary.
I always install lots of utility packages which don't appear on this list, so this list, too, may not be complete:
nvidia-detect
nvidia-driver
libnvidia-encode1
libnppc11
libnppial11
libnppicc11
libnppidei11
libnppif11
libnppig11
libnppim11
libnppist11
libnppisu11
libnppitc11
libnpps11
nvidia-cg-toolkit
nvidia-cuda-gdb
nvidia-cuda-toolkit
nvidia-cuda-toolkit-gcc
nvtop

./configure --enable-nonfree -–enable-cuda-sdk –enable-libnpp 
--extra-cflags=-I/usr/local/cuda/include 
--extra-ldflags=-L/usr/local/cuda/lib64
This incantation had 2 problems.

(1) --enable-cuda-sdk is an obsolete argument.  I changed it to --enable-cuda-nvcc

(2) The configure failed with the report: "ERROR: failed checking for nvcc."  Why?
     The configure script checks nvcc by passing itan obsolete architecture code, "compute_30", which modern nvcc does not 
support.  <https://stackoverflow.com/questions/64774548/unsupported-gpu-architecture-compute-30-on-a-cuda-5-capable-gpu>   I edited the part of the configure script that says:
if enabled cuda_nvcc; then
     nvcc_default="nvcc"
     nvccflags_default="-gencode arch=compute_30,code=sm_30 -O2"
else
     nvcc_default="clang"
     nvccflags_default="--cuda-gpu-arch=sm_30 -O2"
     NVCC_C=""
fi
     ...so it instead said:
if enabled cuda_nvcc; then
     nvcc_default="nvcc"
     nvccflags_default="-gencode arch=compute_75,code=sm_75 -O2"
else
     nvcc_default="clang"
     nvccflags_default="--cuda-gpu-arch=sm_75 -O2"
     NVCC_C=""
fi
     ...although apparently it would have been fine to change "30" to "50" instead of "75" as I did.
     Perhaps ffmpeg developers should make this change, or do something similar.
That done, the configure incantation that actually worked was:
./configure --enable-nonfree --enable-cuda-nvcc --enable-libnpp --extra-cflags=-I/usr/local/cuda/include --extra-ldflags=-L/usr/local/cuda/lib64

make -j 8
No problem here.

make install
No problem here.