Except for a little problem, that I encounter both on my personal machine and the research server. I would not consider myself a very experienced admin for such systems as I come from a research background, I learned to use and understand Linux over the past months and so far, everything we needed for our small team worked like a charm. Before running this installer I always blacklist the Noveau driver as shown here for example.

For this, on both machines, I used the runfile installer from the official Nvidia page here for CUDA which contains the Nvidia driver. I was the one setting up both machines from scratch, so I did a fresh Ubuntu 20.04.3 LTS install on both machines, update, upgrade, installing basic tools, installing the Nvidia driver + CUDA. We are doing deep learning research at work, so I use the machines mostly for running TensorFlow or related tools that make use of the GPU. The personal machine has an RTX2080 Super while the server runs with two RTX3090s. I am using Ubuntu 20.04.3 LTS on two machines (my personal computer and a small server from work), both equipped with Nvidia cards.

Ubuntu 20.04 nvidia driver