Post

Proxmox 7.2 - GPU Passthru

If you have a compatible nVidia graphics card, you can build your own Cloud Gaming Server in as little as 90 minutes! This video is a complete step-by-step tutorial for installing Proxmox, nVidia GRID, and defining custom GPU profiles for allocated memory and supported features.

Proxmox 7.2 GPU Passthru

1
2
3
4
echo 'deb http://download.proxmox.com/debian/pve buster pve-no-subscription' >> /etc/apt/sources.list
apt update
apt -y upgrade
apt -y install git build-essential pve-headers dkms jq mdevctl
1
2
3
git clone https://github.com/DualCoder/vgpu_unlock
git clone https://github.com/mbilker/vgpu_unlock-rs
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
1
wget http://download.proxmox.com/debian/dists/bullseye/pve-no-subscription/binary-amd64/pve-headers-5.13.19-2-pve_5.13.19-4_amd64.deb
1
dpkg -i pve-headers-5.13.19-2-pve_5.13.19-4_amd64.deb

Download v14.0 nVidia vGPU Drivers for Linux KVM from Nvidia. You will need to apply for a 90-day trial to have access to the drivers. A business email address is required

The file needed from the ZIP file is “NVIDIA-Linux-x86_64-510.47.03-vgpu-kvm.run”

Reboot

Configure IOMMU

1
nano /etc/default/grub

Add

1
2
3
GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on iommu=pt"
	- OR -
GRUB_CMDLINE_LINUX_DEFAULT="quiet amd_iommu=on iommu=pt"

Save file and close

1
update-grub

Load VFIO Modules at Boot

1
nano /etc/modules
1
2
3
4
echo 'vfio' >> /etc/modules
echo 'vfio_iommu_type1' >> /etc/modules
echo 'vfio_pci' >> /etc/modules
echo 'vfio_virqfd' >> /etc/modules

Save file and close

1
2
echo "options kvm ignore_msrs=1" > /etc/modprobe.d/kvm.conf
echo "blacklist nouveau" >> /etc/modprobe.d/blacklist.conf
1
update-initramfs -u

Reboot

Install Nvidia + VGPU_UNLOCK

1
2
3
4
chmod -R +x vgpu_unlock
chmod +x NVIDIA------.run

./NVIDIA------.run --dkms
1
2
3
nano /usr/src/nvidia-510.47.03/nvidia/os-interface.c

#include "/root/vgpu_unlock/vgpu_unlock_hooks.c"
1
2
3
nano /usr/src/nvidia-450.80/nvidia/nvidia.Kbuild

ldflags-y += -T /root/vgpu_unlock/kern.ld
1
2
3
4
5
6
cd vgpu_unlock-rs

cargo build --release

mkdir /etc/systemd/system/nvidia-vgpud.service.d
mkdir /etc/systemd/system/nvidia-vgpu-mgr.service.d
1
nano /etc/systemd/system/nvidia-vgpud.service.d/vgpu_unlock.conf

Add

1
2
[Service]
Environment=LD_PRELOAD=/root/vgpu_unlock-rs/target/release/libvgpu_unlock_rs.so
1
nano /etc/systemd/system/nvidia-vgpu-mgr.service.d/vgpu_unlock.conf

Add

1
2
[Service]
Environment=LD_PRELOAD=/root/vgpu_unlock-rs/target/release/libvgpu_unlock_rs.so

####Reboot

Define GPU Profiles

1
mkdir /etc/vgpu_unlock
1
nano /etc/vgpu_unlock/profile_override.toml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
[profile.nvidia-18]
num_displays = 1
display_width = 1920
display_height = 1080
max_pixels = 2073600
cuda_enabled = 1
frl_enabled = 60
framebuffer = 11811160064
pci_id = 0x17F011A0
pci_device_id = 0x17F0

Resolution
	- width x height = max_pixels#

Video Memory
	- framebuffer sizes
		- 1GB = 984263338
		- 2GB = 1968526677
		- 3GB = 2952790016
		- 4GB = 3937053354
		- 6GB = 5905580032
		- 8GB = 7874106708
		- 10GB = 9842633380
		- 11GB = 10826896718
		- 12GB = 11811160064
		- 16GB = 15748213408
		- 24GB = 23622320124

PCI IDs
	- pci_id = 0x####@@@@ (Device ID followed by SubSystem ID)
	- pci_device_id = 0x#### (Device ID only)

		Architecture	Card		pci_device_id	pci_id
		- Maxwell	Quadro M6000	0x17F011A0	    0x17F0
		- Pascal	Quadro P6000	0x1B3011A0	    0x1B30
		- Volta		Quadro GV100	0x1DBA121A	    0x1DBA
		- Turing	Quadro RTX 6000	0x1E3012BA	    0x12BA
		- Kepler			(currently not supported)
		- Ampere 			(currently not supported)
1
nano /etc/pve/qemu/[VM#].conf

Add

1
args: -uuid 00000000-0000-0000-0000-000000000###

Add PCIe device to target VM

Select the GPU you added in mdevctl

Launch VM, install either Linux or Windows 10/11

Install nVidia Driver 511.73, matching the Quadro PCI-ID you entered earlier

Disable Display #1 in Display Settings

Install Parsec, Sunshine/Moonlight, SteamPlay, or other Streaming Server

Parsec

https://parsec.app/

Sunshine Streaming Server

https://github.com/loki-47-6F-64/sunshine

Moonlight Streaming Client

https://moonlight-stream.org/

This post is licensed under CC BY 4.0 by the author.