Proxmox

Enable SR-IOV and PCI passthrough

Add kernel boot parameter

# vi /etc/default/grub

Find the line with "GRUB_CMDLINE_LINUX_DEFAULT", add intel_iommu=on after quiet Then save the chagnes and update grub

# update-grub

If you use AMD CPU, use amd_iommu=on instead.

To verify if the boot parameters take effect, please run

# dmesg | grep -e DMAR -e IOMMU

You should expect to see DMAR: IOMMU enabled

Add kernal modules

# vi /etc/modules 

Add following modules

vfio
vfio_iommu_type1
vfio_pci
vfio_virqfd

Remember to refresh initramfs

# update-initramfs -u -k all

Set virtual functions number

The following command sets 8 VFs on device 0000:03:00.1

# echo 8 > /sys/bus/pci/devices/0000\:03\:00.1/sriov_numvfs

For ethernet card, you may need to set mac address as well. Here eno2 is the card at address 0000:03:00.1

# ip link set eno2 vf 0 mac 98:6C:47:F8:AA:30

Add service file to systemd to automate

Add file /etc/systemd/system/sriov-nic.service

It should look like

[Unit]
Description=Script to enable SR-IOV on boot

[Service]
Type=oneshot
# Starting SR-IOV
ExecStart=/usr/bin/bash -c '/usr/bin/echo 8 > /sys/bus/pci/devices/0000\:03\:00.1/sriov_numvfs'
# Setting static MAC for VFs
ExecStart=/usr/bin/bash -c '/usr/bin/ip link set eno2 vf 0 mac 98:6C:47:F8:AA:30'
ExecStart=/usr/bin/bash -c '/usr/bin/ip link set eno2 vf 1 mac 98:6C:47:F8:AA:31'
ExecStart=/usr/bin/bash -c '/usr/bin/ip link set eno2 vf 2 mac 98:6C:47:F8:AA:32'
ExecStart=/usr/bin/bash -c '/usr/bin/ip link set eno2 vf 3 mac 98:6C:47:F8:AA:33'
ExecStart=/usr/bin/bash -c '/usr/bin/ip link set eno2 vf 4 mac 98:6C:47:F8:AA:34'
ExecStart=/usr/bin/bash -c '/usr/bin/ip link set eno2 vf 5 mac 98:6C:47:F8:AA:35'
ExecStart=/usr/bin/bash -c '/usr/bin/ip link set eno2 vf 6 mac 98:6C:47:F8:AA:36'
ExecStart=/usr/bin/bash -c '/usr/bin/ip link set eno2 vf 7 mac 98:6C:47:F8:AA:37'

[Install]
WantedBy=multi-user.target
# systemctl enable sriov-nic
# systemctl start sriov-nic