r/VFIO • • Mar 21 '21

Meta Help people help you: put some effort in

640 Upvotes

TL;DR: Put some effort into your support requests. If you already feel like reading this post takes too much time, you probably shouldn't join our little VFIO cult because ho boy are you in for a ride.

Okay. We get it.

A popular youtuber made a video showing everyone they can run Valorant in a VM and lots of people want to jump on the bandwagon without first carefully considering the pros and cons of VM gaming, and without wanting to read all the documentation out there on the Arch wiki and other written resources. You're one of those people. That's okay.

You go ahead and start setting up a VM, replicating the precise steps of some other youtuber and at some point hit an issue that you don't know how to resolve because you don't understand all the moving parts of this system. Even this is okay.

But then you come in here and you write a support request that contains as much information as the following sentence: "I don't understand any of this. Help." This is not okay. Online support communities burn out on this type of thing and we're not a large community. And the odds of anyone actually helping you when you do this are slim to none.

So there's a few things you should probably do:

  1. Bite the bullet and start reading. I'm sorry, but even though KVM/Qemu/Libvirt has come a long way since I started using it, it's still far from a turnkey solution that "just works" on everyone's systems. If it doesn't work, and you don't understand the system you're setting up, the odds of getting it to run are slim to none.

    Youtube tutorial videos inevitably skip some steps because the person making the video hasn't hit a certain problem, has different hardware, whatever. Written resources are the thing you're going to need. This shouldn't be hard to accept; after all, you're asking for help on a text-based medium. If you cannot accept this, you probably should give up on running Windows with GPU passthrough in a VM.

  2. Think a bit about the following question: If you're not already a bit familiar with how Linux works, do you feel like learning that and setting up a pretty complex VM system on top of it at the same time? This will take time and effort. If you've never actually used Linux before, start by running it in a VM on Windows, or dual-boot for a while, maybe a few months. Get acquainted with it, so that you understand at a basic level e.g. the permission system with different users, the audio system, etc.

    You're going to need a basic understanding of this to troubleshoot. And most people won't have the patience to teach you while trying to help you get a VM up and running. Consider this a "You must be this tall to ride"-sign.

  3. When asking for help, answer three questions in your post:

    • What exactly did you do?

    • What was the exact result?

    • What did you expect to happen?

    For the first, you can always start with a description of steps you took, from start to finish. Don't point us to a video and expect us to watch it; for one thing, that takes time, for another, we have no way of knowing whether you've actually followed all the steps the way we think you might have. Also provide the command line you're starting qemu with, your libvirt XML, etc. The config, basically.

    For the second, don't say something "doesn't work". Describe where in the boot sequence of the VM things go awry. Libvirt and Qemu give exact errors; give us the errors, pasted verbatim. Get them from your system log, or from libvirt's error dialog, whatever. Be extensive in your description and don't expect us to fish for the information.

    For the third, this may seem silly ("I expected a working VM!") but you should be a bit more detailed in this. Make clear what goal you have, what particular problem you're trying to address. To understand why, consider this problem description: "I put a banana in my car's exhaust, and now my car won't start." To anyone reading this the answer is obviously "Yeah duh, that's what happens when you put a banana in your exhaust." But why did they put a banana in their exhaust? What did they want to achieve? We can remove the banana from the exhaust but then they're no closer to the actual goal they had.

I'm not saying "don't join us".

I'm saying to consider and accept that the technology you want to use isn't "mature for mainstream". You're consciously stepping out of the mainstream, and you'll simply need to put some effort in. The choice you're making commits you to spending time on getting your system to work, and learning how it works. If you can accept that, welcome! If not, however, you probably should stick to dual-booting.


r/VFIO • • Jul 29 '20

Spoof and make your VM Undetectable - No more bullsh*t bans

491 Upvotes

Battleye is banning KVMs, this is now known, but if you've been out of the loop or dont believe me, please read the following posts :

@ https://www.reddit.com/r/VFIO/comments/hts1o1/be_is_banning_kvm_on_r6/

@ https://www.reddit.com/r/VFIO/comments/hznhe8/ubisoft_isnt_lifting_vm_bans_anymore/

In this " guide " I will show you guys how to make your VM undetectable so you will never be banned again.

Boot up Linux. There are 4-5 things you will need to change in order to make your KVM undetected.

Let's start with the most important check, RDTSC. A lot of people claim it is impossible to spoof and I thought this myself until I saw a post on Stack Overflow and had 2 friends try it. It works.

To prevent VMexit, you will need to download the source for the linux kernel and recompile it. Before recompiling it, navigate to arch/x86/kvm/vmx/vmx.c and edit this file.

In VMX.c, at around line 2300, change the line CPU_BASED_RDPMC_EXITING; to CPU_BASED_RDPMC_EXITING | and in the line below, add CPU_BASED_RDTSC_EXITING;

Scroll down in this file until you see the static int (*kvm_vmx_exit_handlers[])(struct kvm_vcpu *vcpu) function.

Before this line, create a function called

static int handle_rdtsc(struct kvm_vcpu *vcpu) 
{     
printk("[vmkernel] handling fake rdtsc from cpl %i\n", vmx_get_cpl(vcpu)); 
uint64_t data;     
data = 123;          
vcpu->arch.regs[VCPU_REGS_RAX] = data & -1u;     vcpu->arch.regs[VCPU_REGS_RDX] = (data >> 32) & -1u;          skip_emulated_instruction(vcpu); 
return 1; }

If you have an error compiling this function modify it so uint64_t data; & data = 123; are at the very top of the function before printk.

In the following function right below it called :

static int (*kvm_vmx_exit_handlers[])(struct kvm_vcpu *vcpu)

Scroll to the very bottom and add this exit handler :

[EXIT_REASON_RDTSC] = handle_rdtsc,

below [EXIT_REASON_ENCLS] = handle_encls, so it looks like

[EXIT_REASON_PREEMPTION_TIMER] = handle_preemption_timer,

[EXIT_REASON_ENCLS] = handle_encls,

[EXIT_REASON_RDTSC] = handle_rdtsc, // <- added exit handler

};

Once you are done, save this file and recompile your Linux kernel.

So you've already spoofed RDTSC timers, what's next? You will need to recompile QEMU and change some strings.

Get the Qemu source and change :

QEMU HARDDISK inside of/hw/ide/core.c & /hw/scsi/scsi-disk.c

QEMU DVD-ROM inside of /hw/ide/core.c & /hw/ide/atapi.c

QEMU CD-ROM inside of /hw/ide/core.c & /hw/scsi/scsi-disk.c

QEMU MICRODRIVE inside of /hw/ide/core.c

QEMU PenPartner tablet inside of /hw/usb/dev-wacom.c & /hw/scsi/scsi-disk.c

padstr inside of /hw/ide/atapi.c

KVMKVMKVM\\0\\0\\0 inside of /target/i386/kvm.c

bochs inside of /block/bochs.c

Bochs Pseudo inside of /roms/ipxe/src/drivers/net/pnic.c

Congrats! Now you have hidden everything related to QEMU. Let's move onto Seabios or OVMF

I used to spoof OVMF myself a while ago, sadly I cannot remember completely what to spoof but you can search for strings on Github. Some of the strings used to be EFI development kit under /OvmfPkg/SmbiosPlatformDxe/SmbiosPlatformDxe.c

For Seabios, change :

Bochs & BXPC inside of src/config.h ( multiple occurances )

/QEMU\/Bochs/ & qemu inside of vgasrc/Kconfig

/06\/23\/99/ inside ofsrc/misc.c

/04\/01\/2014/ inside of src/fw/biostables.c

"s/01\/01\/2011 inside of src/fw/smbios.c

seabios inside of src/fw/biostables.c

Congrats, we are now done with compiling packages and have successfully spoofed RDTSC timers, QEMU, and BIOS related things. It is worth noting these things may change overtime, so check github.

Next, we will edit our VM via Virt Machine Manager and customize the XML.

In your XML,

set your CPU to Host Passthrough with : <cpu mode="host-passthrough" check="none">

IF YOU GET KICKED FOR THIS, SET YOUR CPU MODEL TO SANDY BRIDGE VIA VIRT MACHINE MANAGER

Set a random Vendor ID state with : <vendor_id state="on" value="XXXX"/>

Hide your hypervisor with : <feature policy="disable" name="hypervisor"/>

Hide KVM state with : <kvm>

<hidden state="on"/>

</kvm>

Done!

Now for Virtual Machine Manager :

Set your disk to Sata with type="raw" cache="none" io="native" discard="ignore" detect_zeroes="off"

I also recommend you set a realistic ram & disk size, as well as a realistic disk serial, preferably one of the disk drive name you are using with a few numbers changed.

If all is done properly, your VM should be undetectable by any game software or malware!

I am not sure if this works on Valorant, but it may and you are free to try.

If you want proof this works, try checking for detections with Pafish or Al-Khasser.

If you need proof BE is checking this stuff, read https://www.reddit.com/r/VFIO/comments/hts1o1/be_is_banning_kvm_on_r6/

Thank you to :

Samuel Tulach and Harakter for helping me spoof RDTSC : https://stackoverflow.com/questions/62970242/intercepting-rdtsc-instruction-in-kvm

doomedraven for posting detected QEMU / Seabios Strings on Github @ https://github.com/doomedraven/Tools/blob/master/Virtualization/kvm-qemu.sh

My friends for helping me with other KVM stuff.

I can confirm this works on R6 and I am no longer being kicked.

edit : Thanks for the gold :)

edit 2 : so it is possible that this may be detected as it will return 123 instead of proper numbers with proper timings, according to the original dev. https://www.reddit.com/r/VFIO/comments/i071qx/spoof_and_make_your_vm_undetectable_no_more/fznumah?utm_medium=android_app&utm_source=share This issue can definately be solved however


r/VFIO • • Aug 18 '26

Dragging a Windows app onto my MacBook

403 Upvotes

The app keeps running on Windows. There's a virtual display on each machine — the window gets moved onto the Windows one, captured, and drawn on the Mac as a real macOS window next to my Mac apps.

Median just under 10 ms, capture to display, wired.

NVIDIA only for now. Three months, solo.


r/VFIO • • Sep 03 '26 Bravo

Running games on a VM under Linux while the PC keeps using the same NVIDIA GPU card.

Post image
354 Upvotes

The GPU remains available on the host, one GPU, one display port. No vfio-pci bind. No second GPU. Not just a container. No losing your display when the VM starts. No IOMMU group fights. The host desktop keeps running on the same card the guest is using. Native performance.

This project aims to forward NVIDIA's driver API to the VM with maximal performance. The idea is that you get the CUDA-container experience in VMs, on consumer cards where vGPU isn't available at all.

Currently tested with steamOS and stock distros. The project itself is tested across a wide variety of drivers and GPUs from datacenter to consumer cards (what I could obtain from renting with nested virt or bare metal, primarily on vast.ai).

Experimental, several months, solo. Any bug reports and contributions are welcome.

How it works: Guest kernel module → virtio → QEMU → a per guest process stub to execute the NVIDIA API on the PC → host NVIDIA driver. RM/UVM ioctls are forwarded, not emulated. GPU memory (DMA/BAR) is actual passthrough, not emulated or copied, using regular unprivileged nvidia userspace calls (so it doesn't require root).

Display is forwarded as native as possible, the VM gets a KMS plane (rendering target) from the host display compositor like Wayland, it renders without any cross CPU-GPU copy (unless your display output is on integrated graphics).

Supports CUDA, graphics and display, also intended for compute workloads beyond gaming and graphics.

Geekbench 7 GPU (OpenCL), published to Geekbench's servers so they're independently checkable: RTX 4070 99.6%, RTX 3050 Laptop 99.9%, H100 PCIe 98.8%, A100 80GB 98.0%. clpeak: 7 of 8 subtests at 99.4–100.2%. Not everything is parity, some performance issues are still open.

https://github.com/reindertpelsma/nvkvm-pv - main project, containing the guest module and QEMU patches.

https://github.com/reindertpelsma/nvkvm-steamos - steamOS example

https://github.com/reindertpelsma/nvkvm-kata - kata containers with GPU access

EDIT: Linux amd64 guests only for now (Intel and AMD processors), the guest needs nvkvm's kernel module, and there's no Windows equivalent. A Windows guest means fabricating a GPU that the stock NVIDIA driver will bind to, which is a separate project (kayfabe) and not finished.

If it didn't build or boot on your hardware, that's worth reporting even if it looks like your own setup, install-path bugs and incompatibility reports are the most valuable ones right now.

Only Turing and newer GPUs, older gpu support is not yet finished.


r/VFIO • • Aug 10 '20

Fuck you BattleEYE, like banning VMs is going to stop cheating at all.

Post image
328 Upvotes

r/VFIO • • Sep 12 '20

Tutorial Single GPU Passthrough (VFIO) for Nvidia + Ryzen CPU [Arch-based]

325 Upvotes

Hello,

First post here. I got pretty excited after managing to get my single GPU passthrough working well on my system. I thought it would be far more complicated.

I had to hunt for bits of information from many different places and whilst I don't mind doing this kind of research, I figured it would be good idea to have a guide for others. Here is the link to my repo. Critics/responses/contributions to the information are welcome.

FYI: Contributors are welcome. The guide can become more extensive and include tips for specific kinds of hardware e.g. AMD GPUs, Intel CPUs. Troubleshooting steps can also be added. Thanks!


r/VFIO • • Sep 15 '20

Meta Guys, turns out we don't need GPU passthrough, you can just emulate 2 RTX 3090s!

Post image
305 Upvotes

r/VFIO • • Sep 18 '20

For the curious: 3080 pcie passthrough success!

Post image
295 Upvotes

r/VFIO • • Oct 20 '20

Please upvote for Big Navi Function Level Reset (FLR) awareness.

Thumbnail reddit.com
262 Upvotes

r/VFIO • • Dec 21 '25

When you spent weeks trying to make dgpu passthrough for win vm on optimus laptop work without any success. (Code 43)

Post image
243 Upvotes

r/VFIO • • Mar 30 '21

News Nvidia Resizable BAR drivers released. BIG NEWS: Officially no more Error Code 43!

241 Upvotes

https://www.nvidia.com/en-us/geforce/news/outriders-game-ready-driver/

Windows Virtual Machine Beta Support For GeForce

If you’re primarily a Linux user, you can now enable GeForce GPU passthrough on a Windows Virtual Machine (VM). Play Windows-only games in your VM, or if you’re a developer, more easily test both Windows and Linux code from a single machine, accelerating development.

The beta feature is enabled on all GeForce/TITAN GPUs supported by this driver (Kepler and later for desktop; Maxwell and later for laptop) on Windows 10.


r/VFIO • • Nov 14 '20

News vendor-reset - new project to help AMD users vfio reset woes!

231 Upvotes

If you had not noticed, there is a new channel #vendor-reset in VFIO Discord, this is the official channel for the new project https://github.com/gnif/vendor-reset which is a collaboration between u/belfrypossum and myself.

This project aims to provide an avenue for easily adding complex reset sequences to the kernel without needing to upstream them into the kernel itself. The reason for making this a module is due to the fact that the complexity of these reset routines would prevent them from ever being accepted upstream by the kernel maintainers. While at this time this project only covers AMD's problematic GPUs, it is a framework designed to cover any problematic hardware should it be needed in the future.

Today both u/belfrypossum and I have agreed that the project is ready for use by the general public and would like to announce that it completely succeeds the prior released patches for AMD GPU resets. Currently the project targets (note this is not an exhaustive list and only a few example GPUs for each ASIC are listed here):

* Polaris 10, 11 & 12
* Vega 10 (Vega56/64/FE)
* Vega 20 (Radeon 7)
* Navi 10 (5600XT, 5700, 5700XT)
* Navi 12 (Pro 5600M)
* Navi 14 (Pro 5300, RX 5300, 5500XT)

Usage is very simple, just build the module and modprobe it, or use `dmks` to manage it directly (configuration is included). Nothing more is needed.

There are still conditions under which the GPUs will not reset however we are working to improve them as time permits.

Again, this removes the need to patch your kernel, and it is required that any patches you have applied for GPU resets be removed when using this module.


r/VFIO • • 12d ago Bravo

Resource geforce-sriov: Enabling NVIDIA SR-IOV vGPU on consumer Ampere GPUs

Thumbnail
gallery
230 Upvotes

TLDR: 

  • I got NVIDIA SR-IOV vGPU working on consumer GeForce Ampere and Ada GPUs, tested with 8 concurrent Linux and Windows VMs.

  • Unlike previous projects that enabled vGPU on consumer NVIDIA cards using the older mdev approach, this project uses actual PCIe SR-IOV virtual functions to share a GPU between multiple VMs.

  • It's still experimental, currently verified only on my hardware, and requires NVIDIA's vGPU software and licensing.

Source code and documentation: https://github.com/noon-at-cgn/geforce-sriov

Update (Nov 7, 2026): Just released geforce-sriov v0.2.0 beta, now with Ada support! This release also brings a much simpler setup process, improved documentation, and several other fixes.

Update (Nov 4, 2026): Initial testing with Ada on a 4070 Super borrowed from my friend shows promising results, although the method requires quite a few changes and has more limitations. I'll update the post once I've done more testing.

Introduction

I've been experimenting with the hidden PCIe capabilities on NVIDIA GeForce cards, particularly SR-IOV, and managed to get NVIDIA's vGPU stack working on my RTX 3080s using actual PCIe virtual functions.

My goal was to share the GPUs between multiple VMs without dedicating an entire card to each guest. GPU passthrough works well, but it's not ideal when several Linux and Windows VMs only need a portion of the GPU's resources.

The result is 8 concurrent VMs sharing 2 memory-modded RTX 3080 20GB cards on Proxmox, with both Linux CUDA and Windows Direct3D workloads running.

I've open-sourced the implementation and wanted to share how it works here.

Background: SR-IOV vs the older vGPU unlock approach

Projects like DualCoder/vgpu_unlock and mbilker/vgpu_unlock-rs, along with polloloco/vgpu-proxmox, have helped enable NVIDIA vGPU on consumer Maxwell, Pascal, and Turing cards.

These approaches primarily rely on NVIDIA's older mediated device (mdev) architecture, where the host driver manages multiple virtual GPUs behind a single physical PCIe function.

Ampere and newer professional NVIDIA GPUs use a different architecture based on SR-IOV (Single Root I/O Virtualization).

With SR-IOV, the GPU exposes:

  • Physical Function (PF): The main PCIe function responsible for managing the GPU.

  • Virtual Functions (VFs): Additional PCIe functions that can be assigned to individual VMs.

  • vGPU profiles: NVIDIA's software associates a virtual GPU configuration with each VF, including its framebuffer allocation.

Unlike legacy mdev, these VFs are actual PCIe functions enumerated by Linux.

Consumer NVIDIA GPUs still contain SR-IOV-related capabilities in their PCIe configuration space, even though they aren't exposed through the normal capability discovery process.

How it works

The implementation has five main parts.

1. Exposing the hidden PCIe capabilities

PCIe devices have a configuration space, a region of registers that the operating system uses to discover and configure device capabilities. An offset is a location within that space, measured in bytes from its beginning.

Normally, Linux discovers capabilities by following a linked list of entries in PCIe configuration space.

On Ampere, the SR-IOV extended capability exists at offset 0xBCC, but isn't linked into that list, so Linux doesn't discover it.

So, I wrote a small Linux kernel livepatch module that makes the PCI subsystem discover this hidden capability on consumer GPUs. The module also handles related PCIe features:

  • SR-IOV: Exposes the capability at 0xBCC, allowing Linux to recognize the available VFs.

  • ARI: Exposes the hidden capability at 0xC14 and enables ARI forwarding on the upstream bridge when supported.

  • MSI-X: Reveals the hidden MSI-X capability, which allows the GPU to use multiple independently configurable interrupt vectors instead of relying on a single interrupt.

Since Linux discovers PCIe capabilities during device enumeration, geforce-sriov performs a controlled PCI remove/rescan before NVIDIA's driver binds to the GPU. After the rescan, Linux recognizes the GPU's SR-IOV capability.

Great, with SR-IOV exposed, the next step is getting NVIDIA's vGPU stack to use it.

2. Creating the virtual functions

Once Linux recognizes SR-IOV, geforce-sriov uses NVIDIA's own sriov-manage utility to enable the VFs.

On Ampere, the SR-IOV capability reports support for 16 VFs per GPU, but I can currently get only 4 VFs working. The limitation appears to be related to ARI (Alternative Routing-ID Interpretation), which I'll get into later.

For example:

03:00.0   Physical GPU (PF)
03:00.4   Virtual Function 1
03:00.5   Virtual Function 2
03:00.6   Virtual Function 3
03:00.7   Virtual Function 4

These are real SR-IOV virtual functions exposed by the GPU and enumerated as separate PCIe functions, rather than software-mediated devices managed through the legacy mdev approach.

3. Getting NVIDIA's vGPU software to recognize the card

Now we need to make NVIDIA's vGPU driver accept the card. By default, NVIDIA's vGPU software doesn't recognize consumer GeForce GPUs like my RTX 3080s as supported devices.

My solution uses a small userspace identity hook loaded into nvidia-vgpud. The hook presents my RTX 3080 as an RTX A6000, its professional counterpart based on the same GA102 die, allowing NVIDIA's daemon to recognize the card and register the corresponding vGPU profiles.

Once the profiles are registered, we can choose which vGPU type to assign to each VM. These profiles are defined by NVIDIA and determine the resources available to the guest, including its framebuffer allocation.

The RTX A6000 has 23 profiles in the tested driver version. A few examples:

  • RTXA6000-2Q - 2GB VRAM

  • RTXA6000-4Q - 4GB VRAM

  • RTXA6000-8Q - 8GB VRAM (7GB usable framebuffer)

  • RTXA6000-48Q - 48GB VRAM (45GB usable framebuffer)

For my setup, I used RTXA6000-4Q, allocating 4GB per VM. With 4 VFs per GPU, that's 16GB allocated across four guests on each 20GB card.

Note: Workstation and datacenter GPUs typically have more VRAM than their consumer counterparts. For example, the RTX A6000 has 48GB, while my modified RTX 3080 has 20GB. The available profiles aren't automatically restricted to the physical GPU's memory capacity. Four 4Q profiles fit within 20GB, but four 8Q profiles would require 32GB. You need to account for the combined allocations and driver overhead when choosing profiles.

4. Working around a driver scheduler issue

This was one of the biggest obstacles in the project and required quite a bit of reverse engineering of NVIDIA's proprietary vGPU host driver.

With SR-IOV exposed, the VFs created, and the vGPU profiles registered, the guest VMs could boot and recognize their vGPUs. However, the first actual CUDA workload would consistently hang, with no useful errors in the host logs.

After several hours of tracing through the driver with the help of AI, I narrowed the issue down to the GPU's graphics runlist scheduler being disabled. A callback handling a PMU (Power Management Unit) interrupt was repeatedly disabling the scheduler, which prevented guest workloads from executing. I still don't know exactly why NVIDIA's driver disables the scheduler in this situation.

The workaround is a two-byte driver patch that replaces a conditional jump with NOP instructions, making the callback skip the scheduler-disabling call.

geforce-sriov-patch can automatically apply the change to a copy of the user's own NVIDIA driver object. The patched object is then used to rebuild nvidia.ko. With the patch, both CUDA and D3D11 workloads ran successfully.

5. Driver and licensing requirements

A few things to clarify about geforce-sriov:

  • NVIDIA drivers and proprietary binaries are not distributed. The driver patch is applied to the user's own copy.

  • No vBIOS flashing or firmware modifications are required.

  • No vGPU licensing bypass is provided.

You still need to obtain NVIDIA's vGPU host and guest drivers and licensing yourself to use this project.

Tested configuration

ComponentConfiguration
HypervisorProxmox VE
Kernel7.0.14-16-pve
CPUIntel Xeon E5 v4
GPUs2x memory-modded RTX 3080 20GB (GA102)
PCI device ID10de:2206
NVIDIA host drivervGPU-KVM 580.159.01
Virtual functions4 per GPU, 8 total
Guests4 Linux + 4 Windows 11
vGPU profileRTXA6000-4Q (4GB per VM)

Each physical card runs two Linux guests and two Windows guests.

On Proxmox, a VF can be assigned to a VM using:

hostpci0: 0000:03:00.4,mdev=nvidia-527

Note: Proxmox still uses the mdev= parameter to select the vGPU profile, even though the underlying GPU uses SR-IOV virtual functions rather than legacy mdev sharing.

Initial testing

I ran all eight VMs simultaneously with:

  • CUDA workloads in the Linux VMs.

  • D3D11 graphics workloads in the Windows VMs.

  • Four active vGPUs on each physical card.

The test ran for 30 minutes without any observed failures. This isn't enough to establish long-term stability, but it verifies that multiple independent Linux and Windows GPU workloads can run concurrently.

Performance results

I ran the same PyTorch benchmarks in two configurations to measure virtualization overhead with a single active vGPU:

  • Physical GPU: Direct CUDA access from an LXC container on the Proxmox host.

  • vGPU: A VM using a 4GB RTXA6000-4Q profile on the same card.

Both configurations used the same Debian root filesystem, PyTorch environment, four pinned CPU cores, and 8GB of system RAM.

The workloads were FP32 and FP16 matrix multiplication using 4096 × 4096 matrices, plus a 1GB device memory copy.

BenchmarkPhysical GPU (LXC)vGPU (VM)
FP32 matmul20.47 TFLOPS20.54 TFLOPS
FP16 matmul (Tensor Cores)58.35 TFLOPS58.54 TFLOPS
GPU memory copy681.7 GB/s704.4 GB/s

The FP32 and FP16 results were within 1% of direct physical-GPU access, suggesting very little virtualization overhead for these particular workloads.

Some qualifications:

  • Results are medians of three measured runs per configuration.

  • The physical-GPU baseline used NVIDIA's vGPU host driver, not the regular GeForce driver.

  • Only one vGPU was actively using the card during the comparison.

The full methodology and screenshots are available here:

https://github.com/noon-at-cgn/geforce-sriov/blob/main/docs/showcase.md

The ARI problem

The biggest unresolved issue is ARI (Alternative Routing-ID Interpretation), which is currently limiting me to 4 VFs per GPU.

Normally, PCIe uses a 3-bit function number, allowing up to 2^3 = 8 functions (0–7) per device. ARI repurposes the 5-bit device number field to extend the function number to 8 bits, allowing up to 2^8 = 256 functions on a single device.

This matters for SR-IOV because a GPU may need to expose many VFs. On my RTX 3080s, the SR-IOV capability reports support for 16 VFs.

Without ARI, the function layout looks like this:

Function 0   GPU (Physical Function)
Function 1   GPU HDMI/DP Audio
Function 2   Unused
Function 3   Unused
Function 4   VF 1
Function 5   VF 2
Function 6   VF 3
Function 7   VF 4
------------- 3-bit function limit
Function 8   VF 5  (requires ARI)
Function 9   VF 6  (requires ARI)
...
Function 19  VF 16 (requires ARI)

So although conventional PCIe allows eight functions, only four are available for VFs because the GPU's SR-IOV configuration places the first VF at function 4. Functions 0–3 aren't part of the VF allocation.

To expose all 16 VFs on the same device, the GPU needs to support function numbers beyond 7, which needs ARI.

So far, I've managed to:

  • Expose the GPU's hidden ARI capability at 0xC14.

  • Enable ARI forwarding on the immediate upstream PCIe bridge.

  • Confirm that Linux recognizes ARI as enabled.

  • Successfully create four VFs at functions 4–7.

However, the GPU doesn't appear to correctly decode extended function numbers (8 and above), even with ARI enabled in software.

For now, the working configuration is limited to 4 VFs per card, despite the hardware reporting support for 16.

Current limitations

This is still an experimental project, not a general-purpose GeForce vGPU solution yet.

Verified so far:

  • Two memory-modded RTX 3080 20GB cards (10de:2206, subsystem 10de:146d).

  • Proxmox VE with kernel 7.0.14-16-pve.

  • NVIDIA vGPU host driver 580.159.01.

  • Four working SR-IOV VFs per GPU.

  • Eight concurrent Linux and Windows guests.

  • CUDA, D3D11, and single-vGPU compute benchmarks.

Not yet verified:

  • Other Ampere cards.

  • Ada and Blackwell GeForce GPUs are not expected to work with the current implementation (see below)

Other practical limitations:

  • vGPU profiles must fit within the GPU's physical VRAM.

  • GPUs used as the host's boot display require special preparation.

  • Kernel and driver updates may require rebuilding or reapplying the driver workaround.

  • NVIDIA vGPU software and licensing are required.

Ada and Blackwell compatibility

  • The current implementation is not expected to work with Ada and Blackwell GPUs because their vGPU drivers rely on GSP (GPU System Processor), an onboard processor that runs NVIDIA firmware and handles much of the GPU initialization and resource management.

  • Ampere also has GSP hardware, but my configuration uses the older host-driver path, where these operations are primarily handled by the CPU-side driver. On Ada and Blackwell, more of this functionality runs inside GSP firmware, making it harder to modify or intercept from the host.

  • The current driver workaround doesn't apply to that architecture, and Blackwell also requires NVIDIA's open kernel-module path, which the project doesn't yet support.

I would like to experiment more with both generations, but I don't currently have access to Ada or Blackwell GPUs for testing.

Try it out

Disclaimer: geforce-sriov is an experimental, unofficial project and is not affiliated with or endorsed by NVIDIA. Use it at your own risk, preferably on a non-production system.

The full source code, installation instructions, technical documentation, and benchmarks are available:

GitHub: https://github.com/noon-at-cgn/geforce-sriov

Compatibility reports are welcome as compatibility remains largely unverified beyond my tested setup.


r/VFIO • • Apr 20 '22

Discussion I find it kinda hilarious that this is possible, but why is it possible anyway?

227 Upvotes

r/VFIO • • Sep 26 '21

News One of the Steam Deck’s biggest hurdles just disappeared: EAC has come to Linux and BattlEye is inbound

Thumbnail
reddit.com
210 Upvotes

r/VFIO • • Sep 11 '20

Discussion Battleye is now baiting bans

204 Upvotes

For a long time now, I have been a linux gamer. Playing games through wine, proton, and sometimes in KVM. I while ago, Battleye announced on twitter that they would no longer allow users to play within virtual machines. Their policy was "as always we will ban any users who actively try to bypass our measures. Normal users will only receive a kick" https://twitter.com/TheBattlEye/status/1289027890227621889. However revently, after switching from intel to amd, my kvm required a few options to play games in my kvm. After setting them, there was no vm masking present, windows fully detected "Virtual Machine Yes" and my processor was listed as EPYC. Obviously no spoofing going on here. I was able to play escape from tarkov with no problem. but the next day, I woke up to a ban. If battleye's policy is to kick, why wasn't i kicked. If they were able to detect my vm to ban me, why didnt they just kick me. Obviously something fishy is going on here.

A few months ago, I had contacted EFT support to ask about KVM usage within tarkov. Their first response to me was "We recommend not to use the Virtual Machine utilities to play safe."
Of course, that is vague, play safe in what sense? for my own security? for the best performance? So, I asked more questions, and received the same response "We just do not recommend it. We will inform you if there are any changes in the future."

So, if battleye's policy is a kick to vm users. And EFT's policy is that they "don't recommend it", what did I do to deserve a perma ban on my account. If they were going to restrict access to the game, I want my money back. If you are going to kick me, so be it, just refund me the game, and I won't support the company anymore.

Not only is an infinite kick, the same as a ban, but they clearly stated that they would not ban KVM users unless they tried to evade the anti cheat. How is it, that a system that reports to windows as a Virtual Machine, and with a processor labeled EPYC, could be "evading detection" from the anti cheat.

It was clearly a VM and your anti cheat wrongly banned me, all you had to do was kick me for use of virtual machine. If the anticheat detected my vm to ban me, couldn't it have just notified me that I was no longer allowed to pay for the game I payed 140$ for?

We need justice, for all of the linux users, who's ability to play their games has been revoked, and for those who have been banned falsely by battleye. Our reports are being ignored, cheating is rampant, but now our ability to play the games we payed for has been revoked, and we have been labeled cheaters.


r/VFIO • • Nov 18 '20

Discussion Is it true, that both RTX 3000 and Radeon 6000 solved their issues with passthrough? Screenshot is from LTT video, do you know about other sources confirming this?

Post image
203 Upvotes

r/VFIO • • Jun 20 '21

News Up-vote for better AMD Radeon VFIO support

Thumbnail reddit.com
202 Upvotes

r/VFIO • • Jul 18 '19

Help us make AMD aware of how many people the Vega/Navi reset bug affects!

Thumbnail
reddit.com
199 Upvotes

r/VFIO • • Sep 10 '21

Success Story Finally did it =)).

Post image
182 Upvotes

r/VFIO • • Dec 03 '20

We're ready for One GPU -- Two OSs. Intel Xe, SR-IOV and thoughts on VMs (Level1Linux)

Thumbnail
youtube.com
181 Upvotes

r/VFIO • • Dec 14 '17

Looking Glass has been released.

Thumbnail
github.com
175 Upvotes

r/VFIO • • Mar 22 '25

you can now play fortnite even with a blatant vm, apparently

Thumbnail
gallery
173 Upvotes