System Overview

System Overview

What this page is. The one page to read if you want to understand how Open-IOV works before diving into any subsystem. Condensed from ../docs/03-architecture/01-system-overview.md (with ../docs/03-architecture/09-full-system-picture.md and ../docs/03-architecture/10-interaction-and-scheduling.md for the full picture); the docs tree wins on disagreement.

Status: 2026-10-12, tree master @ 2f066727. Tags: [live] proven with the stock guest driver; [gated] implemented default-off; [wip] partial; [wall] current blocker; [target] designed end state.

1. The core principle

Emulate the interface. Pass through the work. Never patch anyone.

A virtual GPU is a mediated passthrough device. The guest must believe it owns a real NVIDIA GPU with a real GPU System Processor running firmware; the host must arrange for the guest's work to execute on real hardware, because emulating compute in software is orders of magnitude too slow to be useful.

EMULATE  (software; the guest expects virtual hardware)
  PCI configuration space, capability structures, power management
  BAR0 register interface (shadow, not the physical GPU's BAR0)
  BAR1 VRAM aperture (guest-visible video memory)
  GSP firmware boot handshake (FWSEC -> booter -> RISCV -> boot complete)
  GSP-RM message queues, RPC transport, resource-manager controls
  event/notification channels; MSI-X injection (via eventfd)

PASS THROUGH  (real hardware; correctness and performance)
  GPU memory: guest VRAM objects backed by real VRAM       [target]
  page tables / GMMU state used by the GPU                 [wip]
  channel execution: guest push buffers on real channels   [gated/wall]
  compute (CUDA) and, later, 3D/video work                 [wip/planned]
  DMA, completion and fault interrupts relayed to the guest

The line between the two is the interface boundary, and it moves as the project matures: early work emulates everything the guest can observe; later work forwards everything that must be fast.

2. Layer map

GUEST
  application ── CUDA runtime (libcudart) ─┐
  application ── CUDA driver  (libcuda)   ─┤  userspace: CUDASW stack
  OpenGL/Vulkan ── libGLX/libvulkan ──────┘
        │  ioctls / mmaps
  nvidia.ko (unmodified)  ── RM core, resource server, UVM, modeset/drm
        │  MMIO + DMA + MSI-X   (a virtual PCIe device)
════════╪════════════════════════════════════════════════════════
HYPERVISOR (KVM/QEMU, later Xen/...)  VFIO mdev transport + IOMMU
        │
HOST MODULE  vgsp.ko
  ├─ virtual device: config space, BAR0/BAR1/BAR3 windows, MSI-X
  ├─ emulation core: GSP boot + GSP-RM (RPC msgq, controls, objects, events)
  ├─ memory model: guest RAM access, VA<->PA maps, VRAM backing
  └─ work path: channel/push forwarding to the physical GPU
        │ module-owned device + real GSP session
HOST BACKEND  the GPU's own hardware + GSP firmware, driven by this module
        │     (the distribution nvidia.ko stays unmodified and unbound in
        │      this mode)
        v
REAL HARDWARE (engines, memory, interrupts)

The guest never contains Open-IOV code; the host driver is unmodified; the hypervisor is unmodified. The only new component is vgsp.ko.

3. How each side works

• Guest side. The stock driver performs the GSP boot handshake, sets up RPC message queues in guest memory, creates RM objects, programs channels and submits work — exactly as on bare metal (Guest-Stack-and-CUDASW.md, CUDA-Userspace-Driver.md, UVM-in-the-Guest.md).
• Host module. vgsp.ko is a VFIO mediated-device driver: it registers a parent on the physical GPU, exposes virtual GPU types, and each mdev is one virtual GPU for one VM. Inside, it presents a plausible Turing-class device (config space, BAR shadows, MSI-X), answers the GSP boot handshake and the RPC protocol with well-formed replies, and maintains the memory model that resolves guest physical addresses to host addresses (Host-Module.md).
• The interface that makes this possible. Since Turing, the driver's management path is not silicon, it is firmware: the GSP runs a version of NVIDIA's Resource Manager and the driver talks to it over shared-memory queues with a small RPC protocol. That protocol is small, stable and observable — so the project emulates the firmware's interface instead of emulating a GPU (GSP-and-GSP-RM.md, ../docs/03-architecture/03-gsp-and-gsp-rm.md).
• Execution. Guest work that must be fast (copies, kernels) is forwarded to a real pipeline driven through the module's own GSP-RM session (Execution-Plane.md, Physical-GPU-Backend.md, Execution-Frontier.md).

4. Data flows

MMIO from the guest:
  guest nvidia.ko writes BAR0 offset X
    -> KVM exit -> QEMU -> VFIO region write into the mdev
    -> BAR0 shadow state machine (doorbells, VF registers, notify)
    -> may wake the emulation thread

RPC from the guest to "firmware":
  guest writes a command-queue element into a guest-memory msgq
    -> doorbell (BAR0)
    -> emulation thread reads the queue through the hypervisor's guest-RAM
       accessor, parses the element, dispatches the RPC
    -> handler produces a reply element in the status queue
    -> emulation injects an MSI-X interrupt to the guest

5. What runs today

1. Guest boots, enumerates the virtual PCIe device, binds nvidia.ko.
2. The GSP boot handshake completes against the emulation.
3. Message queues are set up, versions negotiated, RM initialization exchanges complete; object creation, capability queries, vaspaces, channels and the engine inventory are answered with well-formed replies (a small set of control IDs is special-cased with real values; reply fidelity for consumed fields keeps improving).
4. nvidia-smi -L enumerates the device; CUDA userspace starts; context creation completes; the full driver-API smoke passes (ALL OK, 64-byte payload round-trips).
5. CE-class copies are forwarded to the physical copy engine (default off) with engine-authored semaphore releases and a fault relay. For GR (graphics/compute), a forwarded guest launch reaches the front-end but the engine has not executed it yet — the current frontier (Execution-Frontier.md).

6. Where the detail lives

• GSP-and-GSP-RM.md — the firmware/RM architecture and the transport we emulate.
• Host-Module.md — the module's internals.
• Guest-Stack-and-CUDASW.md, CUDA-Userspace-Driver.md — the guest side.
• Introduction-to-GMMU.md, UVM-in-the-Guest.md — memory and translation.
• Execution-Plane.md, Physical-GPU-Backend.md, Execution-Frontier.md — work and execution.
• Hardware-Software-Matrix.md — supported combinations and portability.
• ../docs/03-architecture/01-system-overview.md and the rest of the architecture corpus in the repository.