Difference between revisions of "System Overview"
m (2026-11-18 status refresh: GL pixel gate closed (guest GL readback non-black, live t/04); CUDA device enumerates (context-share VAS); acceptance suite gate flags) |
m (2026-11-20 late status refresh: corrected beef-lane diagnosis (PBDMA fetch proven; host-side forensic spin/no-write retire was the floor); r73 beef election + r75 shadow-VTG; v1.0.0-rc1 retracted; latest acceptance red on known flake/race) |
||
| (2 intermediate revisions by the same user not shown) | |||
| Line 5: | Line 5: | ||
<blockquote>'''What this page is.''' The one page to read if you want to understand ''how Open-IOV works'' before diving into any subsystem. Condensed from [https://git.vgpu.io/open-iov.git/tree/docs/03-architecture/01-system-overview.md <code>../docs/03-architecture/01-system-overview.md</code>] (with [https://git.vgpu.io/open-iov.git/tree/docs/03-architecture/09-full-system-picture.md <code>../docs/03-architecture/09-full-system-picture.md</code>] and [https://git.vgpu.io/open-iov.git/tree/docs/03-architecture/10-interaction-and-scheduling.md <code>../docs/03-architecture/10-interaction-and-scheduling.md</code>] for the full picture); the docs tree wins on disagreement.</blockquote> | <blockquote>'''What this page is.''' The one page to read if you want to understand ''how Open-IOV works'' before diving into any subsystem. Condensed from [https://git.vgpu.io/open-iov.git/tree/docs/03-architecture/01-system-overview.md <code>../docs/03-architecture/01-system-overview.md</code>] (with [https://git.vgpu.io/open-iov.git/tree/docs/03-architecture/09-full-system-picture.md <code>../docs/03-architecture/09-full-system-picture.md</code>] and [https://git.vgpu.io/open-iov.git/tree/docs/03-architecture/10-interaction-and-scheduling.md <code>../docs/03-architecture/10-interaction-and-scheduling.md</code>] for the full picture); the docs tree wins on disagreement.</blockquote> | ||
Status: 2026-11- | Status: 2026-11-20, tree <code>master @ 3f34d58a</code>. Tags: <code>[live]</code> proven with the stock guest driver; <code>[gated]</code> implemented default-off; <code>[wip]</code> partial; <code>[wall]</code> current blocker; <code>[target]</code> designed end state. | ||
== 1. The core principle == | == 1. The core principle == | ||
| Line 88: | Line 88: | ||
# Message queues are set up, versions negotiated, RM initialization exchanges complete; object creation, capability queries, vaspaces, channels and the engine inventory are answered with well-formed replies (a small set of control IDs is special-cased with real values; reply fidelity for consumed fields keeps improving). | # Message queues are set up, versions negotiated, RM initialization exchanges complete; object creation, capability queries, vaspaces, channels and the engine inventory are answered with well-formed replies (a small set of control IDs is special-cased with real values; reply fidelity for consumed fields keeps improving). | ||
# <code>nvidia-smi -L</code> enumerates the device; CUDA userspace starts; context creation completes; the full driver-API smoke passes (<code>ALL OK</code>, 64-byte payload round-trips). | # <code>nvidia-smi -L</code> enumerates the device; CUDA userspace starts; context creation completes; the full driver-API smoke passes (<code>ALL OK</code>, 64-byte payload round-trips). | ||
# CE-class copies are forwarded to the physical copy engine (default off) with engine-authored semaphore releases and a fault relay. For GR (graphics/compute), forwarded guest CUDA launches execute on the engine with engine-authored completions relayed to the guest; the direct GL client's GR and CE channels submit and retire, and graphics method streams execute in three shapes ([[Graphics Execution|<code>Graphics-Execution.md</code>]]); guest-readable non-black frames are proven (2026-11-18); the window/screen present is the graphics frontier ([[The Execution Frontier|<code>Execution-Frontier.md</code>]]). | # CE-class copies are forwarded to the physical copy engine (default off) with engine-authored semaphore releases and a fault relay. For GR (graphics/compute), forwarded guest CUDA launches execute on the engine with engine-authored completions relayed to the guest; the direct GL client's GR and CE channels submit and retire, and graphics method streams execute in three shapes ([[Graphics Execution|<code>Graphics-Execution.md</code>]]); guest-readable non-black frames are proven (2026-11-18); the window/screen present is the graphics frontier. On the compute side the acceptance battery gate is '''closed''' (2026-11-19): two consecutive clean-boot <code>guest-accept --cuda</code> runs and a green live <code>t/03</code> (<code>VADD n=32768 bad=0</code>, <code>STATUS OK</code>, <code>EXPECT_CUDA_BATTERY=closed</code>), re-proven in r63 and later rounds; the latest full acceptance run is red on the known UVM/GR-wait lottery ([[The Execution Frontier|<code>Execution-Frontier.md</code>]]). | ||
# On the graphics side, the emulated display engine serves the modeset push buffers and the display notifier/completion (class-aware arm), so nvidia-modeset/NvKms registers (<code>/dev/dri</code>), X settles and the stock NVIDIA GLX path | # On the graphics side, the emulated display engine serves the modeset push buffers and the display notifier/completion (class-aware arm), so nvidia-modeset/NvKms registers (<code>/dev/dri</code>), X settles and the stock NVIDIA GLX path brings up on a good boot (<code>glxinfo -B</code> OpenGL 4.6.0 direct; the current direct-GL stack measures 0.255 FPS in <code>glxgears</code>, r64-r72); the X DDX waits are fixed (the idle-DMA/ScreenInit, USERD-latch, re-probe, kernel-mode-spin and flip-event walls are closed, and X has stayed green >10 minutes), the DDX 3D replay executes, captured GL sections now execute on the engine, the 2D path is green, the direct-GL ring/USERD machinery is hardened, the first non-black GPU-rendered frames and captured videos (45.4 s and 50 s) are proven off-screen, and the guest Vulkan pipeline runs end to end. The direct-GL submission is fully decoded: no client push carries a method-grammar draw header (the work is macro-driven — data-block blobs plus small state pushes invoking MME macros), the recycled pushes are recovered, and the client's draw-state push opens, captures and reaches the engine with the golden MME overlay. The 11-16 wave proved the '''host-side draw chain deterministic end to end''': the real frame pushes are admitted (a persisted colour-target bind from an earlier trusted push, draw markers inside counted method runs, a 14.8-thousand-word window replayed as 40 chunks, a 128-slot candidate ring), the beef draw lane latches a '''clean DS''' (<code>DS-FAST2 ... ds_after=80000000</code>, c963) at the live-derived GS attribute map, capture-less boots get a '''golden 2799-word MME pack''', and <code>MME-CALL-HOLD -> MME-CARRY-FIRE -> MME-PACK-REARM -> MME-SA-REARM</code> fires with the prelude latch kept across the channel recycle. The '''late-11-16 wave crossed the guest submission gap''': an offscreen EGL/GBM shim gives GZDoom a real NVIDIA GL context without Xorg/DDX, the real op=4 <code>CALL(0)</code> <code>0x3800</code> draw (12068 B at push <code>0x402f9d8</code>) is admitted and replayed, <code>CALL(0)</code> executes its captured macro without trapping (eng89t resolves the immediate through client SA[0]/golden and elides not-resident calls), and c1027 reached the first DS with real stage data. '''The GL pixel gate then closed on 2026-11-18''' (<code>b5a68b9a</code>): the first clear/CALL(0)-bearing window is archived and replayed, the engine clear stores red on the guest surface, and the client's CE readback follows the copy's source VA with a generation-pinned snapshot and a with-the-copy fence (the measured ~11 s wait fell to 2.3-7.2 ms). Two consecutive boots read <code>GL_FRAME nonblack=291840 px0=(255,0,0)</code>, live <code>t/04</code> was green on that proof and <code>EXPECT_GL_PIXELS=closed</code> remains set (re-confirmed by the r62b two-boot proof; the latest full run is red on the readback pin/copy race). The remaining gates are the '''X/GLX frame rate''' (<code>glxgears</code> 0.255 FPS, r64-r72: r72b showed the PBDMA fetches the launch in ~1 ms and the engine writes the release, so the floor was the debug-only forensic MME spin plus the host no-write retirement wait; r73's beef election cuts the retire wait and r75 mirrors the VTG program image, but the draw still stores 0 pages and faults <code>Illegal Instruction Encoding</code>, and X/GLX bring-up is a lottery) and the window/screen present; the per-draw MME macro body is a research path. A GZDoom video was separately delivered through the guest's '''software GL''' path (Mesa llvmpipe) as a guaranteed-quality deliverable; it is not a vGPU rendering claim ([[OpenGL|<code>OpenGL.md</code>]], [[Vulkan|<code>Vulkan.md</code>]]). | ||
== 6. Where the detail lives == | == 6. Where the detail lives == | ||
Latest revision as of 20:42, 20 November 2026
An absence of critical technical documentation has historically slowed growth and adoption of developer ecosystems for GPU virtualization.
This CC-BY-4.0 licensed content can either be used with attribution, or used as inspiration for new documentation, created by GPU vendors for public commercial distribution as developer documentation.
Where possible, this documentation will clearly label dates and versions of observed-but-not-guaranteed behaviour vs. vendor-documented stable interfaces/behaviour with guarantees of forward or backward compatibility.
System Overview
What this page is. The one page to read if you want to understand how Open-IOV works before diving into any subsystem. Condensed from
../docs/03-architecture/01-system-overview.md(with../docs/03-architecture/09-full-system-picture.mdand../docs/03-architecture/10-interaction-and-scheduling.mdfor the full picture); the docs tree wins on disagreement.
Status: 2026-11-20, tree master @ 3f34d58a. Tags: [live] proven with the stock guest driver; [gated] implemented default-off; [wip] partial; [wall] current blocker; [target] designed end state.
1. The core principle
Emulate the interface. Pass through the work. Never patch anyone.
A virtual GPU is a mediated passthrough device. The guest must believe it owns a real NVIDIA GPU with a real GPU System Processor running firmware; the host must arrange for the guest's work to execute on real hardware, because emulating compute in software is orders of magnitude too slow to be useful.
EMULATE (software; the guest expects virtual hardware) PCI configuration space, capability structures, power management BAR0 register interface (shadow, not the physical GPU's BAR0) BAR1 VRAM aperture (guest-visible video memory) GSP firmware boot handshake (FWSEC -> booter -> RISCV -> boot complete) GSP-RM message queues, RPC transport, resource-manager controls event/notification channels; MSI-X injection (via eventfd) PASS THROUGH (real hardware; correctness and performance) GPU memory: guest VRAM objects backed by real VRAM [target] page tables / GMMU state used by the GPU [wip] channel execution: guest push buffers on real channels [wip] compute (CUDA) kernel launches on the real engine [live] 3D graphics method streams on the real engine [live] rendered 3D pixels (guest readback proven 2026-11-18) [live] window/screen present, video work [wip] DMA, completion and fault interrupts relayed to the guest
The line between the two is the interface boundary, and it moves as the project matures: early work emulates everything the guest can observe; later work forwards everything that must be fast.
2. Layer map
GUEST
application ── CUDA runtime (libcudart) ─┐
application ── CUDA driver (libcuda) ─┤ userspace: CUDASW stack
OpenGL/Vulkan ── libGLX/libvulkan ──────┘
│ ioctls / mmaps
nvidia.ko (unmodified) ── RM core, resource server, UVM, modeset/drm
│ MMIO + DMA + MSI-X (a virtual PCIe device)
════════╪════════════════════════════════════════════════════════
HYPERVISOR (KVM/QEMU, later Xen/...) VFIO mdev transport + IOMMU
│
HOST MODULE vgsp.ko
├─ virtual device: config space, BAR0/BAR1/BAR3 windows, MSI-X
├─ emulation core: GSP boot + GSP-RM (RPC msgq, controls, objects, events)
├─ memory model: guest RAM access, VA<->PA maps, VRAM backing
└─ work path: channel/push forwarding to the physical GPU
│ module-owned device + real GSP session
HOST BACKEND the GPU's own hardware + GSP firmware, driven by this module
│ (the distribution nvidia.ko stays unmodified and unbound in
│ this mode)
v
REAL HARDWARE (engines, memory, interrupts)
The guest never contains Open-IOV code; the host driver is unmodified; the hypervisor is unmodified. The only new component is vgsp.ko.
3. How each side works
- Guest side. The stock driver performs the GSP boot handshake, sets up RPC message queues in guest memory, creates RM objects, programs channels and submits work — exactly as on bare metal (
Guest-Stack-and-CUDASW.md,CUDA-Userspace-Driver.md,UVM-in-the-Guest.md). - Host module.
vgsp.kois a VFIO mediated-device driver: it registers a parent on the physical GPU, exposes virtual GPU types, and each mdev is one virtual GPU for one VM. Inside, it presents a plausible Turing-class device (config space, BAR shadows, MSI-X), answers the GSP boot handshake and the RPC protocol with well-formed replies, and maintains the memory model that resolves guest physical addresses to host addresses (Host-Module.md). - The interface that makes this possible. Since Turing, the driver's management path is not silicon, it is firmware: the GSP runs a version of NVIDIA's Resource Manager and the driver talks to it over shared-memory queues with a small RPC protocol. That protocol is small, stable and observable — so the project emulates the firmware's interface instead of emulating a GPU (
GSP-and-GSP-RM.md,../docs/03-architecture/03-gsp-and-gsp-rm.md). - Execution. Guest work that must be fast (copies, kernels) is forwarded to a real pipeline driven through the module's own GSP-RM session (
Execution-Plane.md,Physical-GPU-Backend.md,Execution-Frontier.md).
4. Data flows
MMIO from the guest:
guest nvidia.ko writes BAR0 offset X
-> KVM exit -> QEMU -> VFIO region write into the mdev
-> BAR0 shadow state machine (doorbells, VF registers, notify)
-> may wake the emulation thread
RPC from the guest to "firmware":
guest writes a command-queue element into a guest-memory msgq
-> doorbell (BAR0)
-> emulation thread reads the queue through the hypervisor's guest-RAM
accessor, parses the element, dispatches the RPC
-> handler produces a reply element in the status queue
-> emulation injects an MSI-X interrupt to the guest
5. What runs today
- Guest boots, enumerates the virtual PCIe device, binds
nvidia.ko. - The GSP boot handshake completes against the emulation.
- Message queues are set up, versions negotiated, RM initialization exchanges complete; object creation, capability queries, vaspaces, channels and the engine inventory are answered with well-formed replies (a small set of control IDs is special-cased with real values; reply fidelity for consumed fields keeps improving).
nvidia-smi -Lenumerates the device; CUDA userspace starts; context creation completes; the full driver-API smoke passes (ALL OK, 64-byte payload round-trips).- CE-class copies are forwarded to the physical copy engine (default off) with engine-authored semaphore releases and a fault relay. For GR (graphics/compute), forwarded guest CUDA launches execute on the engine with engine-authored completions relayed to the guest; the direct GL client's GR and CE channels submit and retire, and graphics method streams execute in three shapes (
Graphics-Execution.md); guest-readable non-black frames are proven (2026-11-18); the window/screen present is the graphics frontier. On the compute side the acceptance battery gate is closed (2026-11-19): two consecutive clean-bootguest-accept --cudaruns and a green livet/03(VADD n=32768 bad=0,STATUS OK,EXPECT_CUDA_BATTERY=closed), re-proven in r63 and later rounds; the latest full acceptance run is red on the known UVM/GR-wait lottery (Execution-Frontier.md). - On the graphics side, the emulated display engine serves the modeset push buffers and the display notifier/completion (class-aware arm), so nvidia-modeset/NvKms registers (
/dev/dri), X settles and the stock NVIDIA GLX path brings up on a good boot (glxinfo -BOpenGL 4.6.0 direct; the current direct-GL stack measures 0.255 FPS inglxgears, r64-r72); the X DDX waits are fixed (the idle-DMA/ScreenInit, USERD-latch, re-probe, kernel-mode-spin and flip-event walls are closed, and X has stayed green >10 minutes), the DDX 3D replay executes, captured GL sections now execute on the engine, the 2D path is green, the direct-GL ring/USERD machinery is hardened, the first non-black GPU-rendered frames and captured videos (45.4 s and 50 s) are proven off-screen, and the guest Vulkan pipeline runs end to end. The direct-GL submission is fully decoded: no client push carries a method-grammar draw header (the work is macro-driven — data-block blobs plus small state pushes invoking MME macros), the recycled pushes are recovered, and the client's draw-state push opens, captures and reaches the engine with the golden MME overlay. The 11-16 wave proved the host-side draw chain deterministic end to end: the real frame pushes are admitted (a persisted colour-target bind from an earlier trusted push, draw markers inside counted method runs, a 14.8-thousand-word window replayed as 40 chunks, a 128-slot candidate ring), the beef draw lane latches a clean DS (DS-FAST2 ... ds_after=80000000, c963) at the live-derived GS attribute map, capture-less boots get a golden 2799-word MME pack, andMME-CALL-HOLD -> MME-CARRY-FIRE -> MME-PACK-REARM -> MME-SA-REARMfires with the prelude latch kept across the channel recycle. The late-11-16 wave crossed the guest submission gap: an offscreen EGL/GBM shim gives GZDoom a real NVIDIA GL context without Xorg/DDX, the real op=4CALL(0)0x3800draw (12068 B at push0x402f9d8) is admitted and replayed,CALL(0)executes its captured macro without trapping (eng89t resolves the immediate through client SA[0]/golden and elides not-resident calls), and c1027 reached the first DS with real stage data. The GL pixel gate then closed on 2026-11-18 (b5a68b9a): the first clear/CALL(0)-bearing window is archived and replayed, the engine clear stores red on the guest surface, and the client's CE readback follows the copy's source VA with a generation-pinned snapshot and a with-the-copy fence (the measured ~11 s wait fell to 2.3-7.2 ms). Two consecutive boots readGL_FRAME nonblack=291840 px0=(255,0,0), livet/04was green on that proof andEXPECT_GL_PIXELS=closedremains set (re-confirmed by the r62b two-boot proof; the latest full run is red on the readback pin/copy race). The remaining gates are the X/GLX frame rate (glxgears0.255 FPS, r64-r72: r72b showed the PBDMA fetches the launch in ~1 ms and the engine writes the release, so the floor was the debug-only forensic MME spin plus the host no-write retirement wait; r73's beef election cuts the retire wait and r75 mirrors the VTG program image, but the draw still stores 0 pages and faultsIllegal Instruction Encoding, and X/GLX bring-up is a lottery) and the window/screen present; the per-draw MME macro body is a research path. A GZDoom video was separately delivered through the guest's software GL path (Mesa llvmpipe) as a guaranteed-quality deliverable; it is not a vGPU rendering claim (OpenGL.md,Vulkan.md).
6. Where the detail lives
GSP-and-GSP-RM.md— the firmware/RM architecture and the transport we emulate.Host-Module.md— the module's internals.Guest-Stack-and-CUDASW.md,CUDA-Userspace-Driver.md— the guest side.Introduction-to-GMMU.md,UVM-in-the-Guest.md— memory and translation.Execution-Plane.md,Physical-GPU-Backend.md,Execution-Frontier.md— work and execution.Hardware-Software-Matrix.md— supported combinations and portability.../docs/03-architecture/01-system-overview.mdand the rest of the architecture corpus in the repository.