Difference between revisions of "Main Page"

From Open-IOV
Jump to navigation Jump to search
m (2026-11-18 late status refresh: CUDA CE push page + fresh-CE walk seed fixed; the wall is now the guest-side UVM page-tree-init push; GL pixel gate held; checkpoint 5b23a983 noted)
m (2026-11-20 late status refresh: corrected beef-lane diagnosis (PBDMA fetch proven; host-side forensic spin/no-write retire was the floor); r73 beef election + r75 shadow-VTG; v1.0.0-rc1 retracted; latest acceptance red on known flake/race)
 
(One intermediate revision by the same user not shown)
Line 21: Line 21:
* '''[[FAQ]]''' — the questions we get asked most.
* '''[[FAQ]]''' — the questions we get asked most.


== Project status (2026-11-18) ==
== Project status (2026-11-20) ==
{| class="wikitable"
{| class="wikitable"
! Area !! State
! Area !! State
Line 33: Line 33:
| CUDA userspace init and the driver-API smoke (ALL OK, 64-byte payload round-trips) || Working
| CUDA userspace init and the driver-API smoke (ALL OK, 64-byte payload round-trips) || Working
|-
|-
| Guest CUDA kernel execution on real hardware || Working — vector-add, matrix-multiply and transformer kernels run green on fresh boots. On the 2026-11-18 battery the device enumerates (the CUDA GR channels inherit the context-share VAS), the 124 B UVM CE push page and the fresh-CE ring-walk seed are fixed, and cudaMalloc now stalls in the guest's UVM_REGISTER_GPU page-tree-init push (a boot-dependent race). The staged-transfer class and per-guest isolation remain in progress
| Guest CUDA kernel execution on real hardware || Working — vector-add, matrix-multiply and transformer kernels run green on fresh boots, and the acceptance battery gate is closed (2026-11-19): two consecutive clean-boot guest-accept --cuda runs and a green live t/03 (VADD bad=0, STATUS OK, EXPECT_CUDA_BATTERY=closed), re-proven in r63 and later rounds. The latest full acceptance run (2026-11-20 19:49Z) is red on the known UVM/GR-wait lottery (victory_mail rc=1, 205 s); the flag stays closed. The path fixed the CE/GR ring and pool pairing, the GR ctx-init semaphores, the init preamble (host-path SW methods and the SET_VALID_SPAN_OVERFLOW_AREA trio were resetting FECS), the program-page re-rooter, the pa=0 fill/relocate class and the launch lottery. The staged-transfer class and per-guest isolation remain in progress
|-
|-
| Display and KMS (nvidia-modeset/NvKms, /dev/dri, X) || Working — the emulated display engine serves the modeset push buffers and notifier completion; NvKms registers, X settles and glxinfo reports OpenGL 4.6.0 with direct rendering
| Display and KMS (nvidia-modeset/NvKms, /dev/dri, X) || Working — the emulated display engine serves the modeset push buffers and notifier completion; NvKms registers, X settles and glxinfo reports OpenGL 4.6.0 with direct rendering
|-
|-
| 3D graphics: Xorg + OpenGL, Vulkan || In progress — the display/KMS and GL stacks run, X and the stock GLX path are green, the DDX 3D replay executes, and the GL pixel gate closed on 2026-11-18: a stock guest glReadPixels reads a non-black engine-rendered frame (GL_FRAME nonblack=291840), enforced by the live acceptance lane. The gate was re-verified on the later CUDA-side rounds. The remaining frontier is the window/screen present, X start speed and the boot lottery — see [[Graphics Execution]], [[OpenGL]], [[Vulkan]]
| 3D graphics: Xorg + OpenGL, Vulkan || In progress — the display/KMS and GL stacks run, X and the stock GLX path bring up on a good boot, the DDX 3D replay executes, and the GL pixel gate is closed (2026-11-18): a stock guest glReadPixels reads a non-black engine-rendered frame (GL_FRAME nonblack=291840), enforced by the live acceptance lane and re-confirmed by the r62b two-boot proof (red on the readback pin/copy race in the latest full run). The current wall is the X/GLX frame rate: glxgears measures 0.255 FPS (r64-r72), but r72b corrected the earlier diagnosis — the PBDMA fetches the direct-GL client-ring launch in ~1 ms and the engine writes the channel release; the floor was the debug-only forensic MME spin (2.8 s) plus the host no-write retirement wait (~2.45 s when the window stores 0 pages). r73's beef election cuts the retire wait (1337→12 ms), and r75 mirrors the VTG program image into the win0 shadow (3f34d58a); the draw still stores 0 pages and faults Illegal Instruction Encoding, and X/GLX bring-up remains a lottery — see [[Graphics Execution]], [[OpenGL]], [[Vulkan]]
|-
|-
| Windows guests, multi-tenancy, migration, performance || Not yet — multiple mediated devices (different emulated VRAM sizes) and multiple guests already coexist; per-guest execution isolation is in progress
| Windows guests, multi-tenancy, migration, performance || Not yet — multiple mediated devices (different emulated VRAM sizes) and multiple guests already coexist; per-guest execution isolation is in progress

Latest revision as of 20:42, 20 November 2026

An absence of critical technical documentation has historically slowed growth and adoption of developer ecosystems for GPU virtualization.

This CC-BY-4.0 licensed content can either be used with attribution, or used as inspiration for new documentation, created by GPU vendors for public commercial distribution as developer documentation.

Where possible, this documentation will clearly label dates and versions of observed-but-not-guaranteed behaviour vs. vendor-documented stable interfaces/behaviour with guarantees of forward or backward compatibility.


Open-IOV is an open-source virtualization layer for NVIDIA GPUs: it runs stock, unmodified NVIDIA drivers inside virtual machines — Linux and Windows — with CUDA on hardware you own, without proprietary vGPU licenses, without patching the guest driver, the host driver, the hypervisor or the firmware.

Emulate the interface. Pass through the work. Never patch anyone.

The only new component is one host kernel module (vgsp.ko), which implements the interface the guest driver expects to talk to — the GPU System Processor and its resource-manager firmware — and arranges for the guest's work to run on real GPU hardware.

About Open-IOV · Community · Documentation

Start here

Project status (2026-11-20)

Area State
Emulated vGPU device (PCI, BARs, MSI-X, VFIO mdev) Working
GSP boot emulation and GSP-RM message queues / RPC Working
Stock guest driver probe, nvidia-smi (TU104) Working
CUDA userspace init and the driver-API smoke (ALL OK, 64-byte payload round-trips) Working
Guest CUDA kernel execution on real hardware Working — vector-add, matrix-multiply and transformer kernels run green on fresh boots, and the acceptance battery gate is closed (2026-11-19): two consecutive clean-boot guest-accept --cuda runs and a green live t/03 (VADD bad=0, STATUS OK, EXPECT_CUDA_BATTERY=closed), re-proven in r63 and later rounds. The latest full acceptance run (2026-11-20 19:49Z) is red on the known UVM/GR-wait lottery (victory_mail rc=1, 205 s); the flag stays closed. The path fixed the CE/GR ring and pool pairing, the GR ctx-init semaphores, the init preamble (host-path SW methods and the SET_VALID_SPAN_OVERFLOW_AREA trio were resetting FECS), the program-page re-rooter, the pa=0 fill/relocate class and the launch lottery. The staged-transfer class and per-guest isolation remain in progress
Display and KMS (nvidia-modeset/NvKms, /dev/dri, X) Working — the emulated display engine serves the modeset push buffers and notifier completion; NvKms registers, X settles and glxinfo reports OpenGL 4.6.0 with direct rendering
3D graphics: Xorg + OpenGL, Vulkan In progress — the display/KMS and GL stacks run, X and the stock GLX path bring up on a good boot, the DDX 3D replay executes, and the GL pixel gate is closed (2026-11-18): a stock guest glReadPixels reads a non-black engine-rendered frame (GL_FRAME nonblack=291840), enforced by the live acceptance lane and re-confirmed by the r62b two-boot proof (red on the readback pin/copy race in the latest full run). The current wall is the X/GLX frame rate: glxgears measures 0.255 FPS (r64-r72), but r72b corrected the earlier diagnosis — the PBDMA fetches the direct-GL client-ring launch in ~1 ms and the engine writes the channel release; the floor was the debug-only forensic MME spin (2.8 s) plus the host no-write retirement wait (~2.45 s when the window stores 0 pages). r73's beef election cuts the retire wait (1337→12 ms), and r75 mirrors the VTG program image into the win0 shadow (3f34d58a); the draw still stores 0 pages and faults Illegal Instruction Encoding, and X/GLX bring-up remains a lottery — see Graphics Execution, OpenGL, Vulkan
Windows guests, multi-tenancy, migration, performance Not yet — multiple mediated devices (different emulated VRAM sizes) and multiple guests already coexist; per-guest execution isolation is in progress

The full story, with evidence and the current wall: Project Status and The Execution Frontier.

Graphics

  • Graphics Execution — how guest 3D work reaches the physical engines
  • OpenGL — the stock guest GL stack on the emulated vGPU
  • Vulkan — the guest Vulkan pipeline on the emulated vGPU
  • DirectX — the Windows/DirectX target

Deep dives

Firmware interoperability

Project and ecosystem

Community

About Open-IOV

Open-IOV runs unmodified NVIDIA driver stacks inside virtual machines with GPU compute on hardware you own — no proprietary vGPU license, and no patches to the guest driver, the host driver, the hypervisor or the firmware. The project adds one host kernel module (vgsp.ko) that implements the interface the guest driver expects to talk to — the GPU System Processor and its resource-manager firmware — and arranges for the guest's work to run on real GPU hardware.

  • Mission: make NVIDIA GPUs fully virtualizable, by anyone, for any hypervisor, without modifying NVIDIA software, firmware or hardware.
  • Principles: no patches anywhere; honest status carried with dated evidence; an independent, self-contained IP block; durability by owning the interface rather than a snapshot of it.
  • Licensing: all documentation on this site is CC-BY-4.0; the project's code is BSD-3-Clause (GPL-2.0 for the kernel modules); NVIDIA software and firmware are used unmodified and are not redistributed.

Read more about Open-IOV →

About this wiki

This wiki previously hosted documentation from the earlier project era (2021–2023). Those pages are retained for reference under History; current documentation starts at Documentation.