System Overview - Revision history

WikiBot: Fix repository links: point to the cgit tree URLs

2026-10-12T04:41:02Z

Fix repository links: point to the cgit tree URLs

← Older revision		Revision as of 04:41, 12 October 2026
Line 3:		Line 3:
	= System Overview =		= System Overview =

	<blockquote>'''What this page is.''' The one page to read if you want to understand ''how Open-IOV works'' before diving into any subsystem. Condensed from [https://git.vgpu.io/open-iov/~~blob/master~~/docs/03-architecture/01-system-overview.md <code>../docs/03-architecture/01-system-overview.md</code>] (with [https://git.vgpu.io/open-iov/~~blob/master~~/docs/03-architecture/09-full-system-picture.md <code>../docs/03-architecture/09-full-system-picture.md</code>] and [https://git.vgpu.io/open-iov/~~blob/master~~/docs/03-architecture/10-interaction-and-scheduling.md <code>../docs/03-architecture/10-interaction-and-scheduling.md</code>] for the full picture); the docs tree wins on disagreement.</blockquote>		<blockquote>'''What this page is.''' The one page to read if you want to understand ''how Open-IOV works'' before diving into any subsystem. Condensed from [https://git.vgpu.io/open-iov.git/tree/docs/03-architecture/01-system-overview.md <code>../docs/03-architecture/01-system-overview.md</code>] (with [https://git.vgpu.io/open-iov.git/tree/docs/03-architecture/09-full-system-picture.md <code>../docs/03-architecture/09-full-system-picture.md</code>] and [https://git.vgpu.io/open-iov.git/tree/docs/03-architecture/10-interaction-and-scheduling.md <code>../docs/03-architecture/10-interaction-and-scheduling.md</code>] for the full picture); the docs tree wins on disagreement.</blockquote>

	Status: 2026-10-12, tree <code>master @ 2f066727</code>. Tags: <code>[live]</code> proven with the stock guest driver; <code>[gated]</code> implemented default-off; <code>[wip]</code> partial; <code>[wall]</code> current blocker; <code>[target]</code> designed end state.		Status: 2026-10-12, tree <code>master @ 2f066727</code>. Tags: <code>[live]</code> proven with the stock guest driver; <code>[gated]</code> implemented default-off; <code>[wip]</code> partial; <code>[wall]</code> current blocker; <code>[target]</code> designed end state.
Line 60:		Line 60:
	* '''Guest side.''' The stock driver performs the GSP boot handshake, sets up RPC message queues in guest memory, creates RM objects, programs channels and submits work — exactly as on bare metal ([[The Guest Stack and CUDASW\|<code>Guest-Stack-and-CUDASW.md</code>]], [[CUDA Userspace Driver\|<code>CUDA-Userspace-Driver.md</code>]], [[UVM in the Guest\|<code>UVM-in-the-Guest.md</code>]]).		* '''Guest side.''' The stock driver performs the GSP boot handshake, sets up RPC message queues in guest memory, creates RM objects, programs channels and submits work — exactly as on bare metal ([[The Guest Stack and CUDASW\|<code>Guest-Stack-and-CUDASW.md</code>]], [[CUDA Userspace Driver\|<code>CUDA-Userspace-Driver.md</code>]], [[UVM in the Guest\|<code>UVM-in-the-Guest.md</code>]]).
	* '''Host module.''' <code>vgsp.ko</code> is a VFIO mediated-device driver: it registers a parent on the physical GPU, exposes virtual GPU types, and each mdev is one virtual GPU for one VM. Inside, it presents a plausible Turing-class device (config space, BAR shadows, MSI-X), answers the GSP boot handshake and the RPC protocol with well-formed replies, and maintains the memory model that resolves guest physical addresses to host addresses ([[The Host Module (vgsp.ko)\|<code>Host-Module.md</code>]]).		* '''Host module.''' <code>vgsp.ko</code> is a VFIO mediated-device driver: it registers a parent on the physical GPU, exposes virtual GPU types, and each mdev is one virtual GPU for one VM. Inside, it presents a plausible Turing-class device (config space, BAR shadows, MSI-X), answers the GSP boot handshake and the RPC protocol with well-formed replies, and maintains the memory model that resolves guest physical addresses to host addresses ([[The Host Module (vgsp.ko)\|<code>Host-Module.md</code>]]).
	* '''The interface that makes this possible.''' Since Turing, the driver's management path is not ''silicon'', it is ''firmware'': the GSP runs a version of NVIDIA's Resource Manager and the driver talks to it over shared-memory queues with a small RPC protocol. That protocol is small, stable and observable — so the project emulates '''the firmware's interface''' instead of emulating a GPU ([[GSP and GSP-RM\|<code>GSP-and-GSP-RM.md</code>]], [https://git.vgpu.io/open-iov/~~blob/master~~/docs/03-architecture/03-gsp-and-gsp-rm.md <code>../docs/03-architecture/03-gsp-and-gsp-rm.md</code>]).		* '''The interface that makes this possible.''' Since Turing, the driver's management path is not ''silicon'', it is ''firmware'': the GSP runs a version of NVIDIA's Resource Manager and the driver talks to it over shared-memory queues with a small RPC protocol. That protocol is small, stable and observable — so the project emulates '''the firmware's interface''' instead of emulating a GPU ([[GSP and GSP-RM\|<code>GSP-and-GSP-RM.md</code>]], [https://git.vgpu.io/open-iov.git/tree/docs/03-architecture/03-gsp-and-gsp-rm.md <code>../docs/03-architecture/03-gsp-and-gsp-rm.md</code>]).
	* '''Execution.''' Guest work that must be fast (copies, kernels) is forwarded to a real pipeline driven through the module's own GSP-RM session ([[The vGSP Execution Plane (vEP)\|<code>Execution-Plane.md</code>]], [[Physical GPU Backend and Guest Wiring\|<code>Physical-GPU-Backend.md</code>]], [[The Execution Frontier\|<code>Execution-Frontier.md</code>]]).		* '''Execution.''' Guest work that must be fast (copies, kernels) is forwarded to a real pipeline driven through the module's own GSP-RM session ([[The vGSP Execution Plane (vEP)\|<code>Execution-Plane.md</code>]], [[Physical GPU Backend and Guest Wiring\|<code>Physical-GPU-Backend.md</code>]], [[The Execution Frontier\|<code>Execution-Frontier.md</code>]]).

Line 95:		Line 95:
	* [[The vGSP Execution Plane (vEP)\|<code>Execution-Plane.md</code>]], [[Physical GPU Backend and Guest Wiring\|<code>Physical-GPU-Backend.md</code>]], [[The Execution Frontier\|<code>Execution-Frontier.md</code>]] — work and execution.		* [[The vGSP Execution Plane (vEP)\|<code>Execution-Plane.md</code>]], [[Physical GPU Backend and Guest Wiring\|<code>Physical-GPU-Backend.md</code>]], [[The Execution Frontier\|<code>Execution-Frontier.md</code>]] — work and execution.
	* [[Hardware and Software Matrix\|<code>Hardware-Software-Matrix.md</code>]] — supported combinations and portability.		* [[Hardware and Software Matrix\|<code>Hardware-Software-Matrix.md</code>]] — supported combinations and portability.
	* [https://git.vgpu.io/open-iov/~~blob/master~~/docs/03-architecture/01-system-overview.md <code>../docs/03-architecture/01-system-overview.md</code>] and the rest of the architecture corpus in the repository.		* [https://git.vgpu.io/open-iov.git/tree/docs/03-architecture/01-system-overview.md <code>../docs/03-architecture/01-system-overview.md</code>] and the rest of the architecture corpus in the repository.

WikiBot: Carry the site-wide About/licensing message on every page (2026-10-12)

2026-10-12T04:29:52Z

Carry the site-wide About/licensing message on every page (2026-10-12)

← Older revision		Revision as of 04:29, 12 October 2026
Line 1:		Line 1:
			{{About message}}

	= System Overview =		= System Overview =

WikiBot: Add page from tracked wiki source (2026 IA update)

2026-10-12T04:22:16Z

Add page from tracked wiki source (2026 IA update)

New page

= System Overview =

<blockquote>'''What this page is.''' The one page to read if you want to understand ''how Open-IOV works'' before diving into any subsystem. Condensed from [https://git.vgpu.io/open-iov/blob/master/docs/03-architecture/01-system-overview.md <code>../docs/03-architecture/01-system-overview.md</code>] (with [https://git.vgpu.io/open-iov/blob/master/docs/03-architecture/09-full-system-picture.md <code>../docs/03-architecture/09-full-system-picture.md</code>] and [https://git.vgpu.io/open-iov/blob/master/docs/03-architecture/10-interaction-and-scheduling.md <code>../docs/03-architecture/10-interaction-and-scheduling.md</code>] for the full picture); the docs tree wins on disagreement.</blockquote>

Status: 2026-10-12, tree <code>master @ 2f066727</code>. Tags: <code>[live]</code> proven with the stock guest driver; <code>[gated]</code> implemented default-off; <code>[wip]</code> partial; <code>[wall]</code> current blocker; <code>[target]</code> designed end state.

== 1. The core principle ==

'''Emulate the interface. Pass through the work. Never patch anyone.'''

A virtual GPU is a ''mediated passthrough'' device. The guest must believe it owns a real NVIDIA GPU with a real GPU System Processor running firmware; the host must arrange for the guest's work to execute on real hardware, because emulating compute in software is orders of magnitude too slow to be useful.

<pre>EMULATE (software; the guest expects virtual hardware)
PCI configuration space, capability structures, power management
BAR0 register interface (shadow, not the physical GPU's BAR0)
BAR1 VRAM aperture (guest-visible video memory)
GSP firmware boot handshake (FWSEC -> booter -> RISCV -> boot complete)
GSP-RM message queues, RPC transport, resource-manager controls
event/notification channels; MSI-X injection (via eventfd)

PASS THROUGH (real hardware; correctness and performance)
GPU memory: guest VRAM objects backed by real VRAM [target]
page tables / GMMU state used by the GPU [wip]
channel execution: guest push buffers on real channels [gated/wall]
compute (CUDA) and, later, 3D/video work [wip/planned]
DMA, completion and fault interrupts relayed to the guest</pre>

The line between the two is the '''interface boundary''', and it moves as the project matures: early work emulates everything the guest can observe; later work forwards everything that must be fast.

== 2. Layer map ==

<pre>GUEST
application ── CUDA runtime (libcudart) ─┐
application ── CUDA driver (libcuda) ─┤ userspace: CUDASW stack
OpenGL/Vulkan ── libGLX/libvulkan ──────┘
│ ioctls / mmaps
nvidia.ko (unmodified) ── RM core, resource server, UVM, modeset/drm
│ MMIO + DMA + MSI-X (a virtual PCIe device)
════════╪════════════════════════════════════════════════════════
HYPERVISOR (KVM/QEMU, later Xen/...) VFIO mdev transport + IOMMU
│
HOST MODULE vgsp.ko
├─ virtual device: config space, BAR0/BAR1/BAR3 windows, MSI-X
├─ emulation core: GSP boot + GSP-RM (RPC msgq, controls, objects, events)
├─ memory model: guest RAM access, VA<->PA maps, VRAM backing
└─ work path: channel/push forwarding to the physical GPU
│ module-owned device + real GSP session
HOST BACKEND the GPU's own hardware + GSP firmware, driven by this module
│ (the distribution nvidia.ko stays unmodified and unbound in
│ this mode)
v
REAL HARDWARE (engines, memory, interrupts)</pre>

The guest never contains Open-IOV code; the host driver is unmodified; the hypervisor is unmodified. The only new component is <code>vgsp.ko</code>.

== 3. How each side works ==

* '''Guest side.''' The stock driver performs the GSP boot handshake, sets up RPC message queues in guest memory, creates RM objects, programs channels and submits work — exactly as on bare metal ([[The Guest Stack and CUDASW|<code>Guest-Stack-and-CUDASW.md</code>]], [[CUDA Userspace Driver|<code>CUDA-Userspace-Driver.md</code>]], [[UVM in the Guest|<code>UVM-in-the-Guest.md</code>]]).
* '''Host module.''' <code>vgsp.ko</code> is a VFIO mediated-device driver: it registers a parent on the physical GPU, exposes virtual GPU types, and each mdev is one virtual GPU for one VM. Inside, it presents a plausible Turing-class device (config space, BAR shadows, MSI-X), answers the GSP boot handshake and the RPC protocol with well-formed replies, and maintains the memory model that resolves guest physical addresses to host addresses ([[The Host Module (vgsp.ko)|<code>Host-Module.md</code>]]).
* '''The interface that makes this possible.''' Since Turing, the driver's management path is not ''silicon'', it is ''firmware'': the GSP runs a version of NVIDIA's Resource Manager and the driver talks to it over shared-memory queues with a small RPC protocol. That protocol is small, stable and observable — so the project emulates '''the firmware's interface''' instead of emulating a GPU ([[GSP and GSP-RM|<code>GSP-and-GSP-RM.md</code>]], [https://git.vgpu.io/open-iov/blob/master/docs/03-architecture/03-gsp-and-gsp-rm.md <code>../docs/03-architecture/03-gsp-and-gsp-rm.md</code>]).
* '''Execution.''' Guest work that must be fast (copies, kernels) is forwarded to a real pipeline driven through the module's own GSP-RM session ([[The vGSP Execution Plane (vEP)|<code>Execution-Plane.md</code>]], [[Physical GPU Backend and Guest Wiring|<code>Physical-GPU-Backend.md</code>]], [[The Execution Frontier|<code>Execution-Frontier.md</code>]]).

== 4. Data flows ==

<pre>MMIO from the guest:
guest nvidia.ko writes BAR0 offset X
-> KVM exit -> QEMU -> VFIO region write into the mdev
-> BAR0 shadow state machine (doorbells, VF registers, notify)
-> may wake the emulation thread

RPC from the guest to "firmware":
guest writes a command-queue element into a guest-memory msgq
-> doorbell (BAR0)
-> emulation thread reads the queue through the hypervisor's guest-RAM
accessor, parses the element, dispatches the RPC
-> handler produces a reply element in the status queue
-> emulation injects an MSI-X interrupt to the guest</pre>

== 5. What runs today ==

# Guest boots, enumerates the virtual PCIe device, binds <code>nvidia.ko</code>.
# The GSP boot handshake completes against the emulation.
# Message queues are set up, versions negotiated, RM initialization exchanges complete; object creation, capability queries, vaspaces, channels and the engine inventory are answered with well-formed replies (a small set of control IDs is special-cased with real values; reply fidelity for consumed fields keeps improving).
# <code>nvidia-smi -L</code> enumerates the device; CUDA userspace starts; context creation completes; the full driver-API smoke passes (<code>ALL OK</code>, 64-byte payload round-trips).
# CE-class copies are forwarded to the physical copy engine (default off) with engine-authored semaphore releases and a fault relay. For GR (graphics/compute), a forwarded guest launch reaches the front-end but the engine has not executed it yet — the current frontier ([[The Execution Frontier|<code>Execution-Frontier.md</code>]]).

== 6. Where the detail lives ==

* [[GSP and GSP-RM|<code>GSP-and-GSP-RM.md</code>]] — the firmware/RM architecture and the transport we emulate.
* [[The Host Module (vgsp.ko)|<code>Host-Module.md</code>]] — the module's internals.
* [[The Guest Stack and CUDASW|<code>Guest-Stack-and-CUDASW.md</code>]], [[CUDA Userspace Driver|<code>CUDA-Userspace-Driver.md</code>]] — the guest side.
* [[Introduction to GMMU|<code>Introduction-to-GMMU.md</code>]], [[UVM in the Guest|<code>UVM-in-the-Guest.md</code>]] — memory and translation.
* [[The vGSP Execution Plane (vEP)|<code>Execution-Plane.md</code>]], [[Physical GPU Backend and Guest Wiring|<code>Physical-GPU-Backend.md</code>]], [[The Execution Frontier|<code>Execution-Frontier.md</code>]] — work and execution.
* [[Hardware and Software Matrix|<code>Hardware-Software-Matrix.md</code>]] — supported combinations and portability.
* [https://git.vgpu.io/open-iov/blob/master/docs/03-architecture/01-system-overview.md <code>../docs/03-architecture/01-system-overview.md</code>] and the rest of the architecture corpus in the repository.