The Guest Stack and CUDASW - Revision history

WikiBot: Fix repository links: point to the cgit tree URLs

2026-10-12T04:40:56Z

Fix repository links: point to the cgit tree URLs

← Older revision		Revision as of 04:40, 12 October 2026
Line 3:		Line 3:
	= The Guest Stack and CUDASW =		= The Guest Stack and CUDASW =

	<blockquote>'''Scope.''' What runs inside the guest, how the NVIDIA software layers sit on top of the device, and what the unmodified guest driver expects from the emulated GPU. Canonical source: [https://git.vgpu.io/open-iov/~~blob/master~~/docs/03-architecture/02-guest-stack-and-cudasw.md <code>../docs/03-architecture/02-guest-stack-and-cudasw.md</code>]; the docs tree wins on disagreement with this page.</blockquote>		<blockquote>'''Scope.''' What runs inside the guest, how the NVIDIA software layers sit on top of the device, and what the unmodified guest driver expects from the emulated GPU. Canonical source: [https://git.vgpu.io/open-iov.git/tree/docs/03-architecture/02-guest-stack-and-cudasw.md <code>../docs/03-architecture/02-guest-stack-and-cudasw.md</code>]; the docs tree wins on disagreement with this page.</blockquote>

	<blockquote>Status: 2026-10-12, tree <code>master @ 2f066727</code>. Tags: <code>[live]</code> observed with the stock guest driver; <code>[gated]</code> implemented default-off behind supervised gates; <code>[wall]</code> known blocking limit; <code>[target]</code> designed, not built; <code>[planned]</code> roadmap.</blockquote>		<blockquote>Status: 2026-10-12, tree <code>master @ 2f066727</code>. Tags: <code>[live]</code> observed with the stock guest driver; <code>[gated]</code> implemented default-off behind supervised gates; <code>[wall]</code> known blocking limit; <code>[target]</code> designed, not built; <code>[planned]</code> roadmap.</blockquote>
Line 143:		Line 143:

	* The guest's driver and the emulated firmware profile must agree on the RPC message versions. The guest negotiates these at start-up (<code>SET_GUEST_SYSTEM_INFO</code> carries the VGX/RPC version); the emulation answers with a profile that matches the driver family in use.		* The guest's driver and the emulated firmware profile must agree on the RPC message versions. The guest negotiates these at start-up (<code>SET_GUEST_SYSTEM_INFO</code> carries the VGX/RPC version); the emulation answers with a profile that matches the driver family in use.
	* CUDA versions ride on driver versions; the test matrix tracks (driver, CUDA, firmware) triples ([https://git.vgpu.io/open-iov/~~blob/master~~/docs/05-operations/01-performance-correctness-security.md <code>../docs/05-operations/01-performance-correctness-security.md</code>]).		* CUDA versions ride on driver versions; the test matrix tracks (driver, CUDA, firmware) triples ([https://git.vgpu.io/open-iov.git/tree/docs/05-operations/01-performance-correctness-security.md <code>../docs/05-operations/01-performance-correctness-security.md</code>]).

	== 9. Related documentation ==		== 9. Related documentation ==

	* [https://git.vgpu.io/open-iov/~~blob/master~~/docs/03-architecture/02-guest-stack-and-cudasw.md <code>../docs/03-architecture/02-guest-stack-and-cudasw.md</code>] — canonical chapter.		* [https://git.vgpu.io/open-iov.git/tree/docs/03-architecture/02-guest-stack-and-cudasw.md <code>../docs/03-architecture/02-guest-stack-and-cudasw.md</code>] — canonical chapter.
	* [[The Host Module (vgsp.ko)\|<code>Host-Module.md</code>]] — the other side of the boundary.		* [[The Host Module (vgsp.ko)\|<code>Host-Module.md</code>]] — the other side of the boundary.
	* [[UVM in the Guest\|<code>UVM-in-the-Guest.md</code>]], [[CUDA Userspace Driver\|<code>CUDA-Userspace-Driver.md</code>]] — user-space driver internals.		* [[UVM in the Guest\|<code>UVM-in-the-Guest.md</code>]], [[CUDA Userspace Driver\|<code>CUDA-Userspace-Driver.md</code>]] — user-space driver internals.
	* [[Guest Driver Lifecycle: A Field Guide\|<code>Driver-Lifecycle-Field-Guide.md</code>]], [[GSP and GSP-RM\|<code>GSP-and-GSP-RM.md</code>]] — the observed protocol reality.		* [[Guest Driver Lifecycle: A Field Guide\|<code>Driver-Lifecycle-Field-Guide.md</code>]], [[GSP and GSP-RM\|<code>GSP-and-GSP-RM.md</code>]] — the observed protocol reality.
	* [[Physical GPU Backend and Guest Wiring\|<code>Physical-GPU-Backend.md</code>]], [[The Execution Frontier\|<code>Execution-Frontier.md</code>]] — where guest work is headed.		* [[Physical GPU Backend and Guest Wiring\|<code>Physical-GPU-Backend.md</code>]], [[The Execution Frontier\|<code>Execution-Frontier.md</code>]] — where guest work is headed.

WikiBot: Carry the site-wide About/licensing message on every page (2026-10-12)

2026-10-12T04:29:46Z

Carry the site-wide About/licensing message on every page (2026-10-12)

← Older revision		Revision as of 04:29, 12 October 2026
Line 1:		Line 1:
			{{About message}}

	= The Guest Stack and CUDASW =		= The Guest Stack and CUDASW =

WikiBot: Add page from tracked wiki source (2026 IA update)

2026-10-12T04:22:10Z

Add page from tracked wiki source (2026 IA update)

New page

= The Guest Stack and CUDASW =

<blockquote>'''Scope.''' What runs inside the guest, how the NVIDIA software layers sit on top of the device, and what the unmodified guest driver expects from the emulated GPU. Canonical source: [https://git.vgpu.io/open-iov/blob/master/docs/03-architecture/02-guest-stack-and-cudasw.md <code>../docs/03-architecture/02-guest-stack-and-cudasw.md</code>]; the docs tree wins on disagreement with this page.</blockquote>

<blockquote>Status: 2026-10-12, tree <code>master @ 2f066727</code>. Tags: <code>[live]</code> observed with the stock guest driver; <code>[gated]</code> implemented default-off behind supervised gates; <code>[wall]</code> known blocking limit; <code>[target]</code> designed, not built; <code>[planned]</code> roadmap.</blockquote>

== 1. Vocabulary: what "CUDASW" means here ==

'''CUDASW''' is the family of NVIDIA ''software'' components that sit between an application and the GPU driver. The project's compatibility target is the whole stack, not one library:

<pre>application
│ CUDA API (runtime)
libcudart.so ─────────────────────────────┐ runtime: device/stream/event
│ CUDA driver API │ management, kernel launch
libcuda.so ───────────────────────────────┤ driver: contexts, modules,
│ ioctls │ memory, channel submission
/dev/nvidia* ─────────────────────────────┘
nvidia.ko (RM + UVM + modeset)</pre>

Adjacent members of the family: '''libnvidia-ptxjitcompiler''' (PTX→SASS JIT — usually "just works" once the driver initializes), '''libnvidia-fatbinaryloader / nvrtc''', '''UVM''' (<code>nvidia_uvm.ko</code> — [[UVM in the Guest|<code>UVM-in-the-Guest.md</code>]]), the '''graphics stack''' (<code>libGLX_nvidia</code>, <code>libnvidia-glcore</code>, Vulkan ICD, <code>nvidia_modeset</code>/<code>nvidia_drm</code>) and the '''video stack''' (NVENC/NVDEC) — both roadmapped.

== 2. Layering rules the emulation must respect ==

# '''The kernel driver is the contract.''' Everything above it talks to <code>nvidia.ko</code> through stable ioctls, memory mappings and <code>/dev/nvidia*</code> nodes. If the kernel driver initializes and behaves, user space follows.
# '''User space is opaque to us.''' We never intercept user-space APIs. All hooks are in the device model (host side) or in the ''hardware interface'' the kernel driver sees (guest side).
# '''Version skew is normal.''' Driver version, CUDA version and firmware version are three independent axes. The compatibility target is "the driver family that matches the emulated firmware profile", with configuration-driven adaptation for others.

== 3. Kernel-driver anatomy (what the guest brings) ==

{| class="wikitable"
|-
! Component
! Role
! Why it matters here
|-
| RM core
| object model, handles, locks, device lifecycle
| drives the GSP boot and the whole RPC/control surface
|-
| Resource server (<code>resserv</code>)
| allocates/tracks objects, memory, channels
| the client half of the client/server RM split
|-
| Memory manager (<code>memmgr</code>)
| VRAM/sysmem objects, mappings, page tables
| the guest expects the GSP to own page-table leaves for its VA spaces
|-
| FIFO/runlist
| channels, TSGs, GPFIFO, USERD, doorbells
| where the guest's work is submitted
|-
| GSP client
| message queues, RPC transport, event handling
| the interface we emulate
|-
| UVM
| unified memory, page faults, channel manager
| starts early in CUDA init, extremely stateful
|-
| <code>nvidia_modeset</code> / <code>nvidia_drm</code>
| display / DRM
| needed later for graphics
|-
| CE / GR HALs
| engine programming
| needed to ''execute'' forwarded work
|}

== 4. Driver initialization vs runtime ==

Understanding this split is the key to debugging.

'''Initialization (boot/probe)''' — deterministic, mostly one-shot; where compatibility is won or lost [live]:

<pre>PCI probe → identify GPU from config space and BAR0 registers
→ boot the GSP (our emulated handshake)
→ read static configuration (device info tables, FB/BUS info, VBIOS, clocks)
→ create root client/device/subdevice objects
→ initialize RM subsystems (interrupts, FIFO, memory, GMMU, PMU)
→ expose /dev/nvidia* and register with the kernel</pre>

The actual message/control IDs observed at each step are listed in [[Guest Driver Lifecycle: A Field Guide|<code>Driver-Lifecycle-Field-Guide.md</code>]].

'''Runtime (per-process, per-context)''' — driven by user-space workloads; where ''execution'' is won or lost:

<pre>open /dev/nvidia* → cuInit → UVM registration (per process)
→ cuCtxCreate → object/allocations → memory maps
→ channel creation + push buffers + semaphores
→ kernel launches (GR) / copies (CE) / memops
→ faults, completions, events</pre>

Context creation and the full CUDA driver-API smoke now pass (<code>ALL OK</code>, copies matched) [live]. The current focus is execution: the backend is a '''module-owned GPU running its real GSP firmware as the resource-manager server''' ([[Physical GPU Backend and Guest Wiring|<code>Physical-GPU-Backend.md</code>]]), with CE-class copies forwarded to the physical copy engine default-off [gated]. For GR, a forwarded guest launch reaches the front-end but the inline launch-descriptor install has not been observed — the open execution wall ([[The Execution Frontier|<code>Execution-Frontier.md</code>]]) [wall].

== 5. What the guest expects from the device, per phase ==

{| class="wikitable"
|-
! Phase
! Guest action
! Device/firmware expectation
|-
| Probe
| config space reads, BAR sizing, MSI-X capability
| a plausible TU104-family PCIe GPU
|-
| Boot
| BAR0 register writes, WPR setup, firmware boot sequence
| GSP boots, booter completes, "GSP init done"
|-
| Config
| RPC <code>GET_GSP_STATIC_INFO</code>, device tables, VBIOS reads
| consistent, self-describing tables
|-
| RM init
| <code>GSP_RM_CONTROL</code> battery (FB/BUS/CE/INTR/GMMU/...)
| correct shapes and plausible values
|-
| Objects
| <code>GSP_RM_ALLOC</code> (root, device, subdevice, vaspace, channel, CE objects)
| success + correct handles/params
|-
| Runtime
| allocs/controls for maps and channels; pushes
| executed work, semaphores, faults
|}

The RPC surface is described in [[GSP and GSP-RM|<code>GSP-and-GSP-RM.md</code>]] and [[RPC Message Format|<code>RPC-Message-Format.md</code>]]; the boot handshake in [[WPR Management|<code>WPR-Management.md</code>]].

== 6. Why user-space-only interception does not work ==

Two hard-won facts from this project:

# '''<code>libcuda</code> issues raw <code>syscall</code> instructions''', so an <code>LD_PRELOAD</code> shim around <code>ioctl()</code> sees nothing. Observing CUDA's kernel calls requires a <code>ptrace</code>-based tracer (PTRACE_SYSCALL + PTRACE_GET_SYSCALL_INFO), as the project's diagnostics do.
# **The CUDA driver talks to the ''kernel driver'', not to the device.** All device interaction happens in <code>nvidia.ko</code>. This is good news: the emulation boundary is a small set of kernel-side interfaces (MMIO, DMA, interrupts, the GSP protocol) rather than a large user-space API surface.

== 7. Windows guests ==

Windows brings its own kernel-mode driver, <code>nvlddmkm.sys</code>, which uses the same underlying GSP device/firmware model but different host interfaces (WDDM, MSI-X semantics, ROM/VBIOS quirks). The emulation target is the same: satisfy the firmware interface, forward the work. Windows support is a roadmap item ([[Hardware and Software Matrix|<code>Hardware-Software-Matrix.md</code>]]) [planned], and the design goal is that no Windows-specific code lives in the guest.

== 8. Compatibility notes for versions ==

* The guest's driver and the emulated firmware profile must agree on the RPC message versions. The guest negotiates these at start-up (<code>SET_GUEST_SYSTEM_INFO</code> carries the VGX/RPC version); the emulation answers with a profile that matches the driver family in use.
* CUDA versions ride on driver versions; the test matrix tracks (driver, CUDA, firmware) triples ([https://git.vgpu.io/open-iov/blob/master/docs/05-operations/01-performance-correctness-security.md <code>../docs/05-operations/01-performance-correctness-security.md</code>]).

== 9. Related documentation ==

* [https://git.vgpu.io/open-iov/blob/master/docs/03-architecture/02-guest-stack-and-cudasw.md <code>../docs/03-architecture/02-guest-stack-and-cudasw.md</code>] — canonical chapter.
* [[The Host Module (vgsp.ko)|<code>Host-Module.md</code>]] — the other side of the boundary.
* [[UVM in the Guest|<code>UVM-in-the-Guest.md</code>]], [[CUDA Userspace Driver|<code>CUDA-Userspace-Driver.md</code>]] — user-space driver internals.
* [[Guest Driver Lifecycle: A Field Guide|<code>Driver-Lifecycle-Field-Guide.md</code>]], [[GSP and GSP-RM|<code>GSP-and-GSP-RM.md</code>]] — the observed protocol reality.
* [[Physical GPU Backend and Guest Wiring|<code>Physical-GPU-Backend.md</code>]], [[The Execution Frontier|<code>Execution-Frontier.md</code>]] — where guest work is headed.