WikiBot: Fix repository links: point to the cgit tree URLs

2026-10-12T04:40:57Z

Fix repository links: point to the cgit tree URLs

← Older revision		Revision as of 04:40, 12 October 2026
Line 3:		Line 3:
	= The Host Module (vgsp.ko) =		= The Host Module (vgsp.ko) =

	<blockquote>'''Scope.''' The host-side kernel module that presents one virtual Turing-class GPU per <code>mdev</code> instance: the virtual device, the emulation engine, the guest-memory model and the work-forwarding design. Canonical source: [https://git.vgpu.io/open-iov/~~blob/master~~/docs/03-architecture/06-host-module-and-hybrid-model.md <code>../docs/03-architecture/06-host-module-and-hybrid-model.md</code>]; the docs tree wins on disagreement with this page.</blockquote>		<blockquote>'''Scope.''' The host-side kernel module that presents one virtual Turing-class GPU per <code>mdev</code> instance: the virtual device, the emulation engine, the guest-memory model and the work-forwarding design. Canonical source: [https://git.vgpu.io/open-iov.git/tree/docs/03-architecture/06-host-module-and-hybrid-model.md <code>../docs/03-architecture/06-host-module-and-hybrid-model.md</code>]; the docs tree wins on disagreement with this page.</blockquote>

	<blockquote>Status: 2026-10-12, tree <code>master @ 2f066727</code>. Tags: <code>[live]</code> observed with the stock guest driver; <code>[wip]</code> demonstrated live, being productized; <code>[gated]</code> implemented default-off behind supervised gates; <code>[wall]</code> known blocking limit; <code>[target]</code> designed, not built; <code>[planned]</code> roadmap.</blockquote>		<blockquote>Status: 2026-10-12, tree <code>master @ 2f066727</code>. Tags: <code>[live]</code> observed with the stock guest driver; <code>[wip]</code> demonstrated live, being productized; <code>[gated]</code> implemented default-off behind supervised gates; <code>[wall]</code> known blocking limit; <code>[target]</code> designed, not built; <code>[planned]</code> roadmap.</blockquote>
Line 107:		Line 107:
	\|}		\|}

	The physical-backend path adds a default-off set of backend controls (documented in the repository; workflows in [https://git.vgpu.io/open-iov/~~blob/master~~/docs/05-operations/02-debugging-and-maintenance-workflows.md <code>../docs/05-operations/02-debugging-and-maintenance-workflows.md</code>]).		The physical-backend path adds a default-off set of backend controls (documented in the repository; workflows in [https://git.vgpu.io/open-iov.git/tree/docs/05-operations/02-debugging-and-maintenance-workflows.md <code>../docs/05-operations/02-debugging-and-maintenance-workflows.md</code>]).

	== 7. Build and symbol contracts ==		== 7. Build and symbol contracts ==
Line 152:		Line 152:
	== 9. Related documentation ==		== 9. Related documentation ==

	* Canonical chapter: [https://git.vgpu.io/open-iov/~~blob/master~~/docs/03-architecture/06-host-module-and-hybrid-model.md <code>../docs/03-architecture/06-host-module-and-hybrid-model.md</code>] — the source of this condensation.		* Canonical chapter: [https://git.vgpu.io/open-iov.git/tree/docs/03-architecture/06-host-module-and-hybrid-model.md <code>../docs/03-architecture/06-host-module-and-hybrid-model.md</code>] — the source of this condensation.
	* [[GSP RPC Emulation\|<code>GSP-RPC-Emulation.md</code>]], [[BAR Emulation Specification\|<code>BAR-Emulation-Specification.md</code>]], [[WPR Management\|<code>WPR-Management.md</code>]]; [[Physical GPU Backend and Guest Wiring\|<code>Physical-GPU-Backend.md</code>]], [[The Execution Frontier\|<code>Execution-Frontier.md</code>]], [[The Guest Stack and CUDASW\|<code>Guest-Stack-and-CUDASW.md</code>]].		* [[GSP RPC Emulation\|<code>GSP-RPC-Emulation.md</code>]], [[BAR Emulation Specification\|<code>BAR-Emulation-Specification.md</code>]], [[WPR Management\|<code>WPR-Management.md</code>]]; [[Physical GPU Backend and Guest Wiring\|<code>Physical-GPU-Backend.md</code>]], [[The Execution Frontier\|<code>Execution-Frontier.md</code>]], [[The Guest Stack and CUDASW\|<code>Guest-Stack-and-CUDASW.md</code>]].

WikiBot: Carry the site-wide About/licensing message on every page (2026-10-12)

2026-10-12T04:29:48Z

Carry the site-wide About/licensing message on every page (2026-10-12)

← Older revision		Revision as of 04:29, 12 October 2026
Line 1:		Line 1:
			{{About message}}

	= The Host Module (vgsp.ko) =		= The Host Module (vgsp.ko) =

WikiBot: Add page from tracked wiki source (2026 IA update)

2026-10-12T04:22:12Z

Add page from tracked wiki source (2026 IA update)

New page

= The Host Module (vgsp.ko) =

<blockquote>'''Scope.''' The host-side kernel module that presents one virtual Turing-class GPU per <code>mdev</code> instance: the virtual device, the emulation engine, the guest-memory model and the work-forwarding design. Canonical source: [https://git.vgpu.io/open-iov/blob/master/docs/03-architecture/06-host-module-and-hybrid-model.md <code>../docs/03-architecture/06-host-module-and-hybrid-model.md</code>]; the docs tree wins on disagreement with this page.</blockquote>

<blockquote>Status: 2026-10-12, tree <code>master @ 2f066727</code>. Tags: <code>[live]</code> observed with the stock guest driver; <code>[wip]</code> demonstrated live, being productized; <code>[gated]</code> implemented default-off behind supervised gates; <code>[wall]</code> known blocking limit; <code>[target]</code> designed, not built; <code>[planned]</code> roadmap.</blockquote>

== 1. Role and position ==

<pre>guest VM ── PCIe MMIO / DMA / MSI-X through the hypervisor
(KVM/QEMU or Xen: VFIO mdev transport, IOMMU)
│
vgsp.ko: virtual device · emulation core · memory model · work path
│
physical GPU + GSP firmware, driven by this module (host driver unmodified)</pre>

The module registers as an '''mdev parent''' on the physical GPU, exposing one device type per supported profile; each <code>mdev</code> is an independent virtual GPU ([[System Overview|<code>System-Overview.md</code>]], [[The vGSP Execution Plane (vEP)|<code>Execution-Plane.md</code>]]).

== 2. The virtual device ==

The guest sees a plausible Turing-class GPU (IDs, class code, BARs, capability list). Some fields exist only for validation: PCIe link capabilities (the hypervisor can rewrite them), MSI-X masking state, and non-SR-IOV VF windows.

{| class="wikitable"
|-
! BAR
! Size
! Backing
! Contents
|-
| BAR0
| 16 MB
| host <code>vzalloc</code> shadow
| boot, doorbells, VF registers, PRAMIN window, VBIOS
|-
| BAR1
| 256 MB
| host <code>vzalloc</code> vidmem buffer
| guest-visible VRAM aperture
|-
| BAR3
| —
| instance window
| instance/PRAMIN behaviour
|}

MMIO is served by VFIO region callbacks or directly from the shadow when mapped into the guest. BAR1 is host RAM today — correctness first: the real GPU cannot read it, so forwarding must later migrate objects to real VRAM or expose DMA-mapped system memory [target] ([[Physical GPU Backend and Guest Wiring|<code>Physical-GPU-Backend.md</code>]]).

== 3. The emulation core ==

A kernel thread drives the device's "firmware" behaviour [live]:

<pre>BAR0 doorbell / VF-register write → read msgq element from guest memory
→ dispatch RPC → write reply → inject MSI-X (eventfd → irqfd)</pre>

Guest memory is reached via <code>vfio_dma_rw()</code> from kernel-thread context. The RPC philosophy: correct shape first (or the driver walks off into uninitialized memory), plausible values second (values must be self-consistent), never invent input (in/out parameter blocks are echoed; zeroing the <code>GSP_RM_ALLOC</code> reply once faulted <code>libcuda</code>). Handlers cover boot/static-info exchanges, device/FB/BUS/INTR tables, CE and GMMU queries, reserved-PDE contracts, channel/CE allocation and an unknown-control fallback. Push buffers are decoded as method streams and executed in software — memsets, copies, semaphore releases — a '''correctness bridge, not the data plane''' [wip]; emulation-mode CUDA is verified end to end (<code>vectorAdd</code> over 1M elements, <code>bad=0</code>) [live]. Physical-GSP boot is default-off [gated]; the firmware analyzer, version tables, HAL tables and dynamic RPC dispatcher are not yet on the live path [target].

== 4. The guest-memory model ==

'''What the client gives us:''' channel memory descriptors (<code>NV_MEMORY_DESC_PARAMS</code>) with ''physical'' addresses (instance memory, method buffer, userd, ramfc, error notifiers) and the page-table contract for server-owned VA ranges (<code>COPY_SERVER_RESERVED_PDES</code> — table-level PAs, page shifts, VA range, ''empty'' leaves). It gives us none of its own sysmem maps.

'''What we derive instead''' [live] — the module rebuilds the missing mappings from the guest's own bookkeeping, with no cooperation from the guest driver:

# '''VA↔PA pairs live in guest memory''': UVM and RM record both the GPU VA and the kernel CPU VA per object (<code>uvm_rm_mem_t.vas[CPU]/vas[GPU]</code> +0x30/+0x38).
# '''CPU VAs are direct-map addresses''': <code>PA = cpu_va - page_offset_base</code>, the base recovered from guest memory or <code>/proc/kallsyms</code> during development.
# '''Exception:''' control-device <code>mmap()</code> pages have no leaf PTEs; verified pairs arrive via <code>sema_map_add</code> until a durable join exists. The page-granular cache is fed by a trusted GMMU walk ([[Introduction to GMMU|<code>Introduction-to-GMMU.md</code>]]).

'''Load-bearing copy invariants:''' (1) a copy runs at most once per GPFIFO entry — replays are suppressed by entry-word identity plus an executed-operation bitmap; (2) a virtual operand translates per 4 KB page (195 of 256 pages were contiguous in one observed 1 MB range); (3) a consumed slot's work never runs later — the per-put-epoch mark is authoritative until a put wrap.

== 5. The forwarding path (design) ==

<pre>guest push submitted
→ detect (doorbell / GPPUT / push-buffer scan), decode the method stream
→ EMULATE state/notification (release, membar, fence) or
FORWARD real engine work (GR launch, CE copy, memop)
→ FORWARD: submit equivalent work on a real channel bound to the physical GPU
→ relay completions (semaphore / fault / interrupt) back into guest memory</pre>

Proxying through the distribution host driver has '''no callable entry point from kernel context''', so the module '''owns the device and runs its real GSP firmware as the resource-manager server''', with emulation as the control plane. CE copies, release and fault relay are implemented default-off and verified live [gated]; GR execution is the open wall [wall] — a forwarded launch reaches the front-end; the inline launch-descriptor install is unobserved ([[The Execution Frontier|<code>Execution-Frontier.md</code>]]).

== 6. Debug parameters ==

<code>/sys/module/vgsp/parameters/*</code> makes guest memory and the emulated device inspectable without rebuilding:

{| class="wikitable"
|-
! Parameter
! Purpose
|-
| <code>scan_guest_ram=<v></code> / <code>scan_vidmem=<v></code>
| find a 64-bit value in guest RAM (0–8 GB) or the VRAM shadow
|-
| <code>scan_seq=<v1,...></code> / <code>scan_pt=<pa></code> / <code>scan_masked=<v>,<mask></code>
| find a 32-bit sequence, entries pointing at a PA, or <code>(x & mask) == v</code>
|-
| <code>dump_vidmem=<off>,<len></code> / <code>fill_vidmem=<v></code>
| dump/scan or fill the VRAM shadow
|-
| <code>poke_guest=<gpa>,<v></code> / <code>poke_fill=<gpa>,<v>,<n></code>
| write a 32-bit value / fill a range in guest physical memory
|-
| <code>sema_probe=<gpu_va>[,count]</code> / <code>sema_map_add=<va>,<pa></code>
| probe a semaphore VA / teach one verified VA→PA pair
|-
| <code>dump_guest=<gpa>,<len></code>
| dump guest physical memory (capped)
|}

The physical-backend path adds a default-off set of backend controls (documented in the repository; workflows in [https://git.vgpu.io/open-iov/blob/master/docs/05-operations/02-debugging-and-maintenance-workflows.md <code>../docs/05-operations/02-debugging-and-maintenance-workflows.md</code>]).

== 7. Build and symbol contracts ==

* <code>vgsp.ko</code> exports the <code>nvidia_vgpu_vfio_*</code> interface (<code>probe</code>/<code>remove</code>, ops get/set, mdev parent registration) to pair the device with OpenRM host modules.
* OpenRM modules built from source need <code>KBUILD_EXTRA_SYMBOLS=$(pwd)/kernel/Module.symvers</code> and must build '''after''' this module (missing symbols and stale build caches are common otherwise).
* On single-GPU developer systems the distribution driver owns the physical GPU and the module runs standalone; the OpenRM path is only for native-vGPU management, not QEMU.

== 8. Implemented vs designed ==

{| class="wikitable"
|-
! Area
! Status
|-
| mdev/VFIO device, config space, BARs, MSI-X; BAR0 state machine; boot handshake [live]
| implemented
|-
| Message queues + RPC + init controls (stock 570.144) [live]
| implemented
|-
| Guest sysmem access + VA↔PA discovery [wip]
| debug-grade, to be productized
|-
| Push decoding + semaphore completion [wip]
| demonstrated live, being productized
|-
| Emulation-mode CUDA end-to-end [live]
| repeatable 1M-element <code>vectorAdd</code>, <code>bad=0</code>
|-
| Guest-RAM physical path + server-owned page-table population [gated]
| default-off; GPA→HPA via shared-file page cache verified live on the CE
|-
| GPU work forwarding: CE copy, release and fault relay [gated]
| default-off, verified live; GR open [wall]
|-
| Execution backend (module-owned GPU, direct real-GSP session) [wall]
| boot, static configuration and CE copies proven; GR execution open (failed mutating attempts park the device in a locked fail-safe, never auto-retried)
|-
| Real VRAM backing [target]; Windows, Xen, aarch64 [planned]
| designed, not implemented
|}

== 9. Related documentation ==

* Canonical chapter: [https://git.vgpu.io/open-iov/blob/master/docs/03-architecture/06-host-module-and-hybrid-model.md <code>../docs/03-architecture/06-host-module-and-hybrid-model.md</code>] — the source of this condensation.
* [[GSP RPC Emulation|<code>GSP-RPC-Emulation.md</code>]], [[BAR Emulation Specification|<code>BAR-Emulation-Specification.md</code>]], [[WPR Management|<code>WPR-Management.md</code>]]; [[Physical GPU Backend and Guest Wiring|<code>Physical-GPU-Backend.md</code>]], [[The Execution Frontier|<code>Execution-Frontier.md</code>]], [[The Guest Stack and CUDASW|<code>Guest-Stack-and-CUDASW.md</code>]].

The Host Module (vgsp.ko) - Revision history

WikiBot: Fix repository links: point to the cgit tree URLs

WikiBot: Carry the site-wide About/licensing message on every page (2026-10-12)

WikiBot: Add page from tracked wiki source (2026 IA update)