WikiBot: Fix repository links: point to the cgit tree URLs

2026-10-12T04:40:55Z

Fix repository links: point to the cgit tree URLs

← Older revision		Revision as of 04:40, 12 October 2026
Line 3:		Line 3:
	= GSP and GSP-RM =		= GSP and GSP-RM =

	<blockquote>'''What this page is.''' The GPU System Processor hosts NVIDIA's Resource Manager in firmware; since Turing, the guest driver's management path is ''talking to that firmware'', not to silicon. This page explains that architecture and the interface the project emulates. Condensed from [https://git.vgpu.io/open-iov/~~blob/master~~/docs/03-architecture/03-gsp-and-gsp-rm.md <code>../docs/03-architecture/03-gsp-and-gsp-rm.md</code>]; the docs tree wins on disagreement.</blockquote>		<blockquote>'''What this page is.''' The GPU System Processor hosts NVIDIA's Resource Manager in firmware; since Turing, the guest driver's management path is ''talking to that firmware'', not to silicon. This page explains that architecture and the interface the project emulates. Condensed from [https://git.vgpu.io/open-iov.git/tree/docs/03-architecture/03-gsp-and-gsp-rm.md <code>../docs/03-architecture/03-gsp-and-gsp-rm.md</code>]; the docs tree wins on disagreement.</blockquote>

	Status: 2026-10-12, tree <code>master @ 2f066727</code>. Tags: <code>[live]</code> proven with the stock guest driver; <code>[wip]</code> partial; <code>[wall]</code> current blocker; <code>[target]</code> designed end state.		Status: 2026-10-12, tree <code>master @ 2f066727</code>. Tags: <code>[live]</code> proven with the stock guest driver; <code>[wip]</code> partial; <code>[wall]</code> current blocker; <code>[target]</code> designed end state.

WikiBot: Carry the site-wide About/licensing message on every page (2026-10-12)

2026-10-12T04:29:46Z

Carry the site-wide About/licensing message on every page (2026-10-12)

← Older revision		Revision as of 04:29, 12 October 2026
Line 1:		Line 1:
			{{About message}}

	= GSP and GSP-RM =		= GSP and GSP-RM =

WikiBot: Add page from tracked wiki source (2026 IA update)

2026-10-12T04:22:10Z

Add page from tracked wiki source (2026 IA update)

New page

= GSP and GSP-RM =

<blockquote>'''What this page is.''' The GPU System Processor hosts NVIDIA's Resource Manager in firmware; since Turing, the guest driver's management path is ''talking to that firmware'', not to silicon. This page explains that architecture and the interface the project emulates. Condensed from [https://git.vgpu.io/open-iov/blob/master/docs/03-architecture/03-gsp-and-gsp-rm.md <code>../docs/03-architecture/03-gsp-and-gsp-rm.md</code>]; the docs tree wins on disagreement.</blockquote>

Status: 2026-10-12, tree <code>master @ 2f066727</code>. Tags: <code>[live]</code> proven with the stock guest driver; <code>[wip]</code> partial; <code>[wall]</code> current blocker; <code>[target]</code> designed end state.

== 1. What the GSP is ==

The '''GSP''' is a small embedded RISC-V processor (RV64IMAC) inside NVIDIA GPUs. It runs a firmware image (for the 570.144 generation, a ~28 MB ELF) that contains, among other things, a full port of NVIDIA's '''Resource Manager (RM)''' — the component that owns GPU initialization, memory, engines, interrupts and scheduling.

<pre> ┌──────────────────────────── GUEST ────────────────────────────┐
│ nvidia.ko │
│ RM client path ── RPC ──┐ │
│ GSP boot code │ │
└────────────────────────────┼──────────────────────────────────┘
│ shared memory queues + doorbells
┌────────────────────────────┼──────────────────────────────────┐
│ GSP (firmware, RISC-V) │ │
│ RM core (RMCore) ◄─────┘ │
│ RM API (object model, clients, handles) │
│ Resource server (resserv): allocation & lifetime │
│ memory manager, GMMU, FIFO/runlist, interrupts, PMU │
└───────────────────────────────────────────────────────────────┘</pre>

A driver as complex as NVIDIA's is not written to program silicon directly; it is written to talk to an RM, and on modern GPUs a copy of the RM runs on the GSP. The driver is therefore an '''RM client''': it issues requests, receives callbacks, and observes state through a well-defined protocol.

== 2. The boot handshake we emulate ==

The guest driver boots the GSP exactly as it would on bare metal:

# '''WPR setup''' — carve a firmware-protected region of VRAM and write descriptors into BAR0 registers ([[WPR Management|<code>WPR-Management.md</code>]]).
# '''FWSEC''' — the driver verifies firmware signatures.
# '''Booter''' — the boot ROM consumes the firmware image.
# '''RISCV start''' — the RISC-V core begins executing the firmware.
# '''GSP init done''' — the firmware signals readiness through a status mailbox; the driver proceeds to initialization RPCs.

The emulation implements '''stage completion''' rather than the stages themselves — legitimate because the only observable to the driver is the state machine it polls. The same handshake has also been run against real firmware on the physical backend: the firmware comes up, delivers its first status events, serves the static configuration, and stays live until the unload RPC releases it ([[Physical GPU Backend and Guest Wiring|<code>Physical-GPU-Backend.md</code>]]).

== 3. The transport: message queues and RPC ==

GSP-RM's transport is a shared-memory message queue pair:

<pre> guest system memory (allocated by the guest driver, registered with us)
┌───────────────────────────────┐
│ command queue (guest -> GSP) │ element ring, 4 KB elements
├───────────────────────────────┤
│ status queue (GSP -> guest) │ element ring
└───────────────────────────────┘
+ a page table describing the queue pages (the buffer is
non-contiguous: each page is an independent allocation)</pre>

Each element begins with a transport header followed by an '''RPC message''' whose 32-byte header carries header version, length, function, result and sequence number. Observations worth knowing (all verified against the stock guest):

* <code>length</code> covers the fixed part of the message, not variable payloads — do not use it to bound a payload;
* function numbers are a flat enum shared by several protocols (<code>SET_GUEST_SYSTEM_INFO</code> 1, <code>FREE</code> 10, <code>GET_GSP_STATIC_INFO</code> 65, <code>GSP_RM_CONTROL</code> 76, <code>GSP_RM_ALLOC</code> 103, ...);
* replies must be matched to request sequence numbers — the driver runs a reply-matching state machine, and a slow reply is indistinguishable from a hang.

== 4. RMCore, the RM API and the resource server ==

Three RM concepts make every log readable:

* '''RMCore''' — the object system. Everything is an object with a class (a 32-bit class ID), a handle in a per-client handle space, and a parent object forming a tree rooted at a client. Traces of <code>GSP_RM_ALLOC</code> are object creation; traces of <code>GSP_RM_CONTROL</code> are method calls on objects.
* '''RM API''' — how in-kernel components (and user space via ioctls) allocate objects and call controls, with an implicit-handle and an explicit-handle variant. For a GSP client, part of the API becomes RPCs while some calls are handled locally; memory objects, for example, are allocated locally by the client RM.
* '''Resource server (resserv)''' — the allocation authority: it owns handle spaces, enforces privileges and tracks object lifetimes.

== 5. The client/server split (and why page tables matter) ==

Modern RM is client/server: the '''client''' (guest <code>nvidia.ko</code>) owns per-process state and submits work; the '''server''' (GSP-RM) owns GPU-global state — routing, page tables, scheduling domains, engine configuration. For virtualized setups the client tells the server which VA ranges the server owns, and the server manages the deeper page-table levels for those ranges. The client hands over the '''physical addresses of its page-table levels''' plus the VA ranges they cover; on real hardware the server then fills in the lower levels for server-owned ranges and the client never writes them.

This is a central fact for the execution work: when the guest's UVM asks the "GPU" to map memory, the mapping is expected to materialize in page tables that only the server side can write — which is us ([[UVM in the Guest|<code>UVM-in-the-Guest.md</code>]], [[Introduction to GMMU|<code>Introduction-to-GMMU.md</code>]]).

== 6. Event channels and notifications ==

Beyond request/reply, the guest needs asynchronous events: event objects registered with a notification configuration (an MSI-X vector or a memory "semaphore" the GPU can write), completion events when channels finish work or errors occur, delivered as a strict bit/handle/state triple on the exact vector. The emulation's MSI-X path is emulation thread → <code>eventfd</code> → KVM → guest vector, with the pending-bit array emulated so a masked vector's interrupt is not lost.

== 7. What we emulate today ==

{| class="wikitable"
|-
! Layer
! State
! Notes
|-
| GSP boot state machine
| <code>[live]</code>
| stages complete; the driver's poll loop is satisfied
|-
| Message queue transport
| <code>[live]</code>
| including the non-contiguous page table for the queue buffer
|-
| RPC request/reply
| <code>[live]</code>
| every function the driver uses during init is answered; unknown functions get a well-formed success reply
|-
| Static info tables
| <code>[wip]</code>
| device/FB/BUS/INTR/GMMU tables carry plausible reference values; the engine inventory is complete and agrees with the class list
|-
| Controls during init
| <code>[wip]</code>
| ~160 distinct control IDs observed per CUDA run; a small set is special-cased with real values and reply fidelity for the consumed fields keeps improving
|-
| Object creation
| <code>[wip]</code>
| handles/parameters are echoed consistently (zeroing the request parameters corrupts the caller); there is no server-side object model yet
|-
| Page tables for server-owned ranges
| <code>[wip]</code>
| contract received and recorded; leaf population is the execution work
|-
| Event/notification delivery
| <code>[wip]</code>
| MSI-X works; richer event semantics land with forwarding
|-
| Firmware behaviour (routing, scheduling, real execution)
| <code>[wall]</code>
| the execution frontier — see [[The Execution Frontier|<code>Execution-Frontier.md</code>]]
|}

== 8. Where the firmware knowledge lives ==

* [[GSP 570.144W Firmware Analysis|<code>GSP-570.144W-Firmware-Analysis.md</code>]] — how the 28.5 MB image was decompiled and mapped.
* [[Address Reference|<code>Address-Reference.md</code>]] — firmware-side address tables.
* [[RPC Message Format|<code>RPC-Message-Format.md</code>]], [[GSP RPC Emulation|<code>GSP-RPC-Emulation.md</code>]] — historical RPC analysis (superseded; kept for the record).
* [[WPR Management|<code>WPR-Management.md</code>]] — write-protected region semantics.
* [[The vGSP Execution Plane (vEP)|The execution plane]] and [[The Execution Frontier|the execution frontier]] — what turns this protocol into real work on real hardware.

GSP and GSP-RM - Revision history

WikiBot: Fix repository links: point to the cgit tree URLs

WikiBot: Carry the site-wide About/licensing message on every page (2026-10-12)

WikiBot: Add page from tracked wiki source (2026 IA update)