Introduction to GMMU

Introduction to the GMMU

Scope. A from-first-principles introduction to the GPU Memory Management Unit (GMMU): what it translates, how its page tables work on our Turing reference, how the host module uses them, and how the GMMU relates to the CPU MMU and the platform IOMMU/SMMU. The architecture chapter is ../docs/03-architecture/05-gmmu-amap-hal-gr.md; the isolation policy authority is ../docs/03-architecture/14-multi-tenant-gmmu-protection.md (condensed in GMMU-Multi-Tenant-Protection.md).

Evidence discipline. Hardware-format claims condense the tracked field catalogue ../debug/risc-v-gsp-rm-vm/docs/43-GMMU-PTE-DECODER.md and live observations; numeric IDs, layouts and code anchors are cited, and no timing/cycle claim is made.

Status: 2026-10-11, tree master @ 08f39c2f. Tags: [live] observed with the stock guest driver; [proven] hardware-format fact; [hypothesis] unexercised; [design] not built.

1. What the GMMU is

The GMMU (GPU Memory Management Unit) is the address-translation hardware inside the GPU. Every engine — GR (shaders and CUDA kernels), CE (copy engines), SEC2, the video engines — addresses memory through GPU virtual addresses (VAs), and the GMMU translates each VA into a physical address: an offset in GPU local memory (vidmem/VRAM), an address in system memory (sysmem), or a peer GPU's memory across a fabric.

It is a paged MMU, conceptually the CPU MMU's sibling:

• Multi-level page tables describe the mappings; the root is a page directory base (PDB).
• A TLB caches translations so the walk is not paid per access.
• Translation failures raise faults (invalid/missing leaf, missing table), reported through engine fault buffers.
• Each context has its own page directory (VA space), so one context cannot name another context's mappings — this is the hardware isolation boundary the project builds multi-tenancy on.

What makes it not just "the CPU MMU on a GPU":

• The tables live in GPU-visible memory (vidmem or sysmem), and each directory entry says which aperture the next level lives in.
• Entries carry GPU-specific aperture (which physical space) and memory-kind fields that the CPU MMU does not have.
• The "physical" addresses inside the tables are guest-physical addresses or emulated-VRAM offsets in our virtualized setting; the host module owns the resolution to real host addresses (§4.2). On a bare-metal GPU the same structures point at real vidmem/sysmem.

For this project the GMMU matters three ways: the guest driver programs it exactly as on bare metal (no guest patches); the guest's page tables are the ground truth for where its memory actually is; and the per-context page directory is the only proven hardware isolation primitive available on the reference generation (§4.2).

2. The three MMUs: CPU MMU, GMMU, IOMMU/SMMU

	CPU MMU	GMMU	IOMMU / SMMU
Translates	process VA → host physical address	engine GPU VA → GPU physical (vidmem / sysmem / peer)	device DMA address → host physical address
Lives in	the CPU core	the GPU	the host platform (IOMMU: Intel VT-d / AMD-Vi) or the SoC (SMMU, ARM)
Programmed by	the OS kernel (per-process tables)	the GPU driver / RM (the guest driver here)	the hypervisor / OS (per-device or per-VM domains)
Table unit	per-process page tables (e.g. CR3)	per-context page directory (VA space)	per-device/per-VM translation domain
Isolation scope	process ↔ process	context ↔ context (engine-level)	VM/device ↔ VM/device (platform DMA)
In our stack	guest and host kernels	guest driver builds the tables; host walks or plants its own	hypervisor enforces it; the module resolves GPAs itself when it needs host addresses

They are three instances of the same idea — page tables plus a TLB give virtual memory and isolation to an agent — at three different points in the address path, owned by three different software layers. In a virtualized guest a GPU access passes through two of them:

engine (GR / CE)
   │  GPU VA
   ▼
GMMU  (inside the GPU, driven by the guest's page tables)
   │  guest physical address (GPA) / GPU local address
   ▼
IOMMU / SMMU  (in the host platform, programmed by the hypervisor for this VM)
   │  host physical address (HPA)
   ▼
host DRAM / real VRAM

The two stages are independent: the GMMU decides which guest page an engine touches; the IOMMU/SMMU decides whether this device may reach that page at all and where it lives in host memory. GMMU isolation between guest contexts is the GPU driver/RM's job; IOMMU isolation between VMs is the platform's job.

How the two meet in this project:

• In real vGPU, the host vGPU manager uses the platform IOMMU to establish the GPU's view of guest memory. Here the host module does the equivalent resolution itself.
• On the reference host the physical GPU's IOMMU group is an identity domain (all-RAM identity) and the mediated device registers as an emulated-IOMMU VFIO device: its type-1 container holds guest DMA mappings privately and no kernel iommu_domain exists for it, so iommu_iova_to_phys cannot translate guest GPAs. Guest RAM is backed by a QEMU shared file and the module translates GPA→HPA through that file's page cache, behind the active-mdev gate (a per-mdev-gated translation, not itself an IOMMU guarantee).
• The project rule is explicit: memory isolation is the platform IOMMU's job, and we never weaken it — DMA only through the hypervisor-provided interface for the device's own VM, no global mappings, fail closed if a required isolation property cannot be established.
• On aarch64 hosts the SMMU plays the IOMMU's role; page sizes, cache attributes and SMMU behaviour are on the host-portability list for the Grace/GB10 targets.

3. How the GMMU works (Turing reference)

3.1 Page-tree geometry

The Turing page-tree path inherits the Pascal/Volta geometry and uses the VER2 entry encoding. Depths run root (0) to leaf (4); VA width is 49 bits:

Depth	HW level	VA bits	Index bits	Entry size	Role
0	PDE3	48:47	2	8 B	pointer PDE
1	PDE2	46:38	9	8 B	pointer PDE
2	PDE1	37:29	9	8 B	pointer PDE (NO_ATS set here)
3	PDE0	28:21	8	16 B (dual) / 8 B (2 MB leaf)	dual 64K/4K PDE, or a 2 MB leaf PTE
4	PTE	20:12 (4 KB) / 20:16 (64 KB)	9 / 5	8 B	leaf PTE

Supported page sizes are 2 MB (depth 3), 64 KB and 4 KB (depth 4). A leaf entry alone does not say 4 KB vs 64 KB — the table's depth and page-size context do — so decoders must report 4k_or_64k rather than guess.

3.2 Entry encoding (VER2)

Leaf PTE (8 bytes):

Field	Bits	Meaning
valid	0	entry is present
aperture	2:1	0 video, 1 peer, 2 system coherent, 3 system non-coherent
volatile	3	cache/volatility bit
encrypted	4	encrypted memory
privilege	5	privileged access
read_only	6	write protection
atomic_disable	7	atomics disabled
address_sys	53:8	system-aperture address bits (PA 53:12)
address_vid	32:8	video/peer address bits (PA 36:12)
peer_id	35:33	peer/fabric id (peer aperture)
comptagline	55:36	compression tag (overloaded as high address bits for the 47-bit peer case)
kind	63:56	memory kind id

Directory PDE (8 bytes) is the same shape with one discriminator change: bit 0 is is_pde_pointer (0 = pointer to a lower table, 1 = a 2 MB leaf PTE), and aperture 0 means invalid at directory level (it means video memory at leaf level). Depth 2 sets NO_ATS (bit 5).

Dual PDE0 (16 bytes) holds two halves for mixed 64K/4K tables: BIG (low 64 bits) and SMALL (high 64 bits); bit 0 discriminates the whole slot.

Live guest note: the client's leaf entries in the format we observe are PA[47:12] | flags, with flags = 0xe for a valid system-memory leaf. The kind field carries numeric memory-kind ids (243 named values); common compute values are 0x00 PITCH, 0x06 GENERIC_MEMORY, 0x07 INVALID, 0x0F SMSKED_MESSAGE, 0xFE GENERIC_16BX2.

The decoder (sim/gmmu_pte.py, docs/43) is decode-only: unsupported encodings fail closed, set bits not covered by a named field are reported as reserved, and nothing is guessed.

3.3 Walking, caching and faults

A walk starts at the channel's page-directory base and follows PDE3 → PDE2 → PDE1 → PDE0 → PTE; the leaf yields the physical address, page size, aperture and access permissions. The TLB caches results, and there is a second, coarser cache that matters operationally:

• PTE fetch-line rule (learned live): the engine caches PTE fetches at 128-byte (16-PTE) granularity, and the forwarding code has no invalidate path. A runtime PTE write into a line the GMMU has already walked is invisible to the engine. Consequences: page-table slots for a guest window are pre-built before any engine access, and each forwarded copy pair consumes a whole fresh line from a pre-zeroed pool (16 page-table pages, capacity 512 pairs per boot in the current design) — a line is never reused within a boot.
• Faults: invalid or missing leaves and missing tables raise GMMU faults; the driver allocates fault buffers whose sizes come from INTERNAL_GMMU_GET_STATIC_INFO (control 0x20800a59): 64 KB replayable / 4 KB legacy — the values the emulation serves, because a wrong value breaks UVM early.

4. How the GMMU is used

4.1 Guest side (the stock driver)

• The guest driver allocates VA spaces and builds the page tables, then publishes the directory per channel/context through SET_PAGE_DIRECTORY (hVASpace + chId; the host records the directory per owner in kernel/gsp_rm_emulation.c:3307-3319).
• Memory descriptors carry an aperture: ADDR_SYSMEM = 1, ADDR_FBMEM = 2.
• Server-owned ranges: COPY_SERVER_RESERVED_PDES (control 0x90f10106) tells the client that some page-table ranges are populated by the "server" (us). The exact division of labour for RESERVE_ENTRIES/RELEASE_ENTRIES and invalidations is a tracked known-unknown.
• Channels submit push buffers that reference GPU VAs; the engines walk the GMMU to reach the data.

4.2 Host side (vgsp.ko)

• Truth. The host resolves guest VAs by walking the driver-published page directories itself (the trusted GMMU walk in kernel/gsp_sema_service.c, VER2 geometry at :1058-1072, decode :1093, walk :1174), plus injected pins and content-keyed rescue when neither names a mapping. This is what lets the software path apply a semaphore release to the correct host page.
• Forwarding. For real CE copies the module builds its own scratch page tables that map already-resolved host pages (guest RAM through the shared file; shadow pages through the emulated-VRAM mapping), under the fetch-line rules of §3.3. Real guest page tables are not yet programmed into the real MMU — guest submission on hardware is the end state (P6).
• Emulation. Guest vidmem is a private host-RAM buffer per mdev (kernel/bar_init.c:68-77); FB-aperture accesses stay inside the owning mdev's buffer, and guest RAM is reached only through the hypervisor's per-mdev DMA interface (vfio_dma_rw).
• Isolation. The per-context page directory is the only proven hardware isolation primitive on this generation: the VER2 PTE/PDE has no tenant/ACL/PASID field (proven absence in the named field set; peer_id is a fabric peer selector, not a security domain). read_only (bit 6) is a proven format field, but its enforcement is a [hypothesis] until the M7 supervised probe. The multi-tenant design therefore places decisions in topology (which directory a page appears in, which channel may name it) plus software validation and audit in the execution plane — see GMMU-Multi-Tenant-Protection.md.
• Versioning. GMMU controls, static info and entry formats are version/hardware-keyed; formats are expected to differ per generation and must be re-derived by measurement, never guessed.

4.3 Tooling built around it

Tool / doc	Question it answers
PTE decoder sim/gmmu_pte.py + docs/43	decode a captured entry field-by-field (decode-only, fail closed)
Walker sim/gmmu_walk.py + docs/48	walk a captured PDB to the leaf mapping or the exact fault (4K/2M validated against a live captured channel PDB; 64K modelled)
Launch audit harness/launch_audit.py + docs/48	where the engine would read/write each leg of a submission: ok / pa_mismatch / unmapped / access_denied
Oracle workflow docs/44	compare simulated vs observed GMMU/FBIF register values, per register, with fidelity labels
Observe-only windows docs/42	record what firmware touches in the GMMU register windows without faking semantics

5. Limits and open questions

• read_only enforcement is unproven (M7 supervised probe); encryption (CCS) semantics are unstudied. Neither can be assumed as a tenant boundary.
• Invalidation semantics are not fully modelled: the 128-byte fetch-line observation is live evidence, not a complete invalidate contract.
• The 64K dual path is modelled from the format definitions, not yet validated against a capture.
• Newer generations (Ampere/Hopper/Blackwell, GB10) are expected to change the entry format; each must be re-derived. RM 610 work sits behind its own human approval gate.
• No timing/cycle claims are made anywhere here; zero hits in a run are a coverage fact, not proof of dead code.
• Multi-mdev isolation is first a service-state-ownership problem (host tables keyed by guest-chosen handles/VAs alone collide); GMMU policy is built on top of that fix, not instead of it.

6. Related documentation

• ../docs/03-architecture/05-gmmu-amap-hal-gr.md — architecture chapter: GMMU, AMAPlib/HALs, channels, runlists, GR.
• GMMU-Multi-Tenant-Protection.md — the isolation design condensed.
• ../docs/03-architecture/14-multi-tenant-gmmu-protection.md — canonical multi-tenant GMMU policy.
• Execution-Plane.md — how guest work becomes physical execution; the fetch-line rule in context.
• ../docs/03-architecture/11-physical-backend-guest-wiring.md — page-table seams and the guest-memory reachability model.
• ../docs/04-implementation/01-known-knowns-and-unknowns.md — server-owned page tables and the IOMMU reachability note.
• ../docs/05-operations/01-performance-correctness-security.md — threat model and the IOMMU rules.
• ../debug/risc-v-gsp-rm-vm/docs/43-GMMU-PTE-DECODER.md, docs/44, docs/48 — the field catalogue and the offline walk/audit tooling.