Introduction to GMMU

Introduction to the GMMU

1. What the GMMU is

The GMMU (GPU Memory Management Unit) is the address-translation hardware inside the GPU. Every engine — GR (shaders and CUDA kernels), CE (copy engines), SEC2, the video engines — addresses memory through GPU virtual addresses (VAs), and the GMMU translates each VA into a physical address: an offset in GPU local memory (vidmem/VRAM), an address in system memory (sysmem), or a peer GPU's memory across a fabric. It is a paged MMU, conceptually the CPU MMU's sibling:

• Multi-level page tables describe the mappings; the root is a page directory base (PDB).
• A TLB caches translations so the walk is not paid per access.
• Translation failures raise faults (invalid/missing leaf, missing table), reported through engine fault buffers.
• Each context has its own page directory (VA space), so one context cannot name another context's mappings — this is the hardware isolation boundary the project builds multi-tenancy on.

What makes it not just "the CPU MMU on a GPU":

• The tables live in GPU-visible memory (vidmem or sysmem), and each directory entry says which aperture the next level lives in.
• Entries carry GPU-specific aperture (which physical space) and memory-kind fields that the CPU MMU does not have.
• The "physical" addresses inside the tables are guest-physical addresses or emulated-VRAM offsets in our virtualized setting; the host module owns the resolution to real host addresses (§4.2). On a bare-metal GPU the same structures point at real vidmem/sysmem.

For this project the GMMU matters three ways: the guest driver programs it exactly as on bare metal (no guest patches); the guest's page tables are the ground truth for where its memory actually is; and the per-context page directory is the only proven hardware isolation primitive available on the reference generation (§4.2).

2. The three MMUs: CPU MMU, GMMU, IOMMU/SMMU

	CPU MMU	GMMU	IOMMU / SMMU
Translates	process VA → host physical address	engine GPU VA → GPU physical (vidmem / sysmem / peer)	device DMA address → host physical address
Lives in	the CPU core	the GPU	the host platform (IOMMU: Intel VT-d / AMD-Vi) or the SoC (SMMU, ARM)
Programmed by	the OS kernel (per-process tables)	the GPU driver / RM (the guest driver here)	the hypervisor / OS (per-device or per-VM domains)
Table unit	per-process page tables (e.g. CR3)	per-context page directory (VA space)	per-device/per-VM translation domain
Isolation scope	process ↔ process	context ↔ context (engine-level)	VM/device ↔ VM/device (platform DMA)
In our stack	guest and host kernels	guest driver builds the tables; host walks or plants its own	hypervisor enforces it; the module resolves GPAs itself when it needs host addresses

They are three instances of the same idea — page tables plus a TLB give virtual memory and isolation to an agent — at three different points in the address path, owned by three different software layers. In a virtualized guest a GPU access passes through two of them:

<syntaxhighlight lang="text">engine (GR / CE)

  │  GPU VA
  ▼

GMMU (inside the GPU, driven by the guest's page tables)

  │  guest physical address (GPA) / GPU local address
  ▼

IOMMU / SMMU (in the host platform, programmed by the hypervisor for this VM)

  │  host physical address (HPA)
  ▼

host DRAM / real VRAM</syntaxhighlight> The two stages are independent: the GMMU decides which guest page an engine touches; the IOMMU/SMMU decides whether this device may reach that page at all and where it lives in host memory. GMMU isolation between guest contexts is the GPU driver/RM's job; IOMMU isolation between VMs is the platform's job. How the two meet in this project:

• In another virtual GPU implementation, the host vGPU manager uses the platform IOMMU to establish the GPU's view of guest memory. Here the host module does the equivalent resolution itself.
• On the reference host the physical GPU's IOMMU group is an identity domain (all-RAM identity) and the mediated device registers as an emulated-IOMMU VFIO device: its type-1 container holds guest DMA mappings privately and no kernel iommu_domain exists for it, so iommu_iova_to_phys cannot translate guest GPAs. Guest RAM is backed by a QEMU shared file and the module translates GPA→HPA through that file's page cache, behind the active-mdev gate (a per-mdev-gated translation, not itself an IOMMU guarantee).
• The project rule is explicit: memory isolation is the platform IOMMU's job, and we never weaken it — DMA only through the hypervisor-provided interface for the device's own VM, no global mappings, fail closed if a required isolation property cannot be established.
• On aarch64 hosts the SMMU plays the IOMMU's role; page sizes, cache attributes and SMMU behaviour are on the host-portability list for the Grace/GB10 targets.

3. How the GMMU works (Turing reference)

3.1 Page-tree geometry

The Turing page-tree path inherits the Pascal/Volta geometry and uses the VER2 entry encoding. Depths run root (0) to leaf (4); VA width is 49 bits:

Depth	HW level	VA bits	Index bits	Entry size	Role
0	PDE3	48:47	2	8 B	pointer PDE
1	PDE2	46:38	9	8 B	pointer PDE
2	PDE1	37:29	9	8 B	pointer PDE (NO_ATS set here)
3	PDE0	28:21	8	16 B (dual) / 8 B (2 MB leaf)	dual 64K/4K PDE, or a 2 MB leaf PTE
4	PTE	20:12 (4 KB) / 20:16 (64 KB)	9 / 5	8 B	leaf PTE

Supported page sizes are 2 MB (depth 3), 64 KB and 4 KB (depth 4). A leaf entry alone does not say 4 KB vs 64 KB — the table's depth and page-size context do — so decoders must report 4k_or_64k rather than guess.

3.2 Entry encoding (VER2)

Leaf PTE (8 bytes):

Field	Bits	Meaning
valid	0	entry is present
aperture	2:1	0 video, 1 peer, 2 system coherent, 3 system non-coherent
volatile	3	cache/volatility bit
encrypted	4	encrypted memory
privilege	5	privileged access
read_only	6	write protection
atomic_disable	7	atomics disabled
address_sys	53:8	system-aperture address bits (PA 53:12)
address_vid	32:8	video/peer address bits (PA 36:12)
peer_id	35:33	peer/fabric id (peer aperture)
comptagline	55:36	compression tag (overloaded as high address bits for the 47-bit peer case)
kind	63:56	memory kind id

Directory PDE (8 bytes) is the same shape with one discriminator change: bit 0 is is_pde_pointer (0 = pointer to a lower table, 1 = a 2 MB leaf PTE), and aperture 0 means invalid at directory level (it means video memory at leaf level). Depth 2 sets NO_ATS (bit 5). Dual PDE0 (16 bytes) holds two halves for mixed 64K/4K tables: BIG (low 64 bits) and SMALL (high 64 bits); bit 0 discriminates the whole slot. Live guest note: the client's leaf entries in the format we observe are PA[47:12] | flags, with flags = 0xe for a valid system-memory leaf. The kind field carries numeric memory-kind ids (243 named values); common compute values are 0x00 PITCH, 0x06 GENERIC_MEMORY, 0x07 INVALID, 0x0F SMSKED_MESSAGE, 0xFE GENERIC_16BX2. The decoder (sim/gmmu_pte.py) is decode-only: unsupported encodings fail closed, set bits not covered by a named field are reported as reserved, and nothing is guessed.

3.3 Walking, caching and faults

A walk starts at the channel's page-directory base and follows PDE3 → PDE2 → PDE1 → PDE0 → PTE; the leaf yields the physical address, page size, aperture and access permissions. The TLB caches results, and there is a second, coarser cache that matters operationally:

• PTE fetch-line rule (learned live): the engine caches PTE fetches at 128-byte (16-PTE) granularity, and the forwarding code has no invalidate path. A runtime PTE write into a line the GMMU has already walked is invisible to the engine. Consequences: page-table slots for a guest window are pre-built before any engine access, and each forwarded copy pair consumes a whole fresh line from a pre-zeroed pool (16 page-table pages, capacity 512 pairs per boot in the current design) — a line is never reused within a boot.
• Faults: invalid or missing leaves and missing tables raise GMMU faults; the driver allocates fault buffers whose sizes come from INTERNAL_GMMU_GET_STATIC_INFO (control 0x20800a59): 64 KB replayable / 4 KB legacy — the values the emulation serves, because a wrong value breaks UVM early.

4. How the GMMU is used

4.1 Guest side (the stock driver)

• The guest driver allocates VA spaces and builds the page tables, then publishes the directory per channel/context through SET_PAGE_DIRECTORY (hVASpace + chId; the host records the directory per owner in kernel/gsp_rm_emulation.c:3307-3319).
• Memory descriptors carry an aperture: ADDR_SYSMEM = 1, ADDR_FBMEM = 2.
• Server-owned ranges: COPY_SERVER_RESERVED_PDES (control 0x90f10106) tells the client that some page-table ranges are populated by the "server" (us). The exact division of labour for RESERVE_ENTRIES/RELEASE_ENTRIES and invalidations is a tracked known-unknown.
• Channels submit push buffers that reference GPU VAs; the engines walk the GMMU to reach the data.

4.2 Host side (vgsp.ko)

• Truth. The host resolves guest VAs by walking the driver-published page directories itself (the trusted GMMU walk in kernel/gsp_sema_service.c, VER2 geometry at :1058-1072, decode :1093, walk :1174), plus injected pins and content-keyed rescue when neither names a mapping. This is what lets the software path apply a semaphore release to the correct host page.
• Forwarding. For real CE copies the module builds its own scratch page tables that map already-resolved host pages (guest RAM through the shared file; shadow pages through the emulated-VRAM mapping), under the fetch-line rules of §3.3. Real guest page tables are not yet programmed into the real MMU — guest submission on hardware is the end state (P6).
• Emulation. Guest vidmem is a private host-RAM buffer per mdev (kernel/bar_init.c:68-77); FB-aperture accesses stay inside the owning mdev's buffer, and guest RAM is reached only through the hypervisor's per-mdev DMA interface (vfio_dma_rw).
• Isolation. The per-context page directory is the only proven hardware isolation primitive on this generation: the VER2 PTE/PDE has no tenant/ACL/PASID field (proven absence in the named field set; peer_id is a fabric peer selector, not a security domain). read_only (bit 6) is a proven format field, but its enforcement is a [hypothesis] until the M7 supervised probe. The multi-tenant design therefore places decisions in topology (which directory a page appears in, which channel may name it) plus software validation and audit in the execution plane — see GMMU Multi-Tenant Protection.
• Versioning. GMMU controls, static info and entry formats are version/hardware-keyed; formats are expected to differ per generation and must be re-derived by measurement, never guessed.

4.3 Tooling built around it

Tool / doc	Question it answers
decode a captured entry field-by-field (decode-only, fail closed)
walk a captured PDB to the leaf mapping or the exact fault (4K/2M validated against a live captured channel PDB; 64K modelled)
Launch audit harness/launch_audit.py	where the engine would read/write each leg of a submission: ok / pa_mismatch / unmapped / access_denied
compare simulated vs observed GMMU/FBIF register values, per register, with fidelity labels
record what firmware touches in the GMMU register windows without faking semantics

5. Limits and open questions

• read_only enforcement is unproven (M7 supervised probe); encryption (CCS) semantics are unstudied. Neither can be assumed as a tenant boundary.
• Invalidation semantics are not fully modelled: the 128-byte fetch-line observation is live evidence, not a complete invalidate contract.
• The 64K dual path is modelled from the format definitions, not yet validated against a capture.
• Newer generations (Ampere/Hopper/Blackwell, GB10) are expected to change the entry format; each must be re-derived. RM 610 work sits behind its own human approval gate.
• No timing/cycle claims are made anywhere here; zero hits in a run are a coverage fact, not proof of dead code.
• Multi-mdev isolation is first a service-state-ownership problem (host tables keyed by guest-chosen handles/VAs alone collide); GMMU policy is built on top of that fix, not instead of it.

6. Related documentation

• GMMU Multi-Tenant Protection — the isolation design condensed.
• The vGSP Execution Plane (vEP) — how guest work becomes physical execution; the fetch-line rule in context.