GMMU Multi-Tenant Protection

From Open-IOV
Jump to navigation Jump to search

An absence of critical technical documentation has historically slowed growth and adoption of developer ecosystems for GPU virtualization.

This CC-BY-4.0 licensed content can either be used with attribution, or used as inspiration for new documentation, created by GPU vendors for public commercial distribution as developer documentation.

Where possible, this documentation will clearly label dates and versions of observed-but-not-guaranteed behaviour vs. vendor-documented stable interfaces/behaviour with guarantees of forward or backward compatibility.

GMMU Multi-Tenant Protection

Current — companion summary. This page condenses ../docs/03-architecture/14-multi-tenant-gmmu-protection.md (the canonical document: problem statement, full design, RM struct sketches, walked example, portability, M1–M8 roadmap). Read the canonical document for anything you intend to build against; this page is the quick map.

Overview

Open-IOV must isolate multiple mdevs (VMs) and, for multi-tenant inference, multiple tenants inside one guest at the GMMU. The design has two halves:

  1. Service-state ownership (baseline 1.0). The work/completion service (kernel/gsp_sema_service.c) is one global context (static struct sema_svc svc, :676), the pump services the first open mdev (:13352-13364), and channels/VA spaces/translations are keyed by guest-chosen values alone (:847, :988, :113-130). Two guests can pick the same handles and VAs (live: channel 0xcaf000d4; ../debug/issue/issue-number-2.md), so owner-keying is a prerequisite for any GMMU policy.
  2. GMMU policy for paged attention (research). A tenant/agent-aware KV cache protection design across vLLM, a guest utility, RM controls, the vEP, and page tables — post-1.0, dual-gated; never a baseline dependency.

Isolation layers

Layer Mechanism Status
L0 platform IOMMU/SMMU per-VM DMA, KVM memory/vCPU boundary live
L1 transport VFIO mdev per tenant, config/BAR/IRQ mediation live
L2 vEP state per-mdev emulation context + (needed) per-mdev work service live / design
L3 GMMU per-context page directory (VA space) live (guest-owned) / design (host-owned)
L4 arbitration channel ownership, TSG/runlist policy design
L5 completion owner-keyed release resolution; no cross-tenant apply wip
L6 fault fault attributed to the owning channel/tenant only wip
L7 application per-tenant/per-agent VA pooling inside a guest research

Hardware primitives: proven vs hypothesis

Primitive Status
4-level tree, 49-bit VA, 4K/64K/2M pages, dual PDE0 proven (docs/43-GMMU-PTE-DECODER; kernel/gsp_sema_service.c:1058-1072)
VER2 PTE fields incl. read_only (6), privilege (5), encrypted (4), kind proven format (docs/43 §3)
Separate page directory per VA space (SET_PAGE_DIRECTORY with hVASpace/chId) proven (kernel/gsp_rm_emulation.c:3310-3314)
No tenant/ACL/PASID field in the VER2 PTE/PDE proven absence in the field catalogue
RO-leaf write enforcement hypothesis (needs a supervised probe; M7)
Encryption-domain (CCS) as a tenant boundary hypothesis

The design in brief

  • Design A (stock-compatible). Extended vLLM reserves a VA range per agent; a new guest utility (oiv-agentd) publishes tag records to a device-memory tag mailbox the vEP can read (emulation mode: guest vidmem is host RAM, kernel/bar_init.c:68-77). The vEP enforces owner↔VA-subtree match at map time and at submit time. No guest KMD change. Research (M6).
  • Design B (explicit controls). New vGPU-private RM controls — TENANT_BIND (0x90f10120), TAG_RANGE (…21), UNTAG_RANGE (…22), TENANT_SHARE (…23), AGENT_CHANNEL_BIND (…24), TAG_QUERY (…25) — plus struct sketches in the canonical doc §4.5. Requires a guest KMD extension: post-1.0, gated (M8).
  • Granularities. Per-KV-block tagging (finest; software+PTE policy, sharing needs RO enforcement); per-tenant separate page directories (the only proven hardware lever); per-agent separate channels (fault/QoS containment; optional per-agent sub-directories). Degradation ladder and per-granularity cost/failure analysis in canonical §4.7.

Walked example (two tenants, paged_attention_v2)

Two agents A1/A2 each own a VA pool and channels. A malicious T1 kernel naming an A2 VA hits a missing PTE (GMMU fault) and/or is refused by vEP submit-time policy; the fault is attributed to A1 only (T2 untouched); a periodic owner-vs-PTE audit with canaries catches escapes. If only per-tenant separation exists, the deployable contract is one agent per tenant, one tenant per context (canonical §5.4).

Key code anchors (master 1f571dda)

What Where
per-mdev emulation context kernel/include/nvidia-vgpu-vfio.h:133, kernel/gsp_emulation.h:130
global service state kernel/gsp_sema_service.c:676; pump :13352; doorbell :13512
channel/vaspace registries kernel/gsp_sema_service.c:847, :988
GMMU walk (VER2) kernel/gsp_sema_service.c:1174; decode :1093
page directory record kernel/gsp_rm_emulation.c:3320; reserved PDEs :2237-2262
per-mdev FB sizing kernel/bar_init.c:60-77; kernel/include/ioctl.h:39-46
L2 invalidate acked kernel/gsp_rm_emulation.c:2519-2532

Roadmap (acceptance criteria in canonical §7.3)

M1 per-mdev service state — M2 two-guest isolation gate — M3 per-mdev real GSP clients + fairness — M4 independent VRAM sizes/sparse backing — M5 invalidate + eviction/reclaim (I1–I6 invariants) — M6 Design A prototype — M7 RO/permission probe — M8 Design B / CCS / 610 (gated). M1–M5 are the 1.0 programme; M6–M8 are post-1.0 research, dual-gated, duplicate-not- replace, never required by the baseline. RM 610 stays behind its own human gate (docs/23-RM610-READINESS).

Related documentation

Retrieved from "https://open-iov.org/index.php?title=GMMU_Multi-Tenant_Protection&oldid=23969"