Difference between revisions of "GMMU Multi-Tenant Protection"
Jump to navigation
Jump to search
Key code anchors (master
(Remove internal document references) |
m (Re-sync page from tracked wiki source: fix cross-document links and diagram; refresh status) |
||
| Line 1: | Line 1: | ||
= GMMU Multi-Tenant Protection = | = GMMU Multi-Tenant Protection = | ||
< | <blockquote>'''Current — companion summary.''' This page condenses [https://git.vgpu.io/open-iov/blob/master/docs/03-architecture/14-multi-tenant-gmmu-protection.md <code>../docs/03-architecture/14-multi-tenant-gmmu-protection.md</code>] (the canonical document: problem statement, full design, RM struct sketches, walked example, portability, M1–M8 roadmap). Read the canonical document for anything you intend to build against; this page is the quick map.</blockquote> | ||
== Overview == | == Overview == | ||
Open-IOV must isolate '''multiple mdevs (VMs)''' and, for multi-tenant inference, '''multiple tenants inside one guest''' at the GMMU. The design has two halves: | Open-IOV must isolate '''multiple mdevs (VMs)''' and, for multi-tenant inference, '''multiple tenants inside one guest''' at the GMMU. The design has two halves: | ||
# '''Service-state ownership (baseline 1.0).''' The work/completion service (<code>kernel/gsp_sema_service.c</code>) is one global context (<code>static struct sema_svc svc</code>, <code>:676</code>), the pump services the first open mdev (<code>:13352-13364</code>), and channels/VA spaces/translations are keyed by guest-chosen values alone (<code>:847</code>, <code>:988</code>, <code>:113-130</code>). Two guests can pick the same handles and VAs (live: channel <code>0xcaf000d4</code>; <code>../debug/issue/issue-number-2.md</code>), so owner-keying is a prerequisite for any GMMU policy. | |||
# '''GMMU policy for paged attention (research).''' A tenant/agent-aware KV cache protection design across vLLM, a guest utility, RM controls, the vEP, and page tables — post-1.0, dual-gated; never a baseline dependency. | # '''GMMU policy for paged attention (research).''' A tenant/agent-aware KV cache protection design across vLLM, a guest utility, RM controls, the vEP, and page tables — post-1.0, dual-gated; never a baseline dependency. | ||
== Isolation layers == | == Isolation layers == | ||
| Line 50: | Line 51: | ||
|} | |} | ||
== Hardware primitives: proven vs hypothesis == | == Hardware primitives: proven vs hypothesis == | ||
| Line 59: | Line 59: | ||
|- | |- | ||
| 4-level tree, 49-bit VA, 4K/64K/2M pages, dual PDE0 | | 4-level tree, 49-bit VA, 4K/64K/2M pages, dual PDE0 | ||
| '''proven''' (<code>kernel/gsp_sema_service.c:1058-1072</code>) | | '''proven''' (<code>docs/43-GMMU-PTE-DECODER</code>; <code>kernel/gsp_sema_service.c:1058-1072</code>) | ||
|- | |- | ||
| VER2 PTE fields incl. <code>read_only</code> (6), <code>privilege</code> (5), <code>encrypted</code> (4), <code>kind</code> | | VER2 PTE fields incl. <code>read_only</code> (6), <code>privilege</code> (5), <code>encrypted</code> (4), <code>kind</code> | ||
| '''proven format''' | | '''proven format''' (<code>docs/43</code> §3) | ||
|- | |- | ||
| Separate page directory per VA space (<code>SET_PAGE_DIRECTORY</code> with <code>hVASpace</code>/<code>chId</code>) | | Separate page directory per VA space (<code>SET_PAGE_DIRECTORY</code> with <code>hVASpace</code>/<code>chId</code>) | ||
| Line 77: | Line 77: | ||
|} | |} | ||
== The design in brief == | == The design in brief == | ||
* '''Design A (stock-compatible).''' Extended vLLM reserves a VA range per agent; a new guest utility (<code>oiv-agentd</code>) publishes tag records to a device-memory '''tag mailbox''' the vEP can read (emulation mode: guest vidmem is host RAM, <code>kernel/bar_init.c:68-77</code>). The vEP enforces owner↔VA-subtree match at map time and at submit time. No guest KMD change. Research (M6). | * '''Design A (stock-compatible).''' Extended vLLM reserves a VA range per agent; a new guest utility (<code>oiv-agentd</code>) publishes tag records to a device-memory '''tag mailbox''' the vEP can read (emulation mode: guest vidmem is host RAM, <code>kernel/bar_init.c:68-77</code>). The vEP enforces owner↔VA-subtree match at map time and at submit time. No guest KMD change. Research (M6). | ||
Requires a guest KMD extension: post-1.0, gated (M8). | * '''Design B (explicit controls).''' New vGPU-private RM controls — <code>TENANT_BIND</code> (0x90f10120), <code>TAG_RANGE</code> (…21), <code>UNTAG_RANGE</code> (…22), <code>TENANT_SHARE</code> (…23), <code>AGENT_CHANNEL_BIND</code> (…24), <code>TAG_QUERY</code> (…25) — plus struct sketches in the canonical doc §4.5. Requires a guest KMD extension: post-1.0, gated (M8). | ||
* '''Granularities.''' Per-KV-block tagging (finest; software+PTE policy, sharing needs RO enforcement); per-tenant separate page directories (the only proven hardware lever); per-agent separate channels (fault/QoS containment; optional per-agent sub-directories). Degradation ladder and per-granularity cost/failure analysis.7. | * '''Granularities.''' Per-KV-block tagging (finest; software+PTE policy, sharing needs RO enforcement); per-tenant separate page directories (the only proven hardware lever); per-agent separate channels (fault/QoS containment; optional per-agent sub-directories). Degradation ladder and per-granularity cost/failure analysis in canonical §4.7. | ||
== Walked example (two tenants, paged_attention_v2) == | == Walked example (two tenants, paged_attention_v2) == | ||
Two agents A1/A2 each own a VA pool and channels. A malicious T1 kernel naming an A2 VA hits a missing PTE (GMMU fault) and/or is refused by vEP submit-time policy; the fault is attributed to A1 only (T2 untouched); a periodic owner-vs-PTE audit with canaries catches escapes. If only per-tenant separation exists, the deployable contract is '''one agent per tenant, one tenant per context'''4). | Two agents A1/A2 each own a VA pool and channels. A malicious T1 kernel naming an A2 VA hits a missing PTE (GMMU fault) and/or is refused by vEP submit-time policy; the fault is attributed to A1 only (T2 untouched); a periodic owner-vs-PTE audit with canaries catches escapes. If only per-tenant separation exists, the deployable contract is '''one agent per tenant, one tenant per context''' (canonical §5.4). | ||
== Key code anchors (master <code>1f571dda</code>) == | == Key code anchors (master <code>1f571dda</code>) == | ||
| Line 117: | Line 116: | ||
|} | |} | ||
== Roadmap (acceptance criteria.3) == | == Roadmap (acceptance criteria in canonical §7.3) == | ||
M1 per-mdev service state — M2 two-guest isolation gate — M3 per-mdev real GSP clients + fairness — M4 independent VRAM sizes/sparse backing — M5 invalidate + eviction/reclaim (I1–I6 invariants) — M6 Design A prototype — M7 RO/permission probe — M8 Design B / CCS / 610 (gated). M1–M5 are the 1.0 programme; M6–M8 are post-1.0 research, dual-gated, duplicate-not- replace, never required by the baseline. RM 610 stays behind its own human gate (<code>docs/23-RM610-READINESS</code>). | |||
== Related documentation == | == Related documentation == | ||
* [[Introduction to GMMU]] — from-first-principles GMMU introduction (what it is, page tables, IOMMU/MMU/SMMU relationships) | * [[Introduction to GMMU|<code>Introduction-to-GMMU.md</code>]] — from-first-principles GMMU introduction (what it is, page tables, IOMMU/MMU/SMMU relationships) | ||
* [https://git.vgpu.io/open-iov/blob/master/docs/03-architecture/14-multi-tenant-gmmu-protection.md <code>../docs/03-architecture/14-multi-tenant-gmmu-protection.md</code>] — canonical | |||
* [https://git.vgpu.io/open-iov/blob/master/docs/03-architecture/13-vexecution-plane.md <code>../docs/03-architecture/13-vexecution-plane.md</code>] — vEP vocabulary (parallel) | |||
* [https://git.vgpu.io/open-iov/blob/master/docs/03-architecture/05-gmmu-amap-hal-gr.md <code>../docs/03-architecture/05-gmmu-amap-hal-gr.md</code>] — GMMU background | |||
* [https://git.vgpu.io/open-iov/blob/master/docs/03-architecture/09-full-system-picture.md <code>../docs/03-architecture/09-full-system-picture.md</code>] — multi-tenancy and isolation map | |||
* [https://git.vgpu.io/open-iov/blob/master/docs/03-architecture/11-physical-backend-guest-wiring.md <code>../docs/03-architecture/11-physical-backend-guest-wiring.md</code>] — forwarding and page-table seams | |||
* [https://git.vgpu.io/open-iov/blob/master/docs/05-operations/01-performance-correctness-security.md <code>../docs/05-operations/01-performance-correctness-security.md</code>] — threat model, rung 5 | |||
* [https://git.vgpu.io/open-iov/blob/master/debug/issue/issue-number-2.md <code>../debug/issue/issue-number-2.md</code>] — per-client state wedge | |||
* [https://git.vgpu.io/open-iov/blob/master/debug/risc-v-gsp-rm-vm/docs/43-GMMU-PTE-DECODER.md <code>../debug/risc-v-gsp-rm-vm/docs/43-GMMU-PTE-DECODER.md</code>] — GMMU field catalogue | |||
Revision as of 04:10, 12 October 2026
GMMU Multi-Tenant Protection
Current — companion summary. This page condenses
../docs/03-architecture/14-multi-tenant-gmmu-protection.md(the canonical document: problem statement, full design, RM struct sketches, walked example, portability, M1–M8 roadmap). Read the canonical document for anything you intend to build against; this page is the quick map.
Overview
Open-IOV must isolate multiple mdevs (VMs) and, for multi-tenant inference, multiple tenants inside one guest at the GMMU. The design has two halves:
- Service-state ownership (baseline 1.0). The work/completion service (
kernel/gsp_sema_service.c) is one global context (static struct sema_svc svc,:676), the pump services the first open mdev (:13352-13364), and channels/VA spaces/translations are keyed by guest-chosen values alone (:847,:988,:113-130). Two guests can pick the same handles and VAs (live: channel0xcaf000d4;../debug/issue/issue-number-2.md), so owner-keying is a prerequisite for any GMMU policy. - GMMU policy for paged attention (research). A tenant/agent-aware KV cache protection design across vLLM, a guest utility, RM controls, the vEP, and page tables — post-1.0, dual-gated; never a baseline dependency.
Isolation layers
| Layer | Mechanism | Status |
|---|---|---|
| L0 platform | IOMMU/SMMU per-VM DMA, KVM memory/vCPU boundary | live |
| L1 transport | VFIO mdev per tenant, config/BAR/IRQ mediation | live |
| L2 vEP state | per-mdev emulation context + (needed) per-mdev work service | live / design |
| L3 GMMU | per-context page directory (VA space) | live (guest-owned) / design (host-owned) |
| L4 arbitration | channel ownership, TSG/runlist policy | design |
| L5 completion | owner-keyed release resolution; no cross-tenant apply | wip |
| L6 fault | fault attributed to the owning channel/tenant only | wip |
| L7 application | per-tenant/per-agent VA pooling inside a guest | research |
Hardware primitives: proven vs hypothesis
| Primitive | Status |
|---|---|
| 4-level tree, 49-bit VA, 4K/64K/2M pages, dual PDE0 | proven (docs/43-GMMU-PTE-DECODER; kernel/gsp_sema_service.c:1058-1072)
|
VER2 PTE fields incl. read_only (6), privilege (5), encrypted (4), kind
|
proven format (docs/43 §3)
|
Separate page directory per VA space (SET_PAGE_DIRECTORY with hVASpace/chId)
|
proven (kernel/gsp_rm_emulation.c:3310-3314)
|
| No tenant/ACL/PASID field in the VER2 PTE/PDE | proven absence in the field catalogue |
| RO-leaf write enforcement | hypothesis (needs a supervised probe; M7) |
| Encryption-domain (CCS) as a tenant boundary | hypothesis |
The design in brief
- Design A (stock-compatible). Extended vLLM reserves a VA range per agent; a new guest utility (
oiv-agentd) publishes tag records to a device-memory tag mailbox the vEP can read (emulation mode: guest vidmem is host RAM,kernel/bar_init.c:68-77). The vEP enforces owner↔VA-subtree match at map time and at submit time. No guest KMD change. Research (M6). - Design B (explicit controls). New vGPU-private RM controls —
TENANT_BIND(0x90f10120),TAG_RANGE(…21),UNTAG_RANGE(…22),TENANT_SHARE(…23),AGENT_CHANNEL_BIND(…24),TAG_QUERY(…25) — plus struct sketches in the canonical doc §4.5. Requires a guest KMD extension: post-1.0, gated (M8). - Granularities. Per-KV-block tagging (finest; software+PTE policy, sharing needs RO enforcement); per-tenant separate page directories (the only proven hardware lever); per-agent separate channels (fault/QoS containment; optional per-agent sub-directories). Degradation ladder and per-granularity cost/failure analysis in canonical §4.7.
Walked example (two tenants, paged_attention_v2)
Two agents A1/A2 each own a VA pool and channels. A malicious T1 kernel naming an A2 VA hits a missing PTE (GMMU fault) and/or is refused by vEP submit-time policy; the fault is attributed to A1 only (T2 untouched); a periodic owner-vs-PTE audit with canaries catches escapes. If only per-tenant separation exists, the deployable contract is one agent per tenant, one tenant per context (canonical §5.4).
Key code anchors (master 1f571dda)
| What | Where |
|---|---|
| per-mdev emulation context | kernel/include/nvidia-vgpu-vfio.h:133, kernel/gsp_emulation.h:130
|
| global service state | kernel/gsp_sema_service.c:676; pump :13352; doorbell :13512
|
| channel/vaspace registries | kernel/gsp_sema_service.c:847, :988
|
| GMMU walk (VER2) | kernel/gsp_sema_service.c:1174; decode :1093
|
| page directory record | kernel/gsp_rm_emulation.c:3320; reserved PDEs :2237-2262
|
| per-mdev FB sizing | kernel/bar_init.c:60-77; kernel/include/ioctl.h:39-46
|
| L2 invalidate acked | kernel/gsp_rm_emulation.c:2519-2532
|
Roadmap (acceptance criteria in canonical §7.3)
M1 per-mdev service state — M2 two-guest isolation gate — M3 per-mdev real GSP clients + fairness — M4 independent VRAM sizes/sparse backing — M5 invalidate + eviction/reclaim (I1–I6 invariants) — M6 Design A prototype — M7 RO/permission probe — M8 Design B / CCS / 610 (gated). M1–M5 are the 1.0 programme; M6–M8 are post-1.0 research, dual-gated, duplicate-not- replace, never required by the baseline. RM 610 stays behind its own human gate (docs/23-RM610-READINESS).
Related documentation
Introduction-to-GMMU.md— from-first-principles GMMU introduction (what it is, page tables, IOMMU/MMU/SMMU relationships)../docs/03-architecture/14-multi-tenant-gmmu-protection.md— canonical../docs/03-architecture/13-vexecution-plane.md— vEP vocabulary (parallel)../docs/03-architecture/05-gmmu-amap-hal-gr.md— GMMU background../docs/03-architecture/09-full-system-picture.md— multi-tenancy and isolation map../docs/03-architecture/11-physical-backend-guest-wiring.md— forwarding and page-table seams../docs/05-operations/01-performance-correctness-security.md— threat model, rung 5../debug/issue/issue-number-2.md— per-client state wedge../debug/risc-v-gsp-rm-vm/docs/43-GMMU-PTE-DECODER.md— GMMU field catalogue