Performance

Performance

Scope. How Open-IOV's performance is modelled, traced, and planned: the cost structure of the system, the metric set, what is observed today (honestly — there are no published numbers yet), the known gaps with their improvement plans, and the architecture questions the performance work will have to answer. Canonical sources: ../docs/05-operations/01-performance-correctness-security.md (the measurement contract), ../STATE-CURRENT.md (the dated snapshot) and RUNBOOK.md; the docs tree wins on disagreement.

Status: 2026-10-30, tree master @ b4930168. Tags: [live] measured with working code; [wip] measurement framework partial; [planned] designed, not built.

1. The performance model

The cost structure of a vGPU session splits into four layers, and each is measured separately:

Layer	What it costs	Representative metric
Initialization	VM boot → driver probe → cuInit	T_init
Control plane	Resource-manager request/reply round trips (objects, controls, memory)	RPC latency p50/p99
Steady-state data path	Submission → decode → forward → engine → completion → guest wake-up	push overhead; completion latency; kernel throughput
Data movement	Copies between guest memory and device memory	copy bandwidth (H2D/D2H/D2D)

Presentation (3D frames) sits on top of the data path, with its own budget (frame pacing, not just frame rate).

Three project rules govern every number:

1. Correctness before speed. A change that makes the vGPU faster but breaks the acceptance corpus does not count.
2. Proxy first, optimise by measurement. The architecture deliberately prototyped the safe path (emulate the interface, forward the work through the module's own GSP client) before considering a hot path; whether to move the hot path closer to direct submission is a measurement decision, not a design preference.
3. No number without methodology. Numbers are only quoted with their harness, versions, clocks and warm/cold state, and ideally as a ratio against a same-guest, same-hardware baseline (the A/B method below).

2. The metric set

Metric	Definition	Why it matters
T_init	VM boot → cuInit returns	user-visible startup
RPC latency p50/p99	request write → reply visible in the status queue	init and management overhead
Push overhead	guest submission → engine start (forwarded)	steady-state latency
Semaphore completion latency	release written → tracker observes	per-operation sync cost
Copy bandwidth	H2D/D2H/D2D, large and small	data-movement quality
Kernel throughput	kernels/second for a fixed workload	launch overhead
Multi-tenant fairness	per-tenant progress under contention	isolation quality
Frame pacing	inter-frame deltas under load	graphics smoothness (p95 is the budget)

3. How performance is traced today

• Host-side telemetry. The module timestamps its own work: per-run counters in the kernel log (accepted / failed / retried / refused work, decode outcomes, forward outcomes), trace_printk/tracepoints at dispatch entry/exit for distributions, and counter surfaces readable through the kernel's tracing and debug filesystems. Module parameters exist for targeted probes during bring-up — they are debug surfaces, not configuration (Interfaces-and-Stability.md).
• Guest-side instruments. The guest test clients print their own acceptance and timing lines (they are the ground truth for what the application observed); guest-side kernel probes and ptrace tools attribute driver-side waits when a client stalls.
• The A/B harness. Every measurement runs twice on the same machine and preferably the same boot: A = the workload on the physical GPU with the stock driver; B = the identical workload in the guest on the emulated vGPU. Results are reported as B/A ratios; warm and cold, with variance and run counts. One variable at a time; the harness is published so results are comparable.
• Offline tools — with a hard boundary. The risc-v-gsp-rm-debugger workbench models instruction semantics and memory behaviour only; it makes no timing claims (its fidelity levels explicitly exclude time and concurrency). Timing questions are answered on hardware, with instrumentation.
• The evaluation ladder. Performance is only meaningful from the execution rung upwards: initialization → runtime start → execution → sustained → multi-tenant. Claims are made per rung, never across it.

4. Where it stands today (honest)

There are no published performance numbers yet. The framework above is the contract; the first formally recorded measurements are scheduled, not done. What is observed today, clearly labelled as internal observation rather than benchmark:

• Forwarded CUDA launch latency is on the order of hundreds of milliseconds end-to-end on the development setup (one ten-client arc measured p50 ≈ 650 ms). This is the dominant known cost, dominated by the submission trap → decode → forward chain and by polled completion waits — not by the engine itself.
• Data movement is verified correct, not yet measured for throughput: copies through the forwarded path round-trip byte-exact; the A/B bandwidth runs (the nvbandwidth-class suite) are pending.
• Graphics frame delivery is far below interactive rates in the recorded pipeline runs — the defined target is 60 fps sustained with no forcing (Graphics-Execution.md); today's frame numbers come from development instrumentation and are explicitly not a benchmark.
• Initialization is functional: the backend's GSP boot reaches init-done in about a second on the reference hardware; T_init as a guest-visible measurement has not been taken.
• The software path is already cheap where it is work-shaped: decoded memory operations execute at host memory speed; control-plane replies are produced inline in the emulation thread; the expensive parts are the control-heavy structures (per-push passes, scanning, polling).

5. Observed gaps and the plan for each

Gap	Observation	Plan
Forwarded-launch latency	hundreds of ms (internal); dominated by the multi-step forward chain and completion polling	instrument the distribution end to end; reduce per-push passes; move completion to event-driven paths; re-measure per step
Completion latency	releases are detected by polling the engine/guest memory	prefer engine-authored release paths already used by verified launches; doorbell/event wake-ups instead of periodic scans where possible
RPC dispatch cost	all management funnels through one emulation thread	build the RPC round-trip microbenchmark first; batch and prioritise; reduce wake-ups
Scanning overhead	per-pass discovery/scan work (guest RAM, shadow apertures) exists for correctness while registrations are incomplete	convert discovery to registered/learned state so per-pass cost scales with work, not with memory size
BAR0 shadow reads	uncached to the guest, as MMIO must be; bulk reads would be slow	if a consumer bulk-reads, introduce cache-safe sub-regions rather than a slower guest
Memory-model gap	guest VRAM is backed by host RAM today	real VRAM backing is a target; DMA-heavy workloads are not representative until it exists
Graphics frame path	far below 60 fps; forcing/diagnostic assistance excluded by the target definition	the graphics milestone ladder: natural completions with no mirror, then measure pacing
Multi-tenant fairness	two guests coexist; fairness under load is untested	define the fairness metric, run two-VM contention, then add scheduling policy
Initialization	functional but unmeasured	take T_init and the reply-fidelity tail as the next control-plane measurement

6. The improvement programme

1. Build the microbenchmarks first — the four the measurement contract names: RPC round-trip, MSI-X delivery latency, BAR0 shadow read latency, and doorbell-polling cost. These attribute overhead to components before any optimisation.
2. Run and publish the A/B corpus — the canonical suite (vector-add, SGEMM, copy bandwidth, a GL frame-rate app, a framework op loop) with the contract's pass criteria (e.g. ≥95% B/A for compute, ≥90% for GL and framework loops) and full environment capture.
3. Optimise the observed hot structures — fewer per-push passes, batched decode, event-driven completion, shadow-window reuse — each as an A/B change with its numbers kept.
4. Evaluate the hot-path placement decision — keep the module as a GSP client proxy, or move submissions closer to direct channels — using the measured budgets as the criterion.
5. Publish the first graphics measurement — 60 fps sustained, no forcing, p95 inter-frame pacing — as the first end-to-end performance claim.
6. Then multi-tenant: define fairness, measure it, and only then claim it.

7. Architecture areas that may need to improve — open questions

These are the structural questions the performance programme exists to close:

• One ordered workqueue, one in-flight request. The current design serialises forwarded work for ordering safety. Is that a throughput ceiling for multi-channel/multi-launch workloads, and how much pipelining is possible without weakening the ordering guarantees the guest observes?
• Per-pass scanning vs registered state. Ring/USERD discovery, content scans and shadow diffs are correctness mechanisms; can they be replaced (or narrowed) by registrations learned on first sight, so steady-state cost does not scale with aperture size?
• Completion publication machinery. Which of poll, engine-authored release, notifier and event paths should carry the steady state — and what is the p99, not just the median?
• MMIO trap frequency. Doorbells and shadow writes pay a VM-exit cost; which guest patterns produce the most traps, and can they be coalesced or served with less work?
• The transport itself. Guest-system-memory message queues plus doorbells are deliberately conservative; would a shared-ring fast path change the budget materially?
• Hot-path placement. Proxy through the module's own GSP session vs moving hot submissions closer to the hardware — decided by the A/B numbers, not by preference.
• Multi-tenant scheduling. Time-slicing exists only as a default; the fairness policy (and MIG-backed partitioning where available) is a design area, with fairness measurements defined before the claim.
• Fidelity boundary. The offline simulator can never answer timing or concurrency questions; the performance programme therefore owns all timing claims, and no timing claim may cite a simulator session.

8. Related pages

• Interfaces-and-Stability.md — which surfaces are contracts and which are free to change.
• Execution-Frontier.md — the compute path whose latency this page budgets.
• Graphics-Execution.md — the 60 fps target and its milestone ladder.
• Physical-GPU-Backend.md — the backend the work is forwarded to.
• Host-Module.md — the module whose threads, queues and scans carry the costs above.
• ../docs/05-operations/01-performance-correctness-security.md — the measurement contract: ladder, metrics, A/B method, microbenchmarks, targets.