WikiBot: 2026-10-30 wiki refresh: goals, GSP, graphics, performance, interfaces

2026-10-30T17:06:16Z

2026-10-30 wiki refresh: goals, GSP, graphics, performance, interfaces

New page

{{About message}}

= Performance =

<blockquote>'''Scope.''' How Open-IOV's performance is modelled, traced, and planned: the cost structure of the system, the metric set, what is observed today (honestly — there are no published numbers yet), the known gaps with their improvement plans, and the architecture questions the performance work will have to answer. Canonical sources: [https://git.vgpu.io/open-iov.git/tree/docs/05-operations/01-performance-correctness-security.md <code>../docs/05-operations/01-performance-correctness-security.md</code>] (the measurement contract), [https://git.vgpu.io/open-iov.git/tree/STATE-CURRENT.md <code>../STATE-CURRENT.md</code>] (the dated snapshot) and [https://git.vgpu.io/open-iov.git/tree/docs/RUNBOOK.md <code>RUNBOOK.md</code>]; the docs tree wins on disagreement.</blockquote>

Status: 2026-10-30, tree <code>master @ b4930168</code>. Tags: <code>[live]</code> measured with working code; <code>[wip]</code> measurement framework partial; <code>[planned]</code> designed, not built.

== 1. The performance model ==

The cost structure of a vGPU session splits into four layers, and each is measured separately:

{| class="wikitable"
|-
! Layer
! What it costs
! Representative metric
|-
| '''Initialization'''
| VM boot → driver probe → <code>cuInit</code>
| <code>T_init</code>
|-
| '''Control plane'''
| Resource-manager request/reply round trips (objects, controls, memory)
| RPC latency p50/p99
|-
| '''Steady-state data path'''
| Submission → decode → forward → engine → completion → guest wake-up
| push overhead; completion latency; kernel throughput
|-
| '''Data movement'''
| Copies between guest memory and device memory
| copy bandwidth (H2D/D2H/D2D)
|}

Presentation (3D frames) sits on top of the data path, with its own budget (frame pacing, not just frame rate).

Three project rules govern every number:

# '''Correctness before speed.''' A change that makes the vGPU faster but breaks the acceptance corpus does not count.
# '''Proxy first, optimise by measurement.''' The architecture deliberately prototyped the safe path (emulate the interface, forward the work through the module's own GSP client) before considering a hot path; whether to move the hot path closer to direct submission is a ''measurement decision'', not a design preference.
# '''No number without methodology.''' Numbers are only quoted with their harness, versions, clocks and warm/cold state, and ideally as a ratio against a same-guest, same-hardware baseline (the A/B method below).

== 2. The metric set ==

{| class="wikitable"
|-
! Metric
! Definition
! Why it matters
|-
| <code>T_init</code>
| VM boot → <code>cuInit</code> returns
| user-visible startup
|-
| RPC latency p50/p99
| request write → reply visible in the status queue
| init and management overhead
|-
| Push overhead
| guest submission → engine start (forwarded)
| steady-state latency
|-
| Semaphore completion latency
| release written → tracker observes
| per-operation sync cost
|-
| Copy bandwidth
| H2D/D2H/D2D, large and small
| data-movement quality
|-
| Kernel throughput
| kernels/second for a fixed workload
| launch overhead
|-
| Multi-tenant fairness
| per-tenant progress under contention
| isolation quality
|-
| Frame pacing
| inter-frame deltas under load
| graphics smoothness (p95 is the budget)
|}

== 3. How performance is traced today ==

* '''Host-side telemetry.''' The module timestamps its own work: per-run counters in the kernel log (accepted / failed / retried / refused work, decode outcomes, forward outcomes), <code>trace_printk</code>/tracepoints at dispatch entry/exit for distributions, and counter surfaces readable through the kernel's tracing and debug filesystems. Module parameters exist for targeted probes during bring-up — they are debug surfaces, not configuration ([[Interfaces and Stability|<code>Interfaces-and-Stability.md</code>]]).
* '''Guest-side instruments.''' The guest test clients print their own acceptance ''and'' timing lines (they are the ground truth for what the application observed); guest-side kernel probes and ptrace tools attribute driver-side waits when a client stalls.
* '''The A/B harness.''' Every measurement runs twice on the same machine and preferably the same boot: '''A''' = the workload on the physical GPU with the stock driver; '''B''' = the identical workload in the guest on the emulated vGPU. Results are reported as '''B/A ratios'''; warm and cold, with variance and run counts. One variable at a time; the harness is published so results are comparable.
* '''Offline tools — with a hard boundary.''' The risc-v-gsp-rm-debugger workbench models ''instruction semantics and memory behaviour only''; it makes '''no timing claims''' (its fidelity levels explicitly exclude time and concurrency). Timing questions are answered on hardware, with instrumentation.
* '''The evaluation ladder.''' Performance is only meaningful from the ''execution'' rung upwards: initialization → runtime start → execution → sustained → multi-tenant. Claims are made per rung, never across it.

== 4. Where it stands today (honest) ==

'''There are no published performance numbers yet.''' The framework above is the contract; the first formally recorded measurements are scheduled, not done. What ''is'' observed today, clearly labelled as internal observation rather than benchmark:

* '''Forwarded CUDA launch latency is on the order of hundreds of milliseconds''' end-to-end on the development setup (one ten-client arc measured p50 ≈ 650 ms). This is the dominant known cost, dominated by the submission trap → decode → forward chain and by polled completion waits — not by the engine itself.
* '''Data movement is verified correct, not yet measured for throughput''': copies through the forwarded path round-trip byte-exact; the A/B bandwidth runs (the <code>nvbandwidth</code>-class suite) are pending.
* '''Graphics frame delivery is far below interactive rates''' in the recorded pipeline runs — the defined target is 60 fps ''sustained with no forcing'' ([[Graphics Execution|<code>Graphics-Execution.md</code>]]); today's frame numbers come from development instrumentation and are explicitly not a benchmark.
* '''Initialization is functional''': the backend's GSP boot reaches init-done in about a second on the reference hardware; <code>T_init</code> as a guest-visible measurement has not been taken.
* '''The software path is already cheap where it is work-shaped''': decoded memory operations execute at host memory speed; control-plane replies are produced inline in the emulation thread; the expensive parts are the ''control-heavy'' structures (per-push passes, scanning, polling).

== 5. Observed gaps and the plan for each ==

{| class="wikitable"
|-
! Gap
! Observation
! Plan
|-
| Forwarded-launch latency
| hundreds of ms (internal); dominated by the multi-step forward chain and completion polling
| instrument the distribution end to end; reduce per-push passes; move completion to event-driven paths; re-measure per step
|-
| Completion latency
| releases are detected by polling the engine/guest memory
| prefer engine-authored release paths already used by verified launches; doorbell/event wake-ups instead of periodic scans where possible
|-
| RPC dispatch cost
| all management funnels through one emulation thread
| build the RPC round-trip microbenchmark first; batch and prioritise; reduce wake-ups
|-
| Scanning overhead
| per-pass discovery/scan work (guest RAM, shadow apertures) exists for correctness while registrations are incomplete
| convert discovery to registered/learned state so per-pass cost scales with ''work'', not with memory size
|-
| BAR0 shadow reads
| uncached to the guest, as MMIO must be; bulk reads would be slow
| if a consumer bulk-reads, introduce cache-safe sub-regions rather than a slower guest
|-
| Memory-model gap
| guest VRAM is backed by host RAM today
| real VRAM backing is a target; DMA-heavy workloads are not representative until it exists
|-
| Graphics frame path
| far below 60 fps; forcing/diagnostic assistance excluded by the target definition
| the graphics milestone ladder: natural completions with no mirror, then measure pacing
|-
| Multi-tenant fairness
| two guests coexist; fairness under load is untested
| define the fairness metric, run two-VM contention, then add scheduling policy
|-
| Initialization
| functional but unmeasured
| take <code>T_init</code> and the reply-fidelity tail as the next control-plane measurement
|}

== 6. The improvement programme ==

# '''Build the microbenchmarks first''' — the four the measurement contract names: RPC round-trip, MSI-X delivery latency, BAR0 shadow read latency, and doorbell-polling cost. These attribute overhead to components before any optimisation.
# '''Run and publish the A/B corpus''' — the canonical suite (vector-add, SGEMM, copy bandwidth, a GL frame-rate app, a framework op loop) with the contract's pass criteria (e.g. ≥95% B/A for compute, ≥90% for GL and framework loops) and full environment capture.
# '''Optimise the observed hot structures''' — fewer per-push passes, batched decode, event-driven completion, shadow-window reuse — each as an A/B change with its numbers kept.
# '''Evaluate the hot-path placement decision''' — keep the module as a GSP client proxy, or move submissions closer to direct channels — using the measured budgets as the criterion.
# '''Publish the first graphics measurement''' — 60 fps sustained, no forcing, p95 inter-frame pacing — as the first end-to-end performance claim.
# '''Then multi-tenant''': define fairness, measure it, and only then claim it.

== 7. Architecture areas that may need to improve — open questions ==

These are the structural questions the performance programme exists to close:

* '''One ordered workqueue, one in-flight request.''' The current design serialises forwarded work for ordering safety. Is that a throughput ceiling for multi-channel/multi-launch workloads, and how much pipelining is possible without weakening the ordering guarantees the guest observes?
* '''Per-pass scanning vs registered state.''' Ring/USERD discovery, content scans and shadow diffs are correctness mechanisms; can they be replaced (or narrowed) by registrations learned on first sight, so steady-state cost does not scale with aperture size?
* '''Completion publication machinery.''' Which of poll, engine-authored release, notifier and event paths should carry the steady state — and what is the p99, not just the median?
* '''MMIO trap frequency.''' Doorbells and shadow writes pay a VM-exit cost; which guest patterns produce the most traps, and can they be coalesced or served with less work?
* '''The transport itself.''' Guest-system-memory message queues plus doorbells are deliberately conservative; would a shared-ring fast path change the budget materially?
* '''Hot-path placement.''' Proxy through the module's own GSP session vs moving hot submissions closer to the hardware — decided by the A/B numbers, not by preference.
* '''Multi-tenant scheduling.''' Time-slicing exists only as a default; the fairness policy (and MIG-backed partitioning where available) is a design area, with fairness measurements defined before the claim.
* '''Fidelity boundary.''' The offline simulator can never answer timing or concurrency questions; the performance programme therefore owns all timing claims, and no timing claim may cite a simulator session.

== 8. Related pages ==

* [[Interfaces and Stability|<code>Interfaces-and-Stability.md</code>]] — which surfaces are contracts and which are free to change.
* [[The Execution Frontier|<code>Execution-Frontier.md</code>]] — the compute path whose latency this page budgets.
* [[Graphics Execution|<code>Graphics-Execution.md</code>]] — the 60 fps target and its milestone ladder.
* [[Physical GPU Backend and Guest Wiring|<code>Physical-GPU-Backend.md</code>]] — the backend the work is forwarded to.
* [[The Host Module (vgsp.ko)|<code>Host-Module.md</code>]] — the module whose threads, queues and scans carry the costs above.
* [https://git.vgpu.io/open-iov.git/tree/docs/05-operations/01-performance-correctness-security.md <code>../docs/05-operations/01-performance-correctness-security.md</code>] — the measurement contract: ladder, metrics, A/B method, microbenchmarks, targets.

Performance - Revision history

WikiBot: 2026-10-30 wiki refresh: goals, GSP, graphics, performance, interfaces