Difference between revisions of "Project Status"
m (2026-11-14 status refresh: direct-GL submission decoded (macro-driven), push rescues land, draw-state push reaches the engine; MME multi-part completion is the wall; X start lottery is the limiter) |
m (2026-11-15 status refresh: DS stage image authored+live (st0=st1=020a0481; DS 0x86040626->0x84000420, two boots); draw-issue mac=0x22 and stores remain; X start lottery limits the run rate) |
||
| Line 5: | Line 5: | ||
<blockquote>'''What this page is.''' The honest, dated state of the project: what works today, what is in progress, and what is not started. It is refreshed periodically and each claim maps to a maintained document in the repository. Canonical sources: [https://git.vgpu.io/open-iov.git/tree/docs/README.md <code>../docs/README.md</code>] (project front door), [https://git.vgpu.io/open-iov.git/tree/docs/04-implementation/01-known-knowns-and-unknowns.md <code>../docs/04-implementation/01-known-knowns-and-unknowns.md</code>] (known-knowns and known-unknowns) and [https://git.vgpu.io/open-iov.git/tree/docs/04-implementation/03-execution-frontier.md <code>../docs/04-implementation/03-execution-frontier.md</code>] (the execution frontier). When this page and the docs disagree, the docs tree wins.</blockquote> | <blockquote>'''What this page is.''' The honest, dated state of the project: what works today, what is in progress, and what is not started. It is refreshed periodically and each claim maps to a maintained document in the repository. Canonical sources: [https://git.vgpu.io/open-iov.git/tree/docs/README.md <code>../docs/README.md</code>] (project front door), [https://git.vgpu.io/open-iov.git/tree/docs/04-implementation/01-known-knowns-and-unknowns.md <code>../docs/04-implementation/01-known-knowns-and-unknowns.md</code>] (known-knowns and known-unknowns) and [https://git.vgpu.io/open-iov.git/tree/docs/04-implementation/03-execution-frontier.md <code>../docs/04-implementation/03-execution-frontier.md</code>] (the execution frontier). When this page and the docs disagree, the docs tree wins.</blockquote> | ||
Status: 2026-11- | Status: 2026-11-15, tree <code>master @ 692a7eb0</code>. Tags: <code>[live]</code> proven with the stock guest driver; <code>[gated]</code> implemented, default-off or behind supervised gates; <code>[wip]</code> partially implemented; <code>[wall]</code> current blocker; <code>[target]</code> named success state; <code>[planned]</code> designed, not built. | ||
== 1. Milestone ladder == | == 1. Milestone ladder == | ||
| Line 42: | Line 42: | ||
|- | |- | ||
| OpenGL (3D graphics) | | OpenGL (3D graphics) | ||
| <code>[wip]</code> — the display/KMS stack registers, X and the stock NVIDIA GLX path are fully green (<code>glxinfo -B</code> returns 0 with OpenGL 4.6.0 and direct rendering; <code>glxgears</code> renders ~520 FPS with the emulated GPU's NVIDIA renderer), and the '''first non-black GPU-rendered output''' is proven: the pbuffer frame path produced 500/500 non-black 640x480 frames per run and captured videos (45.4 s on 2026-11-11; a 50 s 640x480 h264 capture on 2026-11-12). GZDoom reaches <code>V_Init: allocate screen</code> with its GL 4.6.0 context whenever the shared nvClip <code>modeNum</code> is even. '''The direct-GL submission is now fully decoded:''' no client push carries a method-grammar draw header — the channel's 9 NV906F entries tile one contiguous buffer and its per-frame work is carried as rewritten data-block blobs plus small state pushes (324/476/728 B) that invoke MME macros (the QMD-carrier hypothesis was tested and refuted). A content-gated FB-shadow rescue and a LOW-band rescue recover the recycled pushes: the client's draw-state push now opens a C597 section, is captured and marked for the verbatim 3D lane (the first direct-GL stream3d marks), and reaches the engine (<code>GR LAUNCH submitted</code>/<code>OK</code>, golden MME overlay accepted). '''The | | <code>[wip]</code> — the display/KMS stack registers, X and the stock NVIDIA GLX path are fully green (<code>glxinfo -B</code> returns 0 with OpenGL 4.6.0 and direct rendering; <code>glxgears</code> renders ~520 FPS with the emulated GPU's NVIDIA renderer), and the '''first non-black GPU-rendered output''' is proven: the pbuffer frame path produced 500/500 non-black 640x480 frames per run and captured videos (45.4 s on 2026-11-11; a 50 s 640x480 h264 capture on 2026-11-12). GZDoom reaches <code>V_Init: allocate screen</code> with its GL 4.6.0 context whenever the shared nvClip <code>modeNum</code> is even. '''The direct-GL submission is now fully decoded:''' no client push carries a method-grammar draw header — the channel's 9 NV906F entries tile one contiguous buffer and its per-frame work is carried as rewritten data-block blobs plus small state pushes (324/476/728 B) that invoke MME macros (the QMD-carrier hypothesis was tested and refuted). A content-gated FB-shadow rescue and a LOW-band rescue recover the recycled pushes: the client's draw-state push now opens a C597 section, is captured and marked for the verbatim 3D lane (the first direct-GL stream3d marks), and reaches the engine (<code>GR LAUNCH submitted</code>/<code>OK</code>, golden MME overlay accepted). '''The MME chain now completes and executes the draw macro.''' The multi-part keep-window mechanism won multi-frame windows; the part-launch poison was eliminated (the real fault was the client's own <code>SET_REPORT_SEMAPHORE</code>/<code>SET_NOTIFY</code> writes to client VAs that the host scratch page-table pool remaps — redirected to a mapped scratch slot; 49 cycles with zero poison and zero MMU faults); the 2799-word pack completes (2799/2799); engine-global completion plus keep-done-across-recycle produced the first <code>MME-PACK-REARM</code>; the fixed-CB region is aliased; and the released calls run macros through to '''<code>DRAW_IAB</code> (<code>mac=0x22</code>)'''. Two synthesized-prelude defects were then fixed: the SPH-BIND prelude self-destructed because an auto-dropped run broke its kept/final count (<code>444ae2c1</code>), and the stage block was silently skipped by a double-counted budget; with both fixed the prelude emits 84 words and the DS stage VAs populate (one validated boot carried the full post-REARM chain, <code>mac=0x22</code> and <code>DS 86040624</code>). '''The DS stage-0/1 image is now authored and live''': a VTG SPH is written at <code>family+0x900</code> (<code>SPH-AUTHOR</code>, first word <code>020a0481</code>), the window's own later <code>0x2054</code> slot-1 bind is rewritten to the authored page (<code>SPH-SLOT1-REWRITE</code>), and the DS latch reads the authored image for both vertex slots (<code>DS-IMAGE st0=st1=020a0481 code=ff007321</code>; the DS value fell <code>0x86040626 -> 0x84000420</code>), reproduced on two boots. The remaining DS bits are stage type/context (SPH5/SPH10/SPH18), and the '''<code>DRAW_IAB</code> (<code>mac=0x22</code>)''' call does not run in the 728-B draw window (its released macros are <code>0x33/0x39/0x1/0x0</code>), so stores still miss (<code>wrote=0</code>, <code>px0=0</code>); the window/screen present path is unproven; and the per-boot binding lottery (the GL client usually rides the X channel in indirect mode, and X start keeps a first-generation lottery) is the throughput limiter ([[OpenGL|<code>OpenGL.md</code>]]) | ||
|- | |- | ||
| Vulkan (3D/compute) | | Vulkan (3D/compute) | ||
| Line 79: | Line 79: | ||
== 3. How to see it yourself == | == 3. How to see it yourself == | ||
The reference setup takes about half an hour from a fresh host ([[Install and First Run|<code>Install-and-First-Run.md</code>]]); the current expected outcome is exactly the ladder above — driver probe, <code>nvidia-smi</code>, the full driver-API smoke, and guest CUDA kernels executing and verifying on the physical GPU. On the graphics side the same setup brings up the display/KMS stack and the X server on the emulated vGPU (<code>glxinfo</code> reports OpenGL 4.6.0 with direct rendering, <code>glxgears</code> renders ~520 FPS); the first non-black GPU-rendered frames are proven off-screen (the pbuffer path produced a 500-frame 45.4 s video on 2026-11-11 and a 50 s capture on 2026-11-12), and GZDoom reaches its screen-allocation stage with GL 4.6.0 on an even modeNum. The direct-GL submission is fully decoded | The reference setup takes about half an hour from a fresh host ([[Install and First Run|<code>Install-and-First-Run.md</code>]]); the current expected outcome is exactly the ladder above — driver probe, <code>nvidia-smi</code>, the full driver-API smoke, and guest CUDA kernels executing and verifying on the physical GPU. On the graphics side the same setup brings up the display/KMS stack and the X server on the emulated vGPU (<code>glxinfo</code> reports OpenGL 4.6.0 with direct rendering, <code>glxgears</code> renders ~520 FPS); the first non-black GPU-rendered frames are proven off-screen (the pbuffer path produced a 500-frame 45.4 s video on 2026-11-11 and a 50 s capture on 2026-11-12), and GZDoom reaches its screen-allocation stage with GL 4.6.0 on an even modeNum. The direct-GL submission is fully decoded, the MME multi-part program now completes and the draw macro executes, and the DS stage image is authored and live for both vertex slots; but windowed/screen pixels remain the frontier (the draw-issue <code>mac=0x22</code> and the stores are the next named gates, and X start keeps a first-generation lottery) — see [[OpenGL|<code>OpenGL.md</code>]]. The risc-v-gsp-rm-debugger workbench ([[risc-v-gsp-rm-debugger|<code>Riscv-GSP-RM-Debugger.md</code>]]) is the host-side tooling used to study the firmware side of that frontier. | ||
== 4. How this page is maintained == | == 4. How this page is maintained == | ||
Revision as of 19:09, 15 November 2026
An absence of critical technical documentation has historically slowed growth and adoption of developer ecosystems for GPU virtualization.
This CC-BY-4.0 licensed content can either be used with attribution, or used as inspiration for new documentation, created by GPU vendors for public commercial distribution as developer documentation.
Where possible, this documentation will clearly label dates and versions of observed-but-not-guaranteed behaviour vs. vendor-documented stable interfaces/behaviour with guarantees of forward or backward compatibility.
Project Status
What this page is. The honest, dated state of the project: what works today, what is in progress, and what is not started. It is refreshed periodically and each claim maps to a maintained document in the repository. Canonical sources:
../docs/README.md(project front door),../docs/04-implementation/01-known-knowns-and-unknowns.md(known-knowns and known-unknowns) and../docs/04-implementation/03-execution-frontier.md(the execution frontier). When this page and the docs disagree, the docs tree wins.
Status: 2026-11-15, tree master @ 692a7eb0. Tags: [live] proven with the stock guest driver; [gated] implemented, default-off or behind supervised gates; [wip] partially implemented; [wall] current blocker; [target] named success state; [planned] designed, not built.
1. Milestone ladder
| Milestone | State |
|---|---|
| Emulated vGPU device (PCI, BARs, MSI-X, VFIO mdev) | [live] working
|
| GSP boot emulation (FWSEC / booter / RISCV handshake) | [live] working
|
| GSP-RM message queues and RPC transport | [live] working
|
Stock guest driver probe and nvidia-smi (TU104)
|
[live] working
|
CUDA userspace init and the driver-API smoke (cuInit, device enumeration, cuCtxCreate, allocation, copies, free, destroy)
|
[live] working — the smoke prints ALL OK and the 64-byte payload round-trips exactly
|
| Guest CUDA kernel execution on real hardware | [live] working — verified green on fresh boots: the vector-add (VADD n=32768 bad=0), matrix-multiply (MATMUL n=256 bad=0) and transformer (dh=256 seq=64 bad=0, FFN4 layers=4 bad=0) clients print STATUS OK, engine-authored completions relay to the guest and its synchronisation waits resolve. A ring-block ownership defect that could stall the first client after a reboot was root-caused and fixed. Remaining: the staged-transfer class (some large multi-chunk clients still mis-verify; the failure modes are proven and under repair) and per-guest execution isolation for simultaneous CUDA guests
|
| Physical backend on real hardware (module-owned GPU running its real GSP-RM) | [live] boot, RPC session, object creation, copy-engine work and GR execution proven — the verified guest CUDA kernel clients execute through this backend, and graphics method streams execute on the engine (Graphics-Execution.md)
|
Display and KMS on the emulated vGPU (nvidia-modeset/NvKms, /dev/dri, X)
|
[live] in the campaign configuration — the emulated display engine serves the guest's display push-buffer channels (notifier plus UPDATE -> FINISHED), nvidia-modeset/NvKms registers (/dev/dri/card0, renderD128, fb0), nvidia-smi -L enumerates and X settles. The X DDX idle-DMA/ScreenInit, USERD-latch lifetime, re-probe-thrash, Xorg kernel-mode-spin and flip-event walls were closed (X green for >10 minutes in a validated window); what remains is the per-boot first-generation X start lottery, which by 11-13/11-14 was the dominant throughput limiter for the graphics A/B runs (only an X started immediately after boot passes DDX DMA init; long stretches of dead starts, one green first-start boot). The display-capability seam is parameterised (Graphics-Execution.md)
|
| Multi-tenant isolation (mdev per VM, per-context page directories) | [wip] device-level coexistence works — multiple mediated devices with different emulated VRAM sizes and multiple guests coexist (two guests demonstrated); per-guest execution isolation for simultaneous CUDA guests is in progress
|
| OpenGL (3D graphics) | [wip] — the display/KMS stack registers, X and the stock NVIDIA GLX path are fully green (glxinfo -B returns 0 with OpenGL 4.6.0 and direct rendering; glxgears renders ~520 FPS with the emulated GPU's NVIDIA renderer), and the first non-black GPU-rendered output is proven: the pbuffer frame path produced 500/500 non-black 640x480 frames per run and captured videos (45.4 s on 2026-11-11; a 50 s 640x480 h264 capture on 2026-11-12). GZDoom reaches V_Init: allocate screen with its GL 4.6.0 context whenever the shared nvClip modeNum is even. The direct-GL submission is now fully decoded: no client push carries a method-grammar draw header — the channel's 9 NV906F entries tile one contiguous buffer and its per-frame work is carried as rewritten data-block blobs plus small state pushes (324/476/728 B) that invoke MME macros (the QMD-carrier hypothesis was tested and refuted). A content-gated FB-shadow rescue and a LOW-band rescue recover the recycled pushes: the client's draw-state push now opens a C597 section, is captured and marked for the verbatim 3D lane (the first direct-GL stream3d marks), and reaches the engine (GR LAUNCH submitted/OK, golden MME overlay accepted). The MME chain now completes and executes the draw macro. The multi-part keep-window mechanism won multi-frame windows; the part-launch poison was eliminated (the real fault was the client's own SET_REPORT_SEMAPHORE/SET_NOTIFY writes to client VAs that the host scratch page-table pool remaps — redirected to a mapped scratch slot; 49 cycles with zero poison and zero MMU faults); the 2799-word pack completes (2799/2799); engine-global completion plus keep-done-across-recycle produced the first MME-PACK-REARM; the fixed-CB region is aliased; and the released calls run macros through to DRAW_IAB (mac=0x22). Two synthesized-prelude defects were then fixed: the SPH-BIND prelude self-destructed because an auto-dropped run broke its kept/final count (444ae2c1), and the stage block was silently skipped by a double-counted budget; with both fixed the prelude emits 84 words and the DS stage VAs populate (one validated boot carried the full post-REARM chain, mac=0x22 and DS 86040624). The DS stage-0/1 image is now authored and live: a VTG SPH is written at family+0x900 (SPH-AUTHOR, first word 020a0481), the window's own later 0x2054 slot-1 bind is rewritten to the authored page (SPH-SLOT1-REWRITE), and the DS latch reads the authored image for both vertex slots (DS-IMAGE st0=st1=020a0481 code=ff007321; the DS value fell 0x86040626 -> 0x84000420), reproduced on two boots. The remaining DS bits are stage type/context (SPH5/SPH10/SPH18), and the DRAW_IAB (mac=0x22) call does not run in the 728-B draw window (its released macros are 0x33/0x39/0x1/0x0), so stores still miss (wrote=0, px0=0); the window/screen present path is unproven; and the per-boot binding lottery (the GL client usually rides the X channel in indirect mode, and X start keeps a first-generation lottery) is the throughput limiter (OpenGL.md)
|
| Vulkan (3D/compute) | [wip] — the guest Vulkan pipeline runs end to end on the emulated vGPU (device, allocation, pipeline, command buffer, fence, queue submit, readback) and the engine executes translated launches; frames are still black — program delivery and program-VA mapping are the frontier (Vulkan.md)
|
| DirectX / Windows guests | [planned] (target) — no Windows guest booted yet; the architecture is the same emulation target (DirectX.md)
|
| Video (NVENC/NVDEC) | [planned]
|
| Xen / Argo / OpenXT / Qubes Air transports | [planned] (the emulation core is hypervisor-independent by construction)
|
| Live migration | [planned]
|
| Performance numbers | [planned] — no published numbers yet; any claim will ship with its measurement methodology
|
2. What "CUDA works" means here
A green nvidia-smi or even a passing driver-API smoke is not "CUDA works". The project's definition of done for CUDA requires all of the following (../docs/04-implementation/01-known-knowns-and-unknowns.md):
- Stock guest driver probe,
nvidia-smi, device enumeration — done. - Context creation and the full driver-API surface (
cuMemAlloc, copies, free, destroy) — done. - A guest kernel launch that executes on real hardware and is proven by an engine-authored completion — demonstrated: the verified clients execute on the physical engine and the engine-authored completions relay to the guest.
- Memory the guest's work refers to resolving correctly end to end (the VA↔PA model) — partially, for the paths exercised so far.
- Correct completion, semaphore and fault semantics back to the guest — the GR completion path now relays (engine-authored releases wake the guest's synchronisation waits); fault semantics remain partial.
- Reproducibility: every boot, not a lucky one — two root causes landed: the boot-poisoning defect (the wrong status-queue consumption cursor could desynchronise the reply ring under load) and the ring-block ownership defect that could stall the first client after a reboot. Green clients are reproduced across fresh boots; a formal every-boot acceptance campaign is the remaining work.
- No ad-hoc default-off parameters required for the canonical path — still open (the forwarding and graphics seams remain parameterised while they are validated).
Until every gate holds, the honest formulation is: initialization, the driver-API path and kernel execution on the physical GPU work and verify; the staged-transfer class, per-guest isolation and the graphics direct-GL/present/first-frame frontier remain.
3. How to see it yourself
The reference setup takes about half an hour from a fresh host (Install-and-First-Run.md); the current expected outcome is exactly the ladder above — driver probe, nvidia-smi, the full driver-API smoke, and guest CUDA kernels executing and verifying on the physical GPU. On the graphics side the same setup brings up the display/KMS stack and the X server on the emulated vGPU (glxinfo reports OpenGL 4.6.0 with direct rendering, glxgears renders ~520 FPS); the first non-black GPU-rendered frames are proven off-screen (the pbuffer path produced a 500-frame 45.4 s video on 2026-11-11 and a 50 s capture on 2026-11-12), and GZDoom reaches its screen-allocation stage with GL 4.6.0 on an even modeNum. The direct-GL submission is fully decoded, the MME multi-part program now completes and the draw macro executes, and the DS stage image is authored and live for both vertex slots; but windowed/screen pixels remain the frontier (the draw-issue mac=0x22 and the stores are the next named gates, and X start keeps a first-generation lottery) — see OpenGL.md. The risc-v-gsp-rm-debugger workbench (Riscv-GSP-RM-Debugger.md) is the host-side tooling used to study the firmware side of that frontier.
4. How this page is maintained
Every claim above carries a date and a tag, maps to a maintained document in the repository, and is refreshed as milestones land. The execution story, with its evidence, its resolved theories and its open work, is on Execution-Frontier.md.
5. Related pages
Install-and-First-Run.mdSystem-Overview.mdExecution-Frontier.mdHardware-Software-Matrix.md— which hardware and software combinations are Reference, Target or Experimental.FAQ.md