Difference between revisions of "Graphics Execution"

From Open-IOV
Jump to navigation Jump to search
m (2026-11-10 afternoon refresh: first real GPU readback (pbuffer red pixel); direct-GL draw capture and window/screen present are the walls)
m (2026-11-11 refresh: first non-black GPU frames + 45.4 s video; direct-GL GR ring resolves; CE-ring/window walls)
Line 5: Line 5:
<blockquote>'''Scope.''' How the emulated vGPU runs '''3D graphics''' work — the OpenGL and Vulkan command streams a stock guest submits — on the physical GPU, and what is proven about it. This is the shared machinery behind [[OpenGL|<code>OpenGL.md</code>]], [[Vulkan|<code>Vulkan.md</code>]] and the Windows/DirectX target ([[DirectX|<code>DirectX.md</code>]]); CUDA compute uses the same submission and completion path for a different engine workload ([[The Execution Frontier|<code>Execution-Frontier.md</code>]]). Canonical sources: [https://git.vgpu.io/open-iov.git/tree/docs/04-implementation/05-graphics-execution-plan.md <code>../docs/04-implementation/05-graphics-execution-plan.md</code>] (the plan and milestone ladder) and [https://git.vgpu.io/open-iov.git/tree/docs/04-implementation/04-real-render-walls.md <code>../docs/04-implementation/04-real-render-walls.md</code>] (the measured walls); the docs tree wins on disagreement.</blockquote>
<blockquote>'''Scope.''' How the emulated vGPU runs '''3D graphics''' work — the OpenGL and Vulkan command streams a stock guest submits — on the physical GPU, and what is proven about it. This is the shared machinery behind [[OpenGL|<code>OpenGL.md</code>]], [[Vulkan|<code>Vulkan.md</code>]] and the Windows/DirectX target ([[DirectX|<code>DirectX.md</code>]]); CUDA compute uses the same submission and completion path for a different engine workload ([[The Execution Frontier|<code>Execution-Frontier.md</code>]]). Canonical sources: [https://git.vgpu.io/open-iov.git/tree/docs/04-implementation/05-graphics-execution-plan.md <code>../docs/04-implementation/05-graphics-execution-plan.md</code>] (the plan and milestone ladder) and [https://git.vgpu.io/open-iov.git/tree/docs/04-implementation/04-real-render-walls.md <code>../docs/04-implementation/04-real-render-walls.md</code>] (the measured walls); the docs tree wins on disagreement.</blockquote>


Status: 2026-11-09, tree <code>master @ df6e71bb</code>. Tags: <code>[live]</code> observed end to end; <code>[wip]</code> partially built; <code>[wall]</code> current blocker; <code>[target]</code> defined success state; <code>[planned]</code> designed, not built.
Status: 2026-11-11, tree <code>master @ 4acac093</code>. Tags: <code>[live]</code> observed end to end; <code>[wip]</code> partially built; <code>[wall]</code> current blocker; <code>[target]</code> defined success state; <code>[planned]</code> designed, not built.


== 1. What "real pixels" means ==
== 1. What "real pixels" means ==
Line 33: Line 33:
# '''Verbatim 3D method-stream replay''' <code>[live]</code>. The emulation captures the X server's graphics-class (<code>0xC597</code>, <code>TURING_A</code>) section from its channel, relocates the operand addresses into host-built shadow pages the engine can reach, and submits the stream unchanged. The engine retires it: the uploads (<code>LINE_LENGTH_IN</code>/<code>LAUNCH_DMA</code>), the state, program and texture binds, and the release all execute with no MMU fault. This proves the ''mechanics'' of 3D execution — object bind, window placement, operand relocation, completion — on real hardware.
# '''Verbatim 3D method-stream replay''' <code>[live]</code>. The emulation captures the X server's graphics-class (<code>0xC597</code>, <code>TURING_A</code>) section from its channel, relocates the operand addresses into host-built shadow pages the engine can reach, and submits the stream unchanged. The engine retires it: the uploads (<code>LINE_LENGTH_IN</code>/<code>LAUNCH_DMA</code>), the state, program and texture binds, and the release all execute with no MMU fault. This proves the ''mechanics'' of 3D execution — object bind, window placement, operand relocation, completion — on real hardware.
# '''Synthesized launch from a captured section''' <code>[live]</code>. A compute-shaped launch built from the captured section (operand list, program, a 64-word launch descriptor) reaches the engine and retires real <code>sm_75</code> SASS; the Turing shader-program-header form (<code>V3</code>) is accepted with zero header faults across repeated launches. This proved engine-side dispatch before the verbatim lane existed. The program itself is still a development stand-in at this rung — execution proof, not pixels.
# '''Synthesized launch from a captured section''' <code>[live]</code>. A compute-shaped launch built from the captured section (operand list, program, a 64-word launch descriptor) reaches the engine and retires real <code>sm_75</code> SASS; the Turing shader-program-header form (<code>V3</code>) is accepted with zero header faults across repeated launches. This proved engine-side dispatch before the verbatim lane existed. The program itself is still a development stand-in at this rung — execution proof, not pixels.
# '''Forwarding the client's own pushes''' <code>[wip]</code>. The X DDX channel <code>0x1000014</code> executes its 3D replay (12 launches per cycle), the 2D path is green, and a guest pbuffer <code>glClear(red)+glFinish+glReadPixels</code> probe returns a real pixel (<code>pixel=ff0000ff</code>): the replay and readback machinery works. The direct-GL client's own channel (<code>0xbfef0002</code> this cycle) opens a C597 section but emits only 32/64-byte setup/notify kicks (<code>ops_runs=0</code>), so its draws never reach the host; a synchronous window draw also wedges X. That draw submission and the window/screen present are what remains (§6).
# '''Forwarding the client's own pushes''' <code>[wip]</code>. The X DDX channel <code>0x1000014</code> executes its 3D replay (12 launches per cycle), the 2D path is green, and a guest pbuffer <code>glClear(red)+glFinish+glReadPixels</code> probe returns a real pixel (<code>pixel=ff0000ff</code>): the replay and readback machinery works. The off-screen frame path then produced '''500/500 non-black 640x480 animated frames''' and a captured 45.4 s video. The direct-GL '''GR ring <code>0xbeef0100</code> now resolves''' (<code>4acac093</code>: beef-family always-probe, per-channel promote backoff, rank-collision fallback, parked-candidate eviction); its releases advance frame by frame and GZDoom reaches <code>V_Init: allocate screen</code> with its GL 4.6.0 context. The direct-GL '''CE ring <code>0xbeef0101</code> never resolves''', so the client's CE work does not execute and the guest wedges ~90 s after <code>V_Init</code>; window/screen present is still the frontier (§6).


== 4. The channel and object picture ==
== 4. The channel and object picture ==
Line 47: Line 47:
The acceptance signal is the '''engine-authored release''': a semaphore the engine writes when the work completes. For GL, completion flows through the channel's semaphores/notifications to the X server and the app's swap wait; for Vulkan, the client's fence value is released; for CUDA, the launch's release group wakes the client. The emulation relays engine completions into guest-observable memory (MSI-X plus a poller), and the guest resumes from there.
The acceptance signal is the '''engine-authored release''': a semaphore the engine writes when the work completes. For GL, completion flows through the channel's semaphores/notifications to the X server and the app's swap wait; for Vulkan, the client's fence value is released; for CUDA, the launch's release group wakes the client. The emulation relays engine completions into guest-observable memory (MSI-X plus a poller), and the guest resumes from there.


For the display path, completion is the notifier write and <code>UPDATE -&gt; FINISHED</code>; a completion event deferred by the reply path is force-retried until it posts, so the X server's mode set finishes. For the X DDX's own 2D push buffers, completion also requires the GPU-written engine get word to advance and the notifier to be recycled to <code>0xffff</code> after a kick; both are host-side levers now, and a diagnostic write of the get word released the DDX's reserve loop. For a windowed GL client the present is a kick; the GSP event-IRQ fix now sets the RISC-V SWGEN0 IRQSTAT/IRQMASK/IRQDEST and latches the CPU-tree leaf, so the guest services the event and glxgears completes swaps, and the GR backend no longer times out (the pair-close/pair-safe replay fixes are default ON; draws retire in 3-2400 ms). The DDX 3D replay now renders and reads back a pbuffer pixel (<code>pixel=ff0000ff</code>), and the RM write-lock holder (Xorg in the DDX vidheap scrub wait) is fixed. The next step is the direct-GL client's draw submission/capture and the window/screen present path.
For the display path, completion is the notifier write and <code>UPDATE -&gt; FINISHED</code>; a completion event deferred by the reply path is force-retried until it posts, so the X server's mode set finishes. For the X DDX's own 2D push buffers, completion also requires the GPU-written engine get word to advance and the notifier to be recycled to <code>0xffff</code> after a kick; both are host-side levers now, and a diagnostic write of the get word released the DDX's reserve loop. For a windowed GL client the present is a kick; the GSP event-IRQ fix now sets the RISC-V SWGEN0 IRQSTAT/IRQMASK/IRQDEST and latches the CPU-tree leaf, so the guest services the event and glxgears completes swaps, and the GR backend no longer times out (the pair-close/pair-safe replay fixes are default ON; draws retire in 3-2400 ms). The DDX 3D replay renders and reads back a pbuffer pixel (<code>pixel=ff0000ff</code>), the off-screen frame path produced non-black animated frames and a 45.4 s video, the RM write-lock holder (Xorg in the DDX vidheap scrub wait) is fixed (<code>df6e71bb</code>), and the direct-GL GR ring now resolves with advancing releases. The next step is the direct-GL CE ring <code>0xbeef0101</code> and the window/screen present path.


During bring-up the module also carries a '''diagnostics-only mirror''' for apps that spin on a shared progress word; it exists so a run can proceed while the natural completion path is being finished. The 60-fps milestone explicitly requires it switched off, along with any external poke.
During bring-up the module also carries a '''diagnostics-only mirror''' for apps that spin on a shared progress word; it exists so a run can proceed while the natural completion path is being finished. The 60-fps milestone explicitly requires it switched off, along with any external poke.
Line 53: Line 53:
== 6. Where the frontier is (ranked walls) ==
== 6. Where the frontier is (ranked walls) ==


# '''Direct-GL draw submission/capture''' <code>[wall]</code>. The GL client channel <code>0xbfef0002</code> opens a C597 section but emits only 32/64-byte setup/notify kicks (<code>ops_runs=0</code>; <code>section captured</code> and <code>GR LAUNCH</code> are 0). The DDX channel <code>0x1000014</code> works (12 launches/cycle), so the gap is the direct-GL client's submit path, not the replay machinery.
# '''Direct-GL CE ring resolution''' <code>[wall]</code>. The GR ring <code>0xbeef0100</code> resolves and its releases advance, but the CE ring <code>0xbeef0101</code> (<code>gpfifo=0x4100000</code>) never maps: <code>VAS-PROMOTE</code> walks a parked candidate and returns <code>rc=-2</code>, then <code>NOT mapped by any registered VAS</code> every ~7 s. The client's CE work never executes and GZDoom wedges ~90 s after <code>V_Init</code>. Next: walk all parked candidates regardless of owner VAS and/or keep at least two fresh same-VAS parks.
# '''Window/screen present and X stability''' <code>[wall]</code>. A synchronous window GL draw wedges X (<code>xwd -root</code> writes 0 bytes; <code>XGetImage</code> never completes), while the pbuffer probe works. Screen capture must use the ivshmem frame path, and the X must survive the draw.
# '''Window/screen present and the GLX lottery''' <code>[wall]</code>. The pbuffer and off-screen frame paths are proven (<code>pixel=ff0000ff</code>, 500/500 non-black frames, 45.4 s video), but glxgears' window is black and the GLX probe/frames path is a per-window lottery (hang/black/BadValue). A synchronous window draw wedges X; screen capture must use the ivshmem frame path and the present must be made to land.
# '''X startup / RM degradation''' <code>[wall]</code>. X startup is a lottery (<code>Failed to initialize DMA</code>, push-buffer <code>0x51</code>, occasional vmalloc 96 GB), and X can die after forward activity; a fresh cycle clears it.
# '''X startup / stability''' <code>[wall]</code>. X startup is a per-boot lottery: on 3/6 boots window creation wedges Xorg in an unkillable kernel RM spin (<code>_nv052600rm+0x7b</code>); <code>Failed to initialize DMA</code>, push-buffer <code>0x51</code>, <code>Wait for channel idle timed out</code> and occasional vmalloc failures also appear. A fresh cycle clears it. The shared nvClip <code>modeNum</code> must be even for the GL client's state init to succeed; the odd value is the guest kernel's nvKms deferred close, not module-induced.
# '''The X DDX footprint''' <code>[wall]</code>. The DDX allocates 4 GB of system pages at startup; an 8 GB guest has no headroom once GZDoom's own footprint lands and both are OOM-killed. Current runs use a 12 GB guest; the allocation is a suspected emulation gap (a 512 MiB emulated VRAM falling back to a 4 GB sysmem allocation) to fix.
# '''The X DDX footprint''' <code>[wall]</code>. The DDX allocates 4 GB of system pages at startup; an 8 GB guest has no headroom once GZDoom's own footprint lands and both are OOM-killed. Current runs use a 12 GB guest; the allocation is a suspected emulation gap (a 512 MiB emulated VRAM falling back to a 4 GB sysmem allocation) to fix.
# '''Non-black screen frames and sustained rate''' <code>[wall]</code>/<code>[target]</code>. Once the direct-GL draw capture and the window/screen present land, the capture must show motion, then 60 fps in one VM with no forcing, then two VMs on two mediated devices with non-black frames in both.
# '''Non-black window frames and sustained rate''' <code>[wall]</code>/<code>[target]</code>. Once the CE ring, the window/screen present and X stability land, the capture must show motion in a window, then 60 fps in one VM with no forcing, then two VMs on two mediated devices with non-black frames in both.


== 7. What is proven today ==
== 7. What is proven today ==
Line 97: Line 97:
|-
|-
| First real GPU readback: pbuffer <code>glClear(red)+glReadPixels</code> -&gt; <code>pixel=ff0000ff</code>
| First real GPU readback: pbuffer <code>glClear(red)+glReadPixels</code> -&gt; <code>pixel=ff0000ff</code>
| <code>[live]</code>
|-
| First non-black GPU-rendered frames: 500/500 off-screen frames; 45.4 s video captured (<code>r1769</code>)
| <code>[live]</code>
|-
| Direct-GL GR ring <code>0xbeef0100</code> resolves; releases advance (<code>4acac093</code>)
| <code>[live]</code>
|-
| GZDoom passes <code>V_Init: allocate screen</code> with GL 4.6.0 when shared nvClip <code>modeNum</code> is even
| <code>[live]</code>
| <code>[live]</code>
|-
|-
Line 114: Line 123:
| <code>[live]</code>
| <code>[live]</code>
|-
|-
| Direct-GL channel <code>0xbfef0002</code> submits a captured draw (<code>ops_runs&gt;0</code>)
| Direct-GL CE ring <code>0xbeef0101</code> resolves and the client's CE work executes
| <code>[wall]</code>
|-
| Window/screen present completes without wedging X (ivshmem frame path)
| <code>[wall]</code>
| <code>[wall]</code>
|-
|-
| X startup/RM stability (<code>0x51</code>, vmalloc 96 GB)
| Window/screen present produces non-black window pixels
| <code>[wall]</code>
| <code>[wall]</code>
|-
|-
| NVIDIA Vulkan ICD creates a device
| X startup/stability (<code>_nv052600rm</code> soft-lock, <code>0x51</code>, vmalloc)
| <code>[wall]</code>
| <code>[wall]</code>
|-
|-
Line 135: Line 141:
== 8. What is not claimed ==
== 8. What is not claimed ==


* No non-black pixel has been produced by the emulated vGPU's 3D pipeline yet.
* No '''window/screen''' pixel has been produced by the emulated vGPU's 3D pipeline yet; every screen capture so far is black. The proven non-black frames and the 45.4 s video come from the off-screen pbuffer path (pbuffer -&gt; sysmem -&gt; kvmfr ivshmem -&gt; host reader), not from a windowed X screen capture.
* Display/KMS bring-up (NvKms, <code>/dev/dri</code>, X, a real GL context) is not a rendered-frame claim. The first readback pixel is a pbuffer probe, not a window/screen frame; the direct-GL client still emits only setup/notify kicks, and a window draw wedges X. Those are the current walls.
* Display/KMS bring-up (NvKms, <code>/dev/dri</code>, X, a real GL context) is not a rendered-frame claim. GZDoom reaching <code>V_Init: allocate screen</code> proves the client's GL init and the direct-GL GR lane, not the game loop or a presented frame; the direct-GL CE ring still does not resolve.
* No frame-rate or latency numbers: the scheduled measurement is defined (60 fps over 60 s, no forcing) but not yet met.
* No frame-rate or latency numbers: the scheduled measurement is defined (60 fps over 60 s, no forcing) but not yet met.
* The <code>virgl</code>-based demo videos are a different, labelled path — not this one.
* The <code>virgl</code>-based demo videos are a different, labelled path — not this one.

Revision as of 15:48, 11 November 2026

An absence of critical technical documentation has historically slowed growth and adoption of developer ecosystems for GPU virtualization.

This CC-BY-4.0 licensed content can either be used with attribution, or used as inspiration for new documentation, created by GPU vendors for public commercial distribution as developer documentation.

Where possible, this documentation will clearly label dates and versions of observed-but-not-guaranteed behaviour vs. vendor-documented stable interfaces/behaviour with guarantees of forward or backward compatibility.

Graphics Execution

Scope. How the emulated vGPU runs 3D graphics work — the OpenGL and Vulkan command streams a stock guest submits — on the physical GPU, and what is proven about it. This is the shared machinery behind OpenGL.md, Vulkan.md and the Windows/DirectX target (DirectX.md); CUDA compute uses the same submission and completion path for a different engine workload (Execution-Frontier.md). Canonical sources: ../docs/04-implementation/05-graphics-execution-plan.md (the plan and milestone ladder) and ../docs/04-implementation/04-real-render-walls.md (the measured walls); the docs tree wins on disagreement.

Status: 2026-11-11, tree master @ 4acac093. Tags: [live] observed end to end; [wip] partially built; [wall] current blocker; [target] defined success state; [planned] designed, not built.

1. What "real pixels" means

The acceptance target is strict: a stock guest GL or Vulkan application's draw commands are executed by the physical GPU's fixed-function 3D pipeline (vertex fetch, primitive assembly, rasterisation, fragment work, ROP), the render target is readable by the guest, and that image reaches the capture path through the emulated mdev. The pixels must not come from another GPU, a software rasteriser, or a host display round-trip.

(The project has also published two deliberately-labelled two-VM demo videos that use the standard virtio-gpu/virgl virtual GPU on a host integrated GPU, and a CUDA-animation video that does run on the emulated vGPU through the compute path. The virgl videos are honest demonstrations of the guest experience; they are not this execution path, and their captions say so.)

2. The path a graphics command takes

The graphics execution paths: guest OpenGL/Vulkan/CUDA commands to the physical engines, and completions back.
guest apps (GL / Vulkan / CUDA)
  -> guest user space (GLX client + GL core, Vulkan ICD, libcuda)
  -> guest X server (GLX module + NVIDIA DDX)           [OpenGL]
  -> nvidia.ko: channels, GPFIFO rings, doorbells       [all APIs]
  -> vgsp.ko: capture sections, decode pushes, forward
  -> physical GPU front-end -> GR engine (3D + compute) / copy engines
  <- engine-authored releases written into guest memory
  <- completion relay (MSI-X + wakes) back to the guest
ASCII fallback above; the diagram shows the three execution lanes.

3. The three execution lanes

Graphics work reaches the engines today in one of three shapes, in increasing fidelity:

  1. Verbatim 3D method-stream replay [live]. The emulation captures the X server's graphics-class (0xC597, TURING_A) section from its channel, relocates the operand addresses into host-built shadow pages the engine can reach, and submits the stream unchanged. The engine retires it: the uploads (LINE_LENGTH_IN/LAUNCH_DMA), the state, program and texture binds, and the release all execute with no MMU fault. This proves the mechanics of 3D execution — object bind, window placement, operand relocation, completion — on real hardware.
  2. Synthesized launch from a captured section [live]. A compute-shaped launch built from the captured section (operand list, program, a 64-word launch descriptor) reaches the engine and retires real sm_75 SASS; the Turing shader-program-header form (V3) is accepted with zero header faults across repeated launches. This proved engine-side dispatch before the verbatim lane existed. The program itself is still a development stand-in at this rung — execution proof, not pixels.
  3. Forwarding the client's own pushes [wip]. The X DDX channel 0x1000014 executes its 3D replay (12 launches per cycle), the 2D path is green, and a guest pbuffer glClear(red)+glFinish+glReadPixels probe returns a real pixel (pixel=ff0000ff): the replay and readback machinery works. The off-screen frame path then produced 500/500 non-black 640x480 animated frames and a captured 45.4 s video. The direct-GL GR ring 0xbeef0100 now resolves (4acac093: beef-family always-probe, per-channel promote backoff, rank-collision fallback, parked-candidate eviction); its releases advance frame by frame and GZDoom reaches V_Init: allocate screen with its GL 4.6.0 context. The direct-GL CE ring 0xbeef0101 never resolves, so the client's CE work does not execute and the guest wedges ~90 s after V_Init; window/screen present is still the frontier (§6).

4. The channel and object picture

  • Every submission lives on a channel: a GPFIFO ring plus engine objects, with a USERD the driver advances its put pointer in and a doorbell the host module traps. Channels are typed by engine: a graphics channel carries 3D state and draws; copy-class channels carry memory movements.
  • The graphics object class the guest's 3D stream binds is the Turing graphics class, 0xC597. Streams now come from both the X server's channel and the direct GL client's own channels (0xbeef0100 GR plus the 0xbeef0101/0xbeef0102 CE siblings); the 2D present/readback work rides lighter 2D-class and copy-engine shapes.
  • Display and modeset [live]. The guest's modeset path is served by an emulated display engine: the display push-buffer channels (core 0xc57d, window 0xc57e, immediate/window 0xc57b) are registered, their push buffers parsed, and the notifier/UPDATE completion authored. The arm is class-aware (the window class has no NOTIFY bit; a bound notifier is the arm), and the X DDX's 0x8000 -> 0 notification completion is authored. This is what lets nvidia-modeset/NvKms register (/dev/dri), X settle and GLX clients obtain a real GL context. The display-capability seam is parameterised in the campaign configuration.
  • Cells of state: 3D streams are long—hundreds of method words per draw — and mix state binds (viewport, render target, ZS, blend/ROP), program and constant binds, vertex/index streams and draw-issuing methods. The capture machinery recognises the init-shaped operands today; the per-draw state vocabulary is being extended ([G-B in the plan]( ../docs/04-implementation/05-graphics-execution-plan.md)).
  • Synchronisation: tracking semaphores, notifications and fence values written by the engine; the guest's swap/present waits sit on top of those.

5. Completion, fences and presents

The acceptance signal is the engine-authored release: a semaphore the engine writes when the work completes. For GL, completion flows through the channel's semaphores/notifications to the X server and the app's swap wait; for Vulkan, the client's fence value is released; for CUDA, the launch's release group wakes the client. The emulation relays engine completions into guest-observable memory (MSI-X plus a poller), and the guest resumes from there.

For the display path, completion is the notifier write and UPDATE -> FINISHED; a completion event deferred by the reply path is force-retried until it posts, so the X server's mode set finishes. For the X DDX's own 2D push buffers, completion also requires the GPU-written engine get word to advance and the notifier to be recycled to 0xffff after a kick; both are host-side levers now, and a diagnostic write of the get word released the DDX's reserve loop. For a windowed GL client the present is a kick; the GSP event-IRQ fix now sets the RISC-V SWGEN0 IRQSTAT/IRQMASK/IRQDEST and latches the CPU-tree leaf, so the guest services the event and glxgears completes swaps, and the GR backend no longer times out (the pair-close/pair-safe replay fixes are default ON; draws retire in 3-2400 ms). The DDX 3D replay renders and reads back a pbuffer pixel (pixel=ff0000ff), the off-screen frame path produced non-black animated frames and a 45.4 s video, the RM write-lock holder (Xorg in the DDX vidheap scrub wait) is fixed (df6e71bb), and the direct-GL GR ring now resolves with advancing releases. The next step is the direct-GL CE ring 0xbeef0101 and the window/screen present path.

During bring-up the module also carries a diagnostics-only mirror for apps that spin on a shared progress word; it exists so a run can proceed while the natural completion path is being finished. The 60-fps milestone explicitly requires it switched off, along with any external poke.

6. Where the frontier is (ranked walls)

  1. Direct-GL CE ring resolution [wall]. The GR ring 0xbeef0100 resolves and its releases advance, but the CE ring 0xbeef0101 (gpfifo=0x4100000) never maps: VAS-PROMOTE walks a parked candidate and returns rc=-2, then NOT mapped by any registered VAS every ~7 s. The client's CE work never executes and GZDoom wedges ~90 s after V_Init. Next: walk all parked candidates regardless of owner VAS and/or keep at least two fresh same-VAS parks.
  2. Window/screen present and the GLX lottery [wall]. The pbuffer and off-screen frame paths are proven (pixel=ff0000ff, 500/500 non-black frames, 45.4 s video), but glxgears' window is black and the GLX probe/frames path is a per-window lottery (hang/black/BadValue). A synchronous window draw wedges X; screen capture must use the ivshmem frame path and the present must be made to land.
  3. X startup / stability [wall]. X startup is a per-boot lottery: on 3/6 boots window creation wedges Xorg in an unkillable kernel RM spin (_nv052600rm+0x7b); Failed to initialize DMA, push-buffer 0x51, Wait for channel idle timed out and occasional vmalloc failures also appear. A fresh cycle clears it. The shared nvClip modeNum must be even for the GL client's state init to succeed; the odd value is the guest kernel's nvKms deferred close, not module-induced.
  4. The X DDX footprint [wall]. The DDX allocates 4 GB of system pages at startup; an 8 GB guest has no headroom once GZDoom's own footprint lands and both are OOM-killed. Current runs use a 12 GB guest; the allocation is a suspected emulation gap (a 512 MiB emulated VRAM falling back to a 4 GB sysmem allocation) to fix.
  5. Non-black window frames and sustained rate [wall]/[target]. Once the CE ring, the window/screen present and X stability land, the capture must show motion in a window, then 60 fps in one VM with no forcing, then two VMs on two mediated devices with non-black frames in both.

7. What is proven today

Capability State
X server initialises on the emulated vGPU (driver DMA sanity push, ScreenInit) [live]
Display engine serves the modeset push buffers; notifier plus UPDATE -> FINISHED [live] (campaign configuration)
nvidia-modeset/NvKms registers (/dev/dri/card0, renderD128, fb0); nvidia-smi -L enumerates [live]
glxinfo reports the emulated GPU with the NVIDIA renderer (OpenGL 4.6.0, direct rendering) [live]
X green on attempt 1 in three consecutive cycles (ebf3130f) [live]
GSP event-IRQ fix wakes the guest poll; glxgears completes swaps [live]
2D path end-to-end: accelerated X fills read back via xwd [live]
Guest's real GL draws forward and execute on the physical GPU (work_gr backend_ok=5-7) [live]
GR backend timeouts fixed (backend_timeout=0; draws retire in 3-2400 ms) [live]
DDX 3D replay executes (section captured + GR LAUNCH on 0x1000014, 12 launches/cycle) [live]
First real GPU readback: pbuffer glClear(red)+glReadPixels -> pixel=ff0000ff [live]
First non-black GPU-rendered frames: 500/500 off-screen frames; 45.4 s video captured (r1769) [live]
Direct-GL GR ring 0xbeef0100 resolves; releases advance (4acac093) [live]
GZDoom passes V_Init: allocate screen with GL 4.6.0 when shared nvClip modeNum is even [live]
RM write-lock holder identified (Xorg in the DDX vidheap scrub wait) and fixed (df6e71bb) [live]
GL section capture: the X channel's 0xC597 stream decoded [live]
Verbatim 3D method-stream replay retired by the engine (no MMU fault) [live]
Synthesized launch executes real sm_75 SASS; Turing V3 program header accepted [live]
X server survives the run on a 12 GB guest (the DDX's 4 GB sysmem allocation forces the size) [live]
Direct-GL CE ring 0xbeef0101 resolves and the client's CE work executes [wall]
Window/screen present produces non-black window pixels [wall]
X startup/stability (_nv052600rm soft-lock, 0x51, vmalloc) [wall]
Non-black screen pixels from the emulated vGPU's 3D pipeline [wall]
Windows/DirectX guests [planned]

8. What is not claimed

  • No window/screen pixel has been produced by the emulated vGPU's 3D pipeline yet; every screen capture so far is black. The proven non-black frames and the 45.4 s video come from the off-screen pbuffer path (pbuffer -> sysmem -> kvmfr ivshmem -> host reader), not from a windowed X screen capture.
  • Display/KMS bring-up (NvKms, /dev/dri, X, a real GL context) is not a rendered-frame claim. GZDoom reaching V_Init: allocate screen proves the client's GL init and the direct-GL GR lane, not the game loop or a presented frame; the direct-GL CE ring still does not resolve.
  • No frame-rate or latency numbers: the scheduled measurement is defined (60 fps over 60 s, no forcing) but not yet met.
  • The virgl-based demo videos are a different, labelled path — not this one.

9. Related pages