Difference between revisions of "OpenGL"

From Open-IOV
Jump to navigation Jump to search
m (2026-11-13 status refresh: X/DDX stability walls closed, GR forward lane live, push-body PA-bound fix; direct-GL payload VAS registration is the wall)
m (2026-11-14 status refresh: direct-GL submission decoded (macro-driven), push rescues land, draw-state push reaches the engine; MME multi-part completion is the wall; X start lottery is the limiter)
Line 5: Line 5:
<blockquote>'''Scope.''' How a stock guest OpenGL application's commands travel through the unmodified NVIDIA stack on the emulated vGPU, and the honest state of running OpenGL on it. The host-side execution machinery shared by all graphics APIs is on [[Graphics Execution|<code>Graphics-Execution.md</code>]]; CUDA compute is on [[The Execution Frontier|<code>Execution-Frontier.md</code>]]. Canonical sources: [https://git.vgpu.io/open-iov.git/tree/docs/04-implementation/05-graphics-execution-plan.md <code>../docs/04-implementation/05-graphics-execution-plan.md</code>] (the plan and milestones), [https://git.vgpu.io/open-iov.git/tree/docs/04-implementation/04-real-render-walls.md <code>../docs/04-implementation/04-real-render-walls.md</code>] (the measured walls) and [https://git.vgpu.io/open-iov.git/tree/docs/03-architecture/02-guest-stack-and-cudasw.md <code>../docs/03-architecture/02-guest-stack-and-cudasw.md</code>] (the guest stack). The docs tree wins on disagreement.</blockquote>
<blockquote>'''Scope.''' How a stock guest OpenGL application's commands travel through the unmodified NVIDIA stack on the emulated vGPU, and the honest state of running OpenGL on it. The host-side execution machinery shared by all graphics APIs is on [[Graphics Execution|<code>Graphics-Execution.md</code>]]; CUDA compute is on [[The Execution Frontier|<code>Execution-Frontier.md</code>]]. Canonical sources: [https://git.vgpu.io/open-iov.git/tree/docs/04-implementation/05-graphics-execution-plan.md <code>../docs/04-implementation/05-graphics-execution-plan.md</code>] (the plan and milestones), [https://git.vgpu.io/open-iov.git/tree/docs/04-implementation/04-real-render-walls.md <code>../docs/04-implementation/04-real-render-walls.md</code>] (the measured walls) and [https://git.vgpu.io/open-iov.git/tree/docs/03-architecture/02-guest-stack-and-cudasw.md <code>../docs/03-architecture/02-guest-stack-and-cudasw.md</code>] (the guest stack). The docs tree wins on disagreement.</blockquote>


Status: 2026-11-13, tree <code>master @ 436cbf9d</code>. Tags: <code>[live]</code> observed with the stock guest driver; <code>[wip]</code> partial; <code>[wall]</code> current blocker; <code>[target]</code> defined success state.
Status: 2026-11-14, tree <code>master @ 339da842</code>. Tags: <code>[live]</code> observed with the stock guest driver; <code>[wip]</code> partial; <code>[wall]</code> current blocker; <code>[target]</code> defined success state.


== 1. The guest OpenGL stack ==
== 1. The guest OpenGL stack ==
Line 95: Line 95:
| Client push bodies decode as real GR method streams above 10 GiB via the VAS walk (live-PA bound raised to the registered guest-RAM top, <code>436cbf9d</code>)
| Client push bodies decode as real GR method streams above 10 GiB via the VAS walk (live-PA bound raised to the registered guest-RAM top, <code>436cbf9d</code>)
| <code>[live]</code>
| <code>[live]</code>
|-
| Direct-GL submission fully decoded: nine NV906F entries tile one contiguous buffer; no method-grammar draw header; per-frame work is data-block blobs + real state pushes (324/476/728 B) invoking MME macros; QMD-carrier hypothesis refuted
| <code>[live]</code>
|-
| Recycled direct-GL pushes recovered (content-gated FB-shadow rescue <code>e1eac368</code>, LOW-band rescue <code>b7f09616</code>); the 324-B draw-state push opens a <code>C597</code> section, captures and is marked for the verbatim 3D lane
| <code>[live]</code>
|-
| Direct-GL state push reaches the engine with the golden MME overlay (<code>GR LAUNCH submitted</code> + OK; MME-PRELUDE pack 1701 IR words; MME-GOLDEN slot 0 accepted)
| <code>[live]</code>
|-
| <code>CODESRC</code> fault-park class prevented (program page relocated to the engine-mapped window-0 shadow, <code>17c7ed7c</code>; validated boot 31 relocations, 51 GR launches, 79 GP_GET advances, 0 <code>FAULT_PDE</code>, 0 <code>WEDGE-REARM</code>, <code>backend_ok=35</code>)
| <code>[live]</code>
|-
| RT routing tested and refuted as the store cause: RT0 bind verified 16x with correct target/geometry, still 61/61 <code>wrote=0</code>
| <code>[live]</code> (negative result)
|-
|-
| GZDoom passes <code>V_Init: allocate screen</code> with GL 4.6.0 when shared nvClip <code>modeNum</code> is even
| GZDoom passes <code>V_Init: allocate screen</code> with GL 4.6.0 when shared nvClip <code>modeNum</code> is even
Line 111: Line 126:
| <code>[live]</code>
| <code>[live]</code>
|-
|-
| Direct-GL command-stream delivery (gl setup push at VA <code>0x4001004</code> and draws at <code>0x4250000</code>/<code>0x4250110</code>; the CE ring <code>0xbeef0101</code>)
| Direct-GL MME macro completion (multi-part keep-window: 2799-word packed program + 156-400-word draw window cannot fit the &lt;=2048-word block; <code>live_gen</code> never promotes)
| <code>[wall]</code> — the payloads' user/UVM VAS is never registered; they resolve to zero/implausible PAs and no section opens for <code>0xbeef0100</code>
| <code>[wall]</code> — mechanism committed <code>b44694b5</code> (default off) and enabled in params (<code>339da842</code>); A/B pending
|-
|-
| Window/screen present produces non-black screen pixels
| Window/screen present produces non-black screen pixels
| <code>[wall]</code>
| <code>[wall]</code>
|-
|-
| X screen-init first-generation lottery (<code>modeNum</code> parity; only an immediately-post-boot X passes DDX DMA init)
| X screen-init first-generation lottery (<code>modeNum</code> parity; only an immediately-post-boot X passes DDX DMA init; dominant throughput limiter)
| <code>[wip]</code>
| <code>[wip]</code>
|-
|-
Line 130: Line 145:
== 5. Milestones on this path ==
== 5. Milestones on this path ==


The graphics plan defines M-GL1 (client context-init decodes and executes; the client leaves its retry loop), M-GL2 (<code>glxgears</code> non-black), M-GL3 (GZDoom GL non-black through the mdev), M-GL4 (60 fps, no forcing), M-GL5 (two VMs, two mdevs), M-GL6 (split-screen video). The project sits '''between M-GL1 and M-GL2, with M-GL2's off-screen half proven''': the display/KMS stack registers, X and the stock GLX path are fully green (<code>glxinfo -B</code> rc 0, ~520 FPS <code>glxgears</code>, Xorg WAIT 0), the DDX/X-stability walls were closed (X green for more than 10 minutes), the direct-GL ring/USERD machinery is hardened, and the GPU-rendered pbuffer path produced non-black animated frames and captured videos (45.4 s and 50 s). Captured GL sections now execute on the engine and the client's push bodies decode as real GR method streams above 10 GiB. The remaining gates are '''delivery of the direct-GL client's own command stream''' (the setup push at <code>0x4001004</code> and the draws at <code>0x4250000</code>/<code>0x4250110</code> still resolve to zero/implausible PAs because their user/UVM VAS is never registered), the window/screen present path (glxgears' window is black and a window draw can wedge X), and the X first-generation screen-init lottery — then the first rendered screen frame.
The graphics plan defines M-GL1 (client context-init decodes and executes; the client leaves its retry loop), M-GL2 (<code>glxgears</code> non-black), M-GL3 (GZDoom GL non-black through the mdev), M-GL4 (60 fps, no forcing), M-GL5 (two VMs, two mdevs), M-GL6 (split-screen video). The project sits '''between M-GL1 and M-GL2, with M-GL2's off-screen half proven''': the display/KMS stack registers, X and the stock GLX path are fully green (<code>glxinfo -B</code> rc 0, ~520 FPS <code>glxgears</code>, Xorg WAIT 0), the DDX/X-stability walls were closed (X green for more than 10 minutes), the direct-GL ring/USERD machinery is hardened, and the GPU-rendered pbuffer path produced non-black animated frames and captured videos (45.4 s and 50 s). Captured GL sections now execute on the engine, and the direct-GL client's submission is fully decoded: it carries no classic draw header — the work is macro-driven (data-block blobs plus small state pushes invoking MME macros), the recycled pushes are recovered, and the client's draw-state push opens, captures and reaches the engine. The remaining gates are '''MME macro completion''' (the 2799-word packed program cannot complete inside its &lt;=2048-word block with the 156-400-word draw window, so <code>live_gen</code> never promotes; the multi-part keep-window mechanism is committed default-off and enabled for the pending A/B), the window/screen present path (glxgears' window is black and a window draw can wedge X), and the X first-generation screen-init lottery — then the first rendered screen frame.


== 6. What is not claimed ==
== 6. What is not claimed ==


* No '''window/screen''' frame has rendered real pixels through the emulated vGPU yet; every screen capture so far is black. The non-black frames and the videos (45.4 s and 50 s) come from the off-screen pbuffer path (pbuffer -&gt; sysmem -&gt; kvmfr ivshmem -&gt; host reader), not from a windowed X screen capture. The direct-GL payload VAS is not registered, its draws do not reach the engine, a synchronous window draw wedges X, and X screen-init retains a per-boot first-generation lottery. Those are the current walls.
* No '''window/screen''' frame has rendered real pixels through the emulated vGPU yet; every screen capture so far is black. The non-black frames and the videos (45.4 s and 50 s) come from the off-screen pbuffer path (pbuffer -&gt; sysmem -&gt; kvmfr ivshmem -&gt; host reader), not from a windowed X screen capture. The client's draw-state push now reaches the engine, but the MME macro program does not complete so the draw never stores, a synchronous window draw wedges X, and X start retains a per-boot first-generation lottery. Those are the current walls.
* GZDoom reaching <code>V_Init: allocate screen</code> is not a rendered-frame claim; it proves the client's GL init and the direct-GL GR lane, not the game loop or a presented frame.
* GZDoom reaching <code>V_Init: allocate screen</code> is not a rendered-frame claim; it proves the client's GL init and the direct-GL GR lane, not the game loop or a presented frame.
* No frame-rate or latency numbers.
* No frame-rate or latency numbers.

Revision as of 15:54, 14 November 2026

An absence of critical technical documentation has historically slowed growth and adoption of developer ecosystems for GPU virtualization.

This CC-BY-4.0 licensed content can either be used with attribution, or used as inspiration for new documentation, created by GPU vendors for public commercial distribution as developer documentation.

Where possible, this documentation will clearly label dates and versions of observed-but-not-guaranteed behaviour vs. vendor-documented stable interfaces/behaviour with guarantees of forward or backward compatibility.

OpenGL

Scope. How a stock guest OpenGL application's commands travel through the unmodified NVIDIA stack on the emulated vGPU, and the honest state of running OpenGL on it. The host-side execution machinery shared by all graphics APIs is on Graphics-Execution.md; CUDA compute is on Execution-Frontier.md. Canonical sources: ../docs/04-implementation/05-graphics-execution-plan.md (the plan and milestones), ../docs/04-implementation/04-real-render-walls.md (the measured walls) and ../docs/03-architecture/02-guest-stack-and-cudasw.md (the guest stack). The docs tree wins on disagreement.

Status: 2026-11-14, tree master @ 339da842. Tags: [live] observed with the stock guest driver; [wip] partial; [wall] current blocker; [target] defined success state.

1. The guest OpenGL stack

On Linux, an OpenGL application does not talk to the kernel driver directly: it talks to the GL implementation, and — when a display is involved — through the X server's GLX extension:

application (toolkit / SDL / GLFW / raw GLX)
  -> libGL (dispatch)                    -> libGLX_nvidia (NVIDIA GLX client)
  -> libnvidia-glcore (the vendor GL engine: state, programs, shaders, draws)
  -> ioctls + mappings to nvidia.ko      -> channels, buffers, synchronization
X server (Xorg)
  -> GLX extension module (glxserver_nvidia) and the NVIDIA DDX (X driver)
  -> owns the screen, drawables and the present path

Two forms of operation exist, and both are visible in the guest:

  • Direct rendering — the normal desktop case. The GL client builds GPU commands in its own process and the X server coordinates buffers and presents. The client's channel is the source of truth for per-draw state.
  • Indirect GLX — the client's GL protocol is serialised to the X server, which executes it against its own GL context. Here the X server's channel carries the graphics stream.

On the emulated vGPU the readable command sources today are the direct GL client's own channels — the GR channel 0xbeef0100 and its CE siblings 0xbeef0101/0xbeef0102 — and the X server's channels, whose graphics-class (0xC597, TURING_A) sections the emulation captures and whose DDX kick channel (0xbfef0002) is decoded from its ring head (Graphics-Execution.md §3).

2. What an NVIDIA GL frame involves (observed shapes)

  • Context and state setup — shader programs (SASS) uploaded to device memory, constant/descriptor banks, render targets and depth surfaces allocated and mapped.
  • Per-draw work — a graphics-class method stream: state binds (viewport, render target, ZS, blend/ROP), program and constant binds, vertex/index data, and draw-issuing methods that drive the fixed-function pipeline (vertex fetch, primitive assembly, rasterisation, fragment work, ROP).
  • Program/descriptor delivery — the driver uploads shader code and descriptors through its copy/utility-channel inline-data path. The GL draw path decodes these uploads and the engine executes the draw; Vulkan's inline-to-memory arm is the remaining mirror-image gap (Vulkan.md).
  • Present — the X server's present/readback path runs as light 2D-class and copy-engine operations; the emulator decodes and executes those shapes.

3. Status on the emulated vGPU

Rated honestly, against the milestone ladder in the plan:

Item State
X server initialises on the emulated vGPU (driver push-DMA sanity test, ScreenInit) [live]
nvidia-modeset/NvKms registers (/dev/dri/card0, renderD128, fb0); nvidia-smi -L enumerates [live] (campaign configuration)
glxinfo -B reports the NVIDIA renderer (OpenGL 4.6.0, driver 570.144, direct rendering) [live]
The stock GLX path is fully green: glxinfo -B returns 0, glxgears renders ~520 FPS, Xorg WAIT count 0 [live]
The X driver's RC-watchdog channel registers and its notifier is authored [live]
X green on attempt 1 in three consecutive cycles (ebf3130f) [live]
GSP event-IRQ fix wakes the guest poll; glxgears completes swaps [live]
2D path end-to-end: accelerated X fills read back via xwd [live]
Guest's real GL draws forward and execute on the physical GPU (work_gr backend_ok=5-7) [live]
GR backend timeouts fixed (backend_timeout=0; draws retire in 3-2400 ms) [live]
DDX 3D replay executes (section captured + GR LAUNCH on 0x1000014, 12 launches/cycle) [live]
First real GPU readback: pbuffer glClear(red)+glReadPixels -> pixel=ff0000ff [live]
First non-black GPU-rendered frames: 500/500 off-screen 640x480 frames per run; 45.4 s video (r1769) and a 50 s capture (2026-11-12) [live]
Direct-GL GR ring 0xbeef0100 resolves; releases advance; ring/VAS anchor hardened (verified page-directory discovery, eager push pins, per-rank scans) [live]
Client-allocated USERD resolved from the channel's memory objects (hUserdMemory/userdOffset); foreign-USERD adoption skipped; aperture-aware wire fallback [live]
X DDX idle-DMA/ScreenInit, USERD-latch lifetime, re-probe thrash, Xorg kernel-mode spin (GET-past-PUT) and flip-event walls closed; X green >10 minutes (13a0c5d6, acb51d53, 6ebc7cd4, 770d6b4a, b9213c57, 29690635) [live]
Captured GL sections execute on the engine (GR LAUNCH submitted / GUEST GR LAUNCH OK); data window populated (n_data 0 -> 1024) [live]
Client push bodies decode as real GR method streams above 10 GiB via the VAS walk (live-PA bound raised to the registered guest-RAM top, 436cbf9d) [live]
Direct-GL submission fully decoded: nine NV906F entries tile one contiguous buffer; no method-grammar draw header; per-frame work is data-block blobs + real state pushes (324/476/728 B) invoking MME macros; QMD-carrier hypothesis refuted [live]
Recycled direct-GL pushes recovered (content-gated FB-shadow rescue e1eac368, LOW-band rescue b7f09616); the 324-B draw-state push opens a C597 section, captures and is marked for the verbatim 3D lane [live]
Direct-GL state push reaches the engine with the golden MME overlay (GR LAUNCH submitted + OK; MME-PRELUDE pack 1701 IR words; MME-GOLDEN slot 0 accepted) [live]
CODESRC fault-park class prevented (program page relocated to the engine-mapped window-0 shadow, 17c7ed7c; validated boot 31 relocations, 51 GR launches, 79 GP_GET advances, 0 FAULT_PDE, 0 WEDGE-REARM, backend_ok=35) [live]
RT routing tested and refuted as the store cause: RT0 bind verified 16x with correct target/geometry, still 61/61 wrote=0 [live] (negative result)
GZDoom passes V_Init: allocate screen with GL 4.6.0 when shared nvClip modeNum is even [live]
RM write-lock holder identified (Xorg in the DDX vidheap scrub wait) and fixed (df6e71bb) [live]
Engine executes a launch built from the captured section (real sm_75 SASS retired) [live]
Verbatim 3D method-stream replay retired by the engine, no MMU fault [live]
Emulated FB/GART sizing traps fixed (FB info replies in KB; GART default 256 MiB), removing the 4 GiB guest-allocation class [live]
Direct-GL MME macro completion (multi-part keep-window: 2799-word packed program + 156-400-word draw window cannot fit the <=2048-word block; live_gen never promotes) [wall] — mechanism committed b44694b5 (default off) and enabled in params (339da842); A/B pending
Window/screen present produces non-black screen pixels [wall]
X screen-init first-generation lottery (modeNum parity; only an immediately-post-boot X passes DDX DMA init; dominant throughput limiter) [wip]
60 fps sustained, no forcing (M-GL4) [target]

4. How OpenGL runs are verified

Instruments that matter, in order of usefulness: the X server log and xdpyinfo (did the server initialise?), glxinfo -B (which renderer is real), the application's own progress output, and frame capture at the GL source (a swap-hook capture is the ground truth for pixels — the X screen itself is not, on some stacks). Captures are decoded offline and scored: fraction of pixels above a luminance threshold, colour count, and frame-to-frame motion. A run counts only when the capture shows motion with the NVIDIA renderer in use.

5. Milestones on this path

The graphics plan defines M-GL1 (client context-init decodes and executes; the client leaves its retry loop), M-GL2 (glxgears non-black), M-GL3 (GZDoom GL non-black through the mdev), M-GL4 (60 fps, no forcing), M-GL5 (two VMs, two mdevs), M-GL6 (split-screen video). The project sits between M-GL1 and M-GL2, with M-GL2's off-screen half proven: the display/KMS stack registers, X and the stock GLX path are fully green (glxinfo -B rc 0, ~520 FPS glxgears, Xorg WAIT 0), the DDX/X-stability walls were closed (X green for more than 10 minutes), the direct-GL ring/USERD machinery is hardened, and the GPU-rendered pbuffer path produced non-black animated frames and captured videos (45.4 s and 50 s). Captured GL sections now execute on the engine, and the direct-GL client's submission is fully decoded: it carries no classic draw header — the work is macro-driven (data-block blobs plus small state pushes invoking MME macros), the recycled pushes are recovered, and the client's draw-state push opens, captures and reaches the engine. The remaining gates are MME macro completion (the 2799-word packed program cannot complete inside its <=2048-word block with the 156-400-word draw window, so live_gen never promotes; the multi-part keep-window mechanism is committed default-off and enabled for the pending A/B), the window/screen present path (glxgears' window is black and a window draw can wedge X), and the X first-generation screen-init lottery — then the first rendered screen frame.

6. What is not claimed

  • No window/screen frame has rendered real pixels through the emulated vGPU yet; every screen capture so far is black. The non-black frames and the videos (45.4 s and 50 s) come from the off-screen pbuffer path (pbuffer -> sysmem -> kvmfr ivshmem -> host reader), not from a windowed X screen capture. The client's draw-state push now reaches the engine, but the MME macro program does not complete so the draw never stores, a synchronous window draw wedges X, and X start retains a per-boot first-generation lottery. Those are the current walls.
  • GZDoom reaching V_Init: allocate screen is not a rendered-frame claim; it proves the client's GL init and the direct-GL GR lane, not the game loop or a presented frame.
  • No frame-rate or latency numbers.

7. Related pages