WikiBot: Fix repository links: point to the cgit tree URLs (git.vgpu.io/open-iov.git/tree/...)

2026-10-12T04:40:10Z

Fix repository links: point to the cgit tree URLs (git.vgpu.io/open-iov.git/tree/...)

← Older revision		Revision as of 04:40, 12 October 2026
Line 3:		Line 3:
	= GSP vGPU RPC Emulation =		= GSP vGPU RPC Emulation =

	<blockquote>'''Historical document — superseded.''' This page describes an earlier emulation-mode design and is retained for reference only. The project does '''not''' use NVIDIA's proprietary vGPU stack. Canonical documentation: [https://git.vgpu.io/open-iov/~~blob/master~~/docs/03-architecture/03-gsp-and-gsp-rm.md <code>../docs/03-architecture/03-gsp-and-gsp-rm.md</code>] and [https://git.vgpu.io/open-iov/~~blob/master~~/docs/03-architecture/06-host-module-and-hybrid-model.md <code>../docs/03-architecture/06-host-module-and-hybrid-model.md</code>].</blockquote>		<blockquote>'''Historical document — superseded.''' This page describes an earlier emulation-mode design and is retained for reference only. The project does '''not''' use NVIDIA's proprietary vGPU stack. Canonical documentation: [https://git.vgpu.io/open-iov.git/tree/docs/03-architecture/03-gsp-and-gsp-rm.md <code>../docs/03-architecture/03-gsp-and-gsp-rm.md</code>] and [https://git.vgpu.io/open-iov.git/tree/docs/03-architecture/06-host-module-and-hybrid-model.md <code>../docs/03-architecture/06-host-module-and-hybrid-model.md</code>].</blockquote>

	== Overview ==		== Overview ==

WikiBot: Carry the site-wide About/licensing message on every page (2026-10-12)

2026-10-12T04:29:46Z

Carry the site-wide About/licensing message on every page (2026-10-12)

← Older revision		Revision as of 04:29, 12 October 2026
Line 1:		Line 1:
			{{About message}}

	= GSP vGPU RPC Emulation =		= GSP vGPU RPC Emulation =

WikiBot: Re-sync page from tracked wiki source: fix cross-document links and diagram; refresh status

2026-10-12T04:10:03Z

Re-sync page from tracked wiki source: fix cross-document links and diagram; refresh status

Show changes

WikiBot: Remove unpaired blockquote tags left by reference cleanup

2026-10-12T03:58:31Z

Remove unpaired blockquote tags left by reference cleanup

New page

= GSP vGPU RPC Emulation =


== Overview ==

Open-IOV includes '''emulation mode''' that allows operation with unmodified consumer/OpenRM GSP firmware by emulating missing vGPU RPC handlers locally in kernel and userspace.

== Problem ==

The NVIDIA GSP firmware on consumer drivers (550.163.01) and OpenRM lacks the vGPU-specific RPC handlers that exist only in the GRID host driver firmware:

{| class="wikitable"
|-
! Handler
! Address
! Size
! Purpose
|-
| BAR Trap
| 0x04CB8188
| 40KB
| BAR1/ForceBarAccess/Aperture
|-
| WPR Handler 1
| 0x04F00B0C
| 62KB
| Write/Read Protect Regions
|-
| WPR Handler 2
| 0x04F00C78
| 62KB
| Write/Read Protect Regions
|-
| Migration
| 0x04179ED8/0x04179FBC
| -
| Live migration
|}

Without these handlers, VF creation times out at BAR configuration (RPC to 0x04CB8188).

== Solution: Emulation Mode ==

Emulation mode intercepts vGPU RPC calls and handles them locally instead of sending to GSP:

<pre>┌─────────────────────────────────────────────────────────────────┐
│ Emulation Architecture │
├─────────────────────────────────────────────────────────────────┤
│ │
│ Userspace App (libiov.so) │
│ │ │
│ ▼ │
│ ┌─────────────────┐ ┌─────────────────┐ │
│ │ Kernel IOCTL │────▶│ Emulation Layer │ │
│ │ Handler │ │ (kernel) │ │
│ └─────────────────┘ └────────┬────────┘ │
│ │ │
│ ┌───────────────┼───────────────┐ │
│ ▼ ▼ ▼ │
│ ┌──────────┐ ┌──────────┐ ┌──────────┐ │
│ │ BAR Emu │ │ WPR Emu │ │ Mem Emu │ │
│ │ (kernel) │ │ (kernel) │ │ (kernel) │ │
│ └──────────┘ └──────────┘ └──────────┘ │
│ │ │ │ │
│ ▼ ▼ ▼ │
│ ┌──────────────────────────────────────────┐ │
│ │ Userspace Emulator (libiov.so) │ │
│ │ rpc_emu_bar_*, rpc_emu_wpr_* │ │
│ └──────────────────────────────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────────┘</pre>

== Auto-Detection (Default Behavior) ==

'''Emulation mode is ON by default''' and auto-detects at driver load:

# Driver initializes RPC communication with GSP
# Sends test <code>BAR_TRAP</code> RPC (non-destructive READ)
# '''If GSP responds''' (ret=0) → vGPU handlers present → '''emulation disabled'''
# '''If GSP times out/errors''' (ret=-ETIMEDOUT/-EIO/-ENODEV) → handlers missing → '''emulation enabled'''

<pre>dmesg output:
gsp-vgpu: Auto-detecting GSP vGPU RPC handler support..
gsp-vgpu: GSP lacks vGPU RPC handlers (ret=-110) - ENABLING emulation mode
# or
gsp-vgpu: GSP has vGPU RPC handlers - DISABLING emulation mode</pre>

== Module Parameters ==

{| class="wikitable"
|-
! Parameter
! Type
! Default
! Description
|-
| <code>emulation_mode</code>
| bool
| 1 (ON)
| Current state (read-only, shows auto-detected value)
|-
| <code>emulation_override</code>
| bool
| 0 (auto)
| Override: 1=force emulation ON, 0=force emulation OFF
|}

<syntaxhighlight lang="bash"># Default: auto-detect
sudo insmod gsp-vgpu.ko firmware_name=gsp-570.144.fwimage.bin

# Force emulation ON (testing with GRID GSP)
sudo insmod gsp-vgpu.ko firmware_name=gsp-570.144.fwimage.bin emulation_override=1

# Force emulation OFF (requires GRID GSP)
sudo insmod gsp-vgpu.ko firmware_name=gsp-570.144.fwimage.bin emulation_override=0</syntaxhighlight>

== Emulated RPC Handlers ==


=== Kernel Emulation (<code>driver/</code>) ===

{| class="wikitable"
|-
! RPC Type
! Handler
! Emulation
|-
| <code>GSP_RPC_ALLOC</code> (0x01)
| <code>gsp_vf_mem_alloc</code>
| Synthetic handles/VAs from local counter
|-
| <code>GSP_RPC_PIN</code> (0x05)
| <code>gsp_vf_mem_pin</code>
| Synthetic DMA addresses from local counter
|-
| <code>GSP_RPC_MAP</code> (0x03)
| <code>gsp_vf_mem_map</code>
| Identity map (return requested VA)
|-
| <code>GSP_RPC_BAR_TRAP</code> (0x0A)
| <code>gsp_bar_trap_handler</code>
| Local aperture/force_access state
|-
| <code>GSP_RPC_MAP</code> (WPR)
| <code>gsp_vf_wpr_protect</code>
| Local WPR region list
|-
| <code>GSP_RPC_UNMAP</code> (WPR)
| <code>gsp_vf_wpr_unprotect</code>
| Local WPR region removal
|}


=== Userspace Emulation (<code>userspace/lib/rpc_emulator.c</code>) ===

{| class="wikitable"
|-
! Function
! Purpose
|-
| <code>rpc_emu_bar_config</code>
| Local aperture offset/size + backing store
|-
| <code>rpc_emu_bar_force_access</code>
| Local force_access state
|-
| <code>rpc_emu_bar_save/restore</code>
| Migration state serialization
|-
| <code>rpc_emu_wpr_protect_write/read</code>
| Local WPR region tracking
|-
| <code>rpc_emu_wpr_unprotect</code>
| Local WPR region removal
|-
| <code>rpc_emu_wpr_set_timeout</code>
| Local timeout tracking
|-
| <code>rpc_emu_wpr_check_access</code>
| Access violation detection
|}


== Emulation Data Structures ==


=== BAR State (per VF) ===

<syntaxhighlight lang="c">struct gsp_bar_state {
uint64_t aperture_offset; // BAR1 aperture base
uint64_t aperture_size; // BAR1 aperture size (max 256KB)
bool force_access; // ForceBarAccess enabled
uint32_t force_vf; // VF holding force access
void *backing_store; // Emulated BAR memory
size_t backing_size; // Backing store size
};</syntaxhighlight>

=== WPR Region List (per VF) ===

<syntaxhighlight lang="c">struct gsp_wpr_region {
struct list_head list;
uint64_t base; // Region base address
uint64_t size; // Region size
uint32_t flags; // GSP_WPR_WRITE/READ/EXECUTE
uint32_t vf_id; // Owning VF
void (*completion_cb)(..); // Write/read complete callback
void (*timeout_cb)(..); // Timeout callback
uint64_t timeout_ns; // Timeout in nanoseconds
};</syntaxhighlight>

=== Userspace Emulator State ===

<syntaxhighlight lang="c">struct emu_bar_state {
uint64_t aperture_offset;
uint64_t aperture_size;
bool force_access;
uint32_t force_vf;
void *backing_store;
size_t backing_size;
pthread_mutex_t lock;
};

struct emu_wpr_state {
struct emu_wpr_region *regions; // Linked list
pthread_mutex_t lock;
};</syntaxhighlight>

== Fallback Logic ==

Userspace library (<code>libiov.so</code>) implements fallback:

<syntaxhighlight lang="c">vgpu_error_t vgpu_bar_config(vgpu_vf_t vf, uint64_t offset, uint64_t size) {
// 1. Try kernel IOCTL (sends RPC to GSP)
ret = ioctl(fd, GSP_IOCTL_BAR_CONFIG, &args);

// 2. If GSP lacks handlers (ENOTSUP/ETIMEDOUT/EIO), use emulator
if (ret < 0 && dev->emulation_enabled &&
(errno == ENOTSUP || errno == ETIMEDOUT || errno == EIO)) {
return rpc_emu_bar_config(vf->id, offset, size);
}

return map_errno(errno);
}</syntaxhighlight>

== Limitations ==

{| class="wikitable"
|-
! Feature
! Emulation Status
! Notes
|-
| BAR Aperture Config
| ✅ Full
| Local state + backing store
|-
| ForceBarAccess
| ✅ Full
| Per-VF exclusive access
|-
| BAR Save/Restore
| ✅ Full
| Migration support
|-
| WPR Write Protect
| ✅ Full
| Region tracking + access check
|-
| WPR Read Protect
| ✅ Full
| Region tracking + access check
|-
| WPR Timeout
| ✅ Partial
| Timeout stored, callbacks not wired
|-
| Memory Alloc
| ✅ Basic
| Synthetic handles/VAs
|-
| Memory Pin
| ✅ Basic
| Synthetic DMA addresses
|-
| Memory Map
| ✅ Basic
| Identity mapping
|-
| Migration
| ⚠️ Partial
| BAR/WPR state only
|-
| Scheduling
| ❌ No
| Returns success, no actual scheduling
|-
| Notifications
| ❌ No
| No async event delivery
|}


== Testing Emulation ==

<syntaxhighlight lang="bash"># 1. Load driver (auto-enables emulation on consumer GSP)
sudo insmod driver/gsp-vgpu.ko firmware_name=gsp-570.144.fwimage.bin

# 2. Verify emulation enabled in dmesg
dmesg -T | grep -i emulation
# gsp-vgpu: GSP lacks vGPU RPC handlers (ret=-110) - ENABLING emulation mode

# 3. Run example - should complete without timeouts
sudo LD_LIBRARY_PATH=userspace/lib./userspace/examples/example_vgpu /dev/gsp-vgpu

# 4. Check debugfs for emulated state
cat /sys/kernel/debug/gsp-vgpu/vfs/vf1/bar_state
cat /sys/kernel/debug/gsp-vgpu/vfs/vf1/wpr_regions</syntaxhighlight>

== Future Work ==

* ☐ Wire WPR timeout callbacks to kernel timers
* ☐ Implement emulated scheduling with work queues
* ☐ Add async notification support via eventfd
* ☐ Complete migration state for WPR regions
* ☐ Performance optimization (batch RPC emulation)
* ☐ Conformance tests vs GRID GSP behavior


== Related Documentation ==

* [[BAR Emulation Specification]]
* [[WPR Management]]
* [[RPC Message Format]]
* [[vGPU-IO Integration Guide]]

GSP RPC Emulation - Revision history

WikiBot: Fix repository links: point to the cgit tree URLs (git.vgpu.io/open-iov.git/tree/...)

WikiBot: Carry the site-wide About/licensing message on every page (2026-10-12)

WikiBot: Re-sync page from tracked wiki source: fix cross-document links and diagram; refresh status

WikiBot: Remove unpaired blockquote tags left by reference cleanup