WPR Management

From Open-IOV
Jump to navigation Jump to search

An absence of critical technical documentation has historically slowed growth and adoption of developer ecosystems for GPU virtualization.

This CC-BY-4.0 licensed content can either be used with attribution, or used as inspiration for new documentation, created by GPU vendors for public commercial distribution as developer documentation.

Where possible, this documentation will clearly label dates and versions of observed-but-not-guaranteed behaviour vs. vendor-documented stable interfaces/behaviour with guarantees of forward or backward compatibility.

WPR (Write/Read Protect Region) Management

Historical document — superseded. WPR is the write-protected region of VRAM reserved for firmware, not a generic read/write-protect mechanism; the read-protect framing below is historical reverse-engineering material. Canonical documentation: ../docs/03-architecture/06-host-module-and-hybrid-model.md.

Overview

WPR (Write Protect Region / Read Protect Region) provides memory isolation for vGPU by protecting guest memory regions from unauthorized access. The GSP firmware implements WPR through two handlers at 0x04F00B0C and 0x04F00C78 (62 KB each).

Key Addresses

Handler Address Size Role
WPR Handler 1 0x04F00B0C 62 KB Primary WPR management
WPR Handler 2 0x04F00C78 62 KB Secondary/timeout handling
String Table 60 locations — Completion/timeout strings

String References (60 Total)

Completion Events

WPR_WRITE_COMPLETE      0x5E478F8
WPR_READ_COMPLETE       0x5E478E0
NV_WPR_WRITE_COMPLETE   0x5E478C8
NV_WPR_READ_COMPLETE    0x5E478B0

Timeout Events

WPR_WRITE_TIMEOUT       0x5E47898
WPR_READ_TIMEOUT        0x5E47880
NV_WPR_WRITE_TIMEOUT    0x5E47868
NV_WPR_READ_TIMEOUT     0x5E47850

WPR State Machine

Handler 1: 0x04F00B0C (Primary)

typedef enum {
    WPR_STATE_IDLE = 0,
    WPR_STATE_WRITE_PENDING = 1,
    WPR_STATE_READ_PENDING = 2,
    WPR_STATE_COMPLETE = 3,
    WPR_STATE_TIMEOUT = 4,
    WPR_STATE_ERROR = 0xFFFF
} wpr_state_t;

typedef struct {
    uint64_t region_base;      // Protected region base
    uint64_t region_size;      // Region size (page-aligned)
    uint32_t vf_id;            // Owning VF
    uint32_t flags;            // WPR_WRITE | WPR_READ | WPR_EXECUTE
    wpr_state_t state;
    uint64_t timeout_ns;       // Operation timeout
    void *completion_cb;       // Callback on complete
    void *timeout_cb;          // Callback on timeout
} wpr_region_t;

Operation Flow

Write Protect

1. VF requests WPR_WRITE on region [base, base+size)
2. Handler validates alignment (4KB) and permissions
3. Programs MMU/IOMMU write-protect bits
4. Starts timeout timer (configurable)
5. On GPU write attempt:
   a. Trap to GSP via page fault
   b. Check if VF owns region
   c. If allowed: emulate write, return OK
   d. If denied: trigger WPR_WRITE_COMPLETE/NV_WPR_WRITE_COMPLETE
6. On timeout: trigger WPR_WRITE_TIMEOUT/NV_WPR_WRITE_TIMEOUT

Read Protect

1. VF requests WPR_READ on region
2. Programs MMU/IOMMU read-protect bits
3. On GPU read attempt:
   a. Trap to GSP
   b. Return zero/poison data or emulate
   c. Trigger WPR_READ_COMPLETE/NV_WPR_READ_COMPLETE
4. On timeout: trigger WPR_READ_TIMEOUT/NV_WPR_READ_TIMEOUT

VBIOS Parsing (Handler 2: 0x04F00C78)

The WPR handlers parse VBIOS structures for legacy compatibility: 

int parse_vbios_for_wpr(void *vbios_ptr, size_t size) {
    // Search for signatures:
    // 0xAA55 - Standard VBIOS signature
    // 0x57464F4E ("WFON") - WPR enable
    // 0x5746564E ("WFVN") - WPR version
    
    for (offset = 0; offset < size; offset += 0x1000) {
        uint32_t sig = *(uint32_t*)(vbios_ptr + offset);
        
        if (sig == 0xAA55) {
            // Check for WPR structures
            parse_wpr_tables(vbios_ptr + offset);
        }
        else if (sig == 0x57464F4E || sig == 0x5746564E) {
            // WPR enabled in VBIOS
            wpr_vbios_enabled = true;
            wpr_vbios_offset = offset;
        }
    }
}

Integration with GSP Subsystems

Memory Allocator (0x429B450)

// Allocate WPR metadata structures
wpr_region_t *wpr_alloc_region() {
    return UNK_ram_058A1A9C(0x429B450, sizeof(wpr_region_t));
}

// Allocate bounce buffers for emulation
void *wpr_alloc_bounce(size_t size) {
    return UNK_ram_058A1A9C(0x429B450, size);
}

DMA/Copy Engine (0x0588F9C4, 0x05AEC86C)

// Copy data for WPR emulation
int wpr_emulate_access(wpr_region_t *rgn, void *dst, void *src, size_t size) {
    return UNK_ram_0588F9C4(dma_engine, src, dst, size);
}

Logging (0x05AB7DA0)

// Log WPR events
void wpr_log_event(const char *event_str, uint32_t vf_id, uint64_t addr) {
    UNK_ram_05AB7DA0(1, &event_str);  // Log string
    UNK_ram_05AB7DA0(2, &vf_id);      // Log VF ID
    UNK_ram_05AB7DA0(3, &addr);       // Log address
}

vGPU-IO Integration

Kernel WPR API

struct gsp_wpr_ops {
    int (*protect_write)(struct gsp_device *gsp, 
                         uint32_t vf_id,
                         uint64_t base, uint64_t size);
    int (*protect_read)(struct gsp_device *gsp,
                        uint32_t vf_id,
                        uint64_t base, uint64_t size);
    int (*unprotect)(struct gsp_device *gsp,
                     uint32_t vf_id,
                     uint64_t base);
    int (*set_timeout)(struct gsp_device *gsp,
                       uint32_t vf_id,
                       uint64_t timeout_ns);
    int (*register_callback)(struct gsp_device *gsp,
                             uint32_t vf_id,
                             enum wpr_event_type type,
                             void (*cb)(struct wpr_event *));
};

static const struct gsp_wpr_ops gsp_wpr_ops = {
    .protect_write = gsp_wpr_write_protect,
    .protect_read = gsp_wpr_read_protect,
    .unprotect = gsp_wpr_unprotect,
    .set_timeout = gsp_wpr_set_timeout,
    .register_callback = gsp_wpr_register_callback,
};

VF WPR Lifecycle

// VF initialization: register WPR regions
int gsp_vf_wpr_init(struct gsp_vf *vf) {
    // Protect VF's page tables
    gsp_wpr_ops.protect_write(vf->gsp, vf->id,
                              vf->pt_base, vf->pt_size);
    
    // Protect VF's command buffers
    gsp_wpr_ops.protect_write(vf->gsp, vf->id,
                              vf->cmd_base, vf->cmd_size);
    
    // Set default timeout (10ms)
    gsp_wpr_ops.set_timeout(vf->gsp, vf->id, 10000000);
    
    // Register completion callbacks
    gsp_wpr_ops.register_callback(vf->gsp, vf->id,
                                  WPR_EVENT_WRITE_COMPLETE,
                                  vf_wpr_write_complete_cb);
    gsp_wpr_ops.register_callback(vf->gsp, vf->id,
                                  WPR_EVENT_TIMEOUT,
                                  vf_wpr_timeout_cb);
    
    return 0;
}

// VF cleanup
void gsp_vf_wpr_fini(struct gsp_vf *vf) {
    gsp_wpr_ops.unprotect(vf->gsp, vf->id, vf->pt_base);
    gsp_wpr_ops.unprotect(vf->gsp, vf->id, vf->cmd_base);
    // ... unregister callbacks
}

Callback Handling

static void vf_wpr_write_complete_cb(struct wpr_event *evt) {
    struct gsp_vf *vf = evt->vf;
    
    // Log the write attempt
    dev_dbg(vf->dev, "WPR write complete: VF=%u addr=0x%llx\n",
            vf->id, evt->address);
    
    // If this was a page table update, invalidate TLB
    if (evt->address >= vf->pt_base && 
        evt->address < vf->pt_base + vf->pt_size) {
        gsp_tlb_invalidate(vf->gsp, vf->id);
    }
}

static void vf_wpr_timeout_cb(struct wpr_event *evt) {
    struct gsp_vf *vf = evt->vf;
    
    dev_warn(vf->dev, "WPR timeout: VF=%u type=%d addr=0x%llx\n",
             vf->id, evt->type, evt->address);
    
    // Escalate: reset VF or throttle
    if (evt->type == WPR_EVENT_WRITE_TIMEOUT) {
        gsp_vf_throttle(vf);
    } else {
        gsp_vf_reset(vf);
    }
}

Error Codes

Code Name Description
0x1E ETIME Operation timed out
0x1F EDEV Device error
0x25 EOVERFLOW Region too large
0x37 EINVAL Invalid alignment
0x51 ENOMEM Out of memory
0x56 EBUSY Region busy
0xFFFF EPERM Permission denied

Debugging

Monitor These Addresses

0x04296048  : Global error flag
0x0429B750+0x111300 : Error status register
0x05E47850-0x05E478F8 : WPR string table

GSP Log Patterns

WPR_WRITE_COMPLETE    VF=<id> ADDR=<addr>
WPR_READ_COMPLETE     VF=<id> ADDR=<addr>
WPR_WRITE_TIMEOUT     VF=<id> ADDR=<addr>
WPR_READ_TIMEOUT      VF=<id> ADDR=<addr>
NV_WPR_WRITE_COMPLETE VF=<id> ADDR=<addr>
NV_WPR_READ_COMPLETE  VF=<id> ADDR=<addr>
NV_WPR_WRITE_TIMEOUT  VF=<id> ADDR=<addr>
NV_WPR_READ_TIMEOUT   VF=<id> ADDR=<addr>

Trace Points

// In kernel driver
trace_gsp_wpr_protect(vf_id, base, size, type);
trace_gsp_wpr_event(vf_id, event_type, address, status);
trace_gsp_wpr_timeout(vf_id, address, elapsed_ns);

Reference

  • Handler 1: FUN_ram_04f00b0c_ram_04f00b0c.c (62 KB)
  • Handler 2: FUN_ram_04f00c78_ram_04f00c78.c (62 KB)
  • RPC Types: Handled via primary/secondary dispatchers
Retrieved from "https://open-iov.org/index.php?title=WPR_Management&oldid=23992"