BAR Emulation Specification

From Open-IOV
Revision as of 04:09, 12 October 2026 by WikiBot (talk | contribs) (Re-sync page from tracked wiki source: fix cross-document links and diagram; refresh status)
Jump to navigation Jump to search

BAR Emulation Specification (GSP 570.144W)

Historical document — superseded. The BAR_TRAP RPC (type 0x0A) described here is not used: in emulation mode it returns -ENOTSUPP, and BAR access is synchronous MMIO through the BAR shadow. Guest-visible BAR windows are BAR0 (16 MB register MMIO), BAR1 (256 MB VRAM aperture) and BAR3 (instance window). Canonical documentation: ../docs/03-architecture/06-host-module-and-hybrid-model.md.

Overview

The GSP firmware implements BAR (Base Address Register) emulation for virtual GPU (vGPU) through the BAR_TRAP RPC (type 0x0A). The main handler at 0x04CB8188 (40 KB) manages BAR1 aperture, forced access, and memory backing. This RPC is not used by the live path: in emulation mode BAR_TRAP returns -ENOTSUPP; guest BAR access is handled synchronously as MMIO through the BAR shadow (../kernel/bar_emulation.c), not via an RPC.

Key Addresses

Component Address Description
BAR Trap Handler 0x04CB8188 Main emulation logic (40 KB)
RPC Jump Table Entry 0x057675B7 BAR_TRAP in secondary dispatcher
String: BAR1 0x0406570E, 0x04085DE0 BAR1 identifier
String: Bar1Rest 0x040656DA BAR1 restore state
String: ForceBarAccess 0x040656F2 Forced access control
String: Bar1Aperture 0x04065722 Aperture configuration
Memory Allocator 0x429B450 Arena for BAR backing store
Cache Management 0x429B428 Flush/invalidate

Handler State Machine

Main Loop (0x04CB8188)

The handler runs a continuous event loop processing BAR trap requests: 

while (true) {
    // Wait for trap event (bitmask in unaff_s5)
    event = wait_for_bar_event();
    
    switch (event & 0x3F) {
        case 0x01:  // BAR1 write access
            handle_bar1_write();
            break;
        case 0x02:  // BAR1 read access
            handle_bar1_read();
            break;
        case 0x08:  // Aperture configuration
            handle_aperture_config();
            break;
        case 0x10:  // ForceBarAccess
            handle_force_access();
            break;
        case 0x20:  // Bar1Rest (restore)
            handle_bar1_restore();
            break;
        default:
            return ERROR_UNKNOWN_EVENT;
    }
}

Event Bitmask

Bit Event Description
0 BAR1_WRITE Guest VF wrote to BAR1
1 BAR1_READ Guest VF read from BAR1
3 APERTURE Aperture size/offset change
4 FORCE_ACCESS ForceBarAccess triggered
5 BAR1_REST Restore BAR1 state

Memory Management

Allocator Arena (0x429B450)

// Allocate backing store for BAR emulation
void *bar_alloc(size_t size) {
    // Calls UNK_ram_058A1A9C(0x429B450, size)
    // Returns pointer in 0x429B000 region
}

// Free backing store
void bar_free(void *ptr) {
    // Calls UNK_ram_05895BF4(0x429B428, ptr)
}

Allocation Sizes

  • Standard BAR1 page: 0x1000 (4 KB)
  • Aperture buffer: 0x40000 (256 KB) max
  • Force access buffer: 0x1000 (4 KB)

Cache Coherency

// Flush cache for DMA
void bar_cache_flush(void *addr, size_t size) {
    UNK_ram_05895BF4(0x429B428, addr);
    UNK_ram_05895BF4(0x429B428, addr + size);
}

BAR1 Emulation Logic

Write Path (Event 0x01)

int handle_bar1_write(bar_trap_t *trap) {
    // 1. Validate offset within aperture
    if (trap->offset >= bar1_aperture_size)
        return ERROR_OVERFLOW;
    
    // 2. Check ForceBarAccess flag
    if (force_bar_access && !(trap->flags & FORCE_ALLOWED))
        return ERROR_PERMISSION;
    
    // 3. Translate to backing store
    uint8_t *backing = bar1_backing + trap->offset;
    
    // 4. Perform write (1/2/4/8 bytes)
    switch (trap->size) {
        case 1: *backing = trap->value; break;
        case 2: *(uint16_t*)backing = trap->value; break;
        case 4: *(uint32_t*)backing = trap->value; break;
        case 8: *(uint64_t*)backing = trap->value; break;
    }
    
    // 5. Flush to device if needed
    if (device_needs_flush)
        bar_cache_flush(backing, trap->size);
    
    return OK;
}

Read Path (Event 0x02)

int handle_bar1_read(bar_trap_t *trap) {
    if (trap->offset >= bar1_aperture_size)
        return ERROR_OVERFLOW;
    
    uint8_t *backing = bar1_backing + trap->offset;
    
    switch (trap->size) {
        case 1: *trap->out_value = *backing; break;
        case 2: *trap->out_value = *(uint16_t*)backing; break;
        case 4: *trap->out_value = *(uint32_t*)backing; break;
        case 8: *trap->out_value = *(uint64_t*)backing; break;
    }
    
    return OK;
}

Aperture Configuration (Event 0x08)

int handle_aperture_config(bar_config_t *cfg) {
    // Validate new aperture
    if (cfg->size > MAX_APERTURE_SIZE)  // 0x40000
        return ERROR_INVALID;
    
    if (cfg->offset & 0xFFF)  // Must be 4KB aligned
        return ERROR_ALIGNMENT;
    
    // Reallocate backing store if size changed
    if (cfg->size != bar1_aperture_size) {
        void *new_backing = bar_alloc(cfg->size);
        if (!new_backing) return ERROR_NOMEM;
        
        // Copy existing data
        memcpy(new_backing, bar1_backing, 
               min(cfg->size, bar1_aperture_size));
        
        bar_free(bar1_backing);
        bar1_backing = new_backing;
        bar1_aperture_size = cfg->size;
    }
    
    bar1_aperture_offset = cfg->offset;
    return OK;
}

ForceBarAccess (Event 0x10)

int handle_force_access(force_access_t *fa) {
    // ForceBarAccess allows hypervisor to bypass VF permissions
    // Used for live migration, checkpoint/restore
    
    force_bar_access = fa->enable;
    force_access_vf = fa->vf_id;
    force_access_range = fa->range;
    
    // Allocate shadow buffer for forced operations
    if (fa->enable) {
        force_buffer = bar_alloc(0x1000);
        if (!force_buffer) return ERROR_NOMEM;
    } else {
        bar_free(force_buffer);
        force_buffer = NULL;
    }
    
    return OK;
}

Bar1Rest - State Restore (Event 0x20)

int handle_bar1_restore(restore_state_t *rs) {
    // Restore BAR1 state from checkpoint
    // Used during VF live migration
    
    bar1_aperture_offset = rs->aperture_offset;
    bar1_aperture_size = rs->aperture_size;
    force_bar_access = rs->force_access;
    
    // Restore backing store from serialized data
    if (rs->backing_size > 0) {
        void *new_backing = bar_alloc(rs->backing_size);
        if (!new_backing) return ERROR_NOMEM;
        
        memcpy(new_backing, rs->backing_data, rs->backing_size);
        bar_free(bar1_backing);
        bar1_backing = new_backing;
    }
    
    return OK;
}

vGPU-IO Integration

Kernel BAR Trap Registration

struct gsp_bar_ops {
    int (*bar_trap)(struct gsp_device *gsp, 
                    struct bar_trap_msg *msg);
    int (*configure_aperture)(struct gsp_device *gsp,
                              uint32_t vf_id,
                              uint64_t offset, uint64_t size);
    int (*force_access)(struct gsp_device *gsp,
                        uint32_t vf_id, bool enable);
    int (*save_state)(struct gsp_device *gsp,
                      uint32_t vf_id, void *buf, size_t *size);
    int (*restore_state)(struct gsp_device *gsp,
                         uint32_t vf_id, void *buf, size_t size);
};

static const struct gsp_bar_ops gsp_bar_ops = {
    .bar_trap = gsp_bar_trap_handler,
    .configure_aperture = gsp_bar_config_aperture,
    .force_access = gsp_bar_force_access,
    .save_state = gsp_bar_save_state,
    .restore_state = gsp_bar_restore_state,
};

VF BAR Configuration

int gsp_vf_bar_init(struct gsp_vf *vf) {
    struct bar_config cfg = {
        .vf_id = vf->id,
        .bar = 1,  // BAR1
        .aperture_offset = vf->bar1_offset,
        .aperture_size = vf->bar1_size,
        .flags = BAR_FLAG_COHERENT,
    };
    
    return gsp_rpc_send(vf->gsp, RPC_BAR_TRAP, 
                        BAR_SUBCMD_CONFIG, &cfg, sizeof(cfg));
}

Live Migration Support

// Save BAR state for migration
int gsp_vf_bar_save(struct gsp_vf *vf, void *buf, size_t *size) {
    struct bar_save_msg msg = {
        .vf_id = vf->id,
        .bar = 1,
    };
    
    return gsp_rpc_send(vf->gsp, RPC_BAR_TRAP,
                        BAR_SUBCMD_SAVE, &msg, sizeof(msg),
                        buf, size);
}

// Restore BAR state on destination
int gsp_vf_bar_restore(struct gsp_vf *vf, void *buf, size_t size) {
    struct bar_restore_msg msg = {
        .vf_id = vf->id,
        .bar = 1,
        .data = buf,
        .data_size = size,
    };
    
    return gsp_rpc_send(vf->gsp, RPC_BAR_TRAP,
                        BAR_SUBCMD_RESTORE, &msg, sizeof(msg));
}

Error Handling

Error Code Recovery
OOM (0x51) Allocator exhausted Shrink aperture, retry
Timeout (0x1E) Cache flush timeout Reset BAR, reinitialize
Invalid (0x1F) Bad aperture config Validate alignment/size
Permission (0xFFFF) ForceAccess violation Check VF permissions
Overflow (0x25) Offset beyond aperture Bounds check in VF driver

Debugging

Key Registers to Monitor

0x04296048  : Global error flag (cRam0000000004296048)
0x0429B750  : Error status (0x111300 offset)
0x429B428   : Cache management base
0x429B450   : Allocator arena base

Log Strings

Search for these in GSP logs:

  • BAR1 - BAR1 access
  • Bar1Rest - State restore
  • ForceBarAccess - Forced access
  • Bar1Aperture - Aperture changes
  • Bar1Size - Size queries

Reference

  • Handler: FUN_ram_04cb8188_ram_04cb8188.c (40 KB)
  • RPC Type: 0x0A (BAR_TRAP)
  • Jump Table Entry: 0x057675B7
Retrieved from "https://open-iov.org/index.php?title=BAR_Emulation_Specification&oldid=23846"