Difference between revisions of "Virtual I/O Internals"
| Line 21: | Line 21: | ||
|Guest is ignorant of host supported software such as support libraries. | |Guest is ignorant of host supported software such as support libraries. | ||
|} | |} | ||
==RPC Mode== | ==RPC Mode== | ||
| Line 29: | Line 28: | ||
=== Memory Management === | === Memory Management === | ||
Guests | Guests may be presented with emulated memory regions which use indirect emulated communication requiring a VM-exit (slow) or instead the guest may be presented with passthrough memory regions which use direct communication requiring no VM-exit (fast). | ||
=== Scheduling === | |||
Scheduling | |||
| Line 41: | Line 44: | ||
==SR-IOV Mode== | ==SR-IOV Mode== | ||
=== Instruction Execution === | |||
SR-IOV Mode involves the communication of instructions from a virtual function (VF) through direct communication to the [https://infogalactic.com/info/PCI_configuration_space PCI BAR]. | SR-IOV Mode involves the communication of instructions from a virtual function (VF) through direct communication to the [https://infogalactic.com/info/PCI_configuration_space PCI BAR]. | ||
=== Memory Management === | |||
Guests are presenting with passthrough memory regions by the device firmware. | |||
=== Scheduling === | |||
| Line 49: | Line 59: | ||
HPA<->GPA Boundary Enforcement. | HPA<->GPA Boundary Enforcement. | ||
== Both Modes == | |||
=== VFIO file descriptor === | |||
VFIO devices are mapped based on a file offset representing the virtual PCI device. | |||
{| class="wikitable" | |||
|+BAR regions in a VGA PCI device. | |||
!00:00.0 VGA compatible controller | |||
|- | |||
|Region 0 Bar0 (starts at offset 0) | |||
|- | |||
|Region 1 Bar1 | |||
|- | |||
|Region 2 Bar2 | |||
|- | |||
|Region 3 Bar3 | |||
|- | |||
|Region 4 Bar4 | |||
|- | |||
|Region 5 Bar5 (IO port space) | |||
|- | |||
|Expansion ROM | |||
|} | |||
{| class="wikitable" | |||
|+VFIO representation of PCI BAR regions offsets. | |||
! colspan="4" |<- File Offset -> | |||
! | |||
|- | |||
!0 -> A | |||
!A -> A+B | |||
!A+B -> A+B+C | |||
!A+B+C -> A+B+C+D | |||
!... | |||
|- | |||
|Region 0 (A) | |||
|Region 1 (B) | |||
|Region 2 (C) | |||
|Region 3 (D) | |||
|... | |||
|} | |||
=== IRQ remapping === | |||
Revision as of 21:25, 24 April 2022
The following document will attempt to detail the internals of a Virtual Function IO (VFIO) driven Mediated Device (Mdev).
| RPC Mode | SR-IOV Mode |
|---|---|
| Host requires insight about guest of workload. | Host ignorance of guest workload. |
| Error reporting. | No guest driver error reporting. |
| In depth dynamic monitoring. | Basic dynamic monitoring. |
| Software defined MMU guest separation. | Firmware defined MMU guest separation. |
| Requires deferred instructions to be supported by host software (support libraries). | Guest is ignorant of host supported software such as support libraries. |
RPC Mode
Instruction Execution
RPC Mode moves instruction information across a virtual function interface (VF) using Remote Procedure Calls generally by way of soft interrupt (IOCTLs). Guest GPU instructions passed from the guest as Remote Procedure Calls are Just-in-time recompiled on the host for execution by a device driver.
Memory Management
Guests may be presented with emulated memory regions which use indirect emulated communication requiring a VM-exit (slow) or instead the guest may be presented with passthrough memory regions which use direct communication requiring no VM-exit (fast).
Scheduling
Scheduling
RPC Mode Requirements:
Sensitive Instruction List.
Instruction Shim/Binary Translator.
HPA<->GPA Boundary Enforcement.
SR-IOV Mode
Instruction Execution
SR-IOV Mode involves the communication of instructions from a virtual function (VF) through direct communication to the PCI BAR.
Memory Management
Guests are presenting with passthrough memory regions by the device firmware.
Scheduling
RPC Mode Requirements:
Device SR-IOV support.
HPA<->GPA Boundary Enforcement.
Both Modes
VFIO file descriptor
VFIO devices are mapped based on a file offset representing the virtual PCI device.
| 00:00.0 VGA compatible controller |
|---|
| Region 0 Bar0 (starts at offset 0) |
| Region 1 Bar1 |
| Region 2 Bar2 |
| Region 3 Bar3 |
| Region 4 Bar4 |
| Region 5 Bar5 (IO port space) |
| Expansion ROM |
| <- File Offset -> | ||||
|---|---|---|---|---|
| 0 -> A | A -> A+B | A+B -> A+B+C | A+B+C -> A+B+C+D | ... |
| Region 0 (A) | Region 1 (B) | Region 2 (C) | Region 3 (D) | ... |