Difference between revisions of "GPU Support"
Tag: Reverted |
|||
(18 intermediate revisions by the same user not shown) | |||
Line 1: | Line 1: | ||
== | == Abstract == | ||
The following document will detail GPU Support for SR-IOV, SIOV, and VFIO-Mdev functionality. | The following document will detail GPU Support for SR-IOV, SIOV, and VFIO-Mdev functionality.<blockquote>An absence of critical technical documentation has historically slowed growth and adoption of developer ecosystems for GPU virtualization. | ||
This [https://creativecommons.org/licenses/by/4.0/ CC-BY-4.0] licensed content can either be used with attribution, or used as inspiration for new documentation, created by GPU vendors for public commercial distribution as developer documentation. | |||
Where possible, this documentation will clearly label dates and versions of observed-but-not-guaranteed behaviour vs. vendor-documented stable interfaces/behaviour with guarantees of forward or backward compatibility.</blockquote> | |||
== Overview == | == Overview == | ||
Line 8: | Line 12: | ||
! colspan="4" |Drivers | ! colspan="4" |Drivers | ||
! colspan="3" |[https://openmdev.io/index.php/Virtual_IO_Internals Virtual I/O Assistance Modes] | ! colspan="3" |[https://openmdev.io/index.php/Virtual_IO_Internals Virtual I/O Assistance Modes] | ||
! colspan="2" |Userspace | |||
! colspan=" | |||
|- | |- | ||
!Vendor | !Vendor | ||
!Architecture | !Architecture | ||
!Model | !Model | ||
![https://www.fsf.org/about/what-is-free-software Libre] Drivers | ![https://www.fsf.org/about/what-is-free-software Libre] Drivers | ||
! | !Blob Drivers | ||
![https://openmdev.io/index.php/Merged_Drivers 'Merged' Host+Guest Acceleration] | ![https://openmdev.io/index.php/Merged_Drivers 'Merged' Host+Guest Acceleration] | ||
!Manufacturer Support | |||
![https://openmdev.io/index.php/Virtual_IO_Internals#Mdev_Mode Mdev Support] | ![https://openmdev.io/index.php/Virtual_IO_Internals#Mdev_Mode Mdev Support] | ||
![https://openmdev.io/index.php/Virtual_IO_Internals#SR-IOV_Mode SR-IOV Support] | ![https://openmdev.io/index.php/Virtual_IO_Internals#SR-IOV_Mode SR-IOV Support] | ||
![https://openmdev.io/index.php/Virtual_IO_Internals#SIOV_Mode SIOV Support] | ![https://openmdev.io/index.php/Virtual_IO_Internals#SIOV_Mode SIOV Support] | ||
![https://docs.linux-gvm.org/ GVM-cli Support] | ![https://docs.linux-gvm.org/ GVM-cli Support] | ||
![https://libvf.io/ LibVF.IO Support] | ![https://libvf.io/ LibVF.IO Support] | ||
|- | |- | ||
Line 30: | Line 30: | ||
|6th Generation (Skylake) | |6th Generation (Skylake) | ||
|*HD | |*HD | ||
|[https:// | |[https://open-iov.org/index.php/GPU_Driver_Internals#i915 i915] | ||
| | | | ||
|Yes | |Yes | ||
|Yes ([https://community.intel.com/t5/Graphics/Graphics-Driver-Support-Update-for-10th-Generation-and-Older/m-p/1403969/thread-id/108899 Legacy]) | |||
|Yes | |Yes | ||
|No | |No | ||
|No | |No | ||
|No | |No | ||
|Yes | |Yes | ||
|- | |- | ||
Line 46: | Line 43: | ||
|7th Generation (Kaby Lake) | |7th Generation (Kaby Lake) | ||
|*HD | |*HD | ||
|[https:// | |[https://open-iov.org/index.php/GPU_Driver_Internals#i915 i915] | ||
| | | | ||
|Yes | |Yes | ||
|Yes ([https://community.intel.com/t5/Graphics/Graphics-Driver-Support-Update-for-10th-Generation-and-Older/m-p/1403969/thread-id/108899 Legacy]) | |||
|Yes | |Yes | ||
|No | |No | ||
|No | |No | ||
|No | |No | ||
|Yes | |Yes | ||
|- | |- | ||
Line 62: | Line 56: | ||
|8th Generation (Coffee Lake) | |8th Generation (Coffee Lake) | ||
|*HD | |*HD | ||
|[https:// | |[https://open-iov.org/index.php/GPU_Driver_Internals#i915 i915] | ||
| | | | ||
|Yes | |Yes | ||
|Yes ([https://community.intel.com/t5/Graphics/Graphics-Driver-Support-Update-for-10th-Generation-and-Older/m-p/1403969/thread-id/108899 Legacy]) | |||
|Yes | |Yes | ||
|No | |No | ||
|No | |No | ||
|No | |No | ||
|Yes | |Yes | ||
|- | |- | ||
Line 78: | Line 69: | ||
|9th Generation (Cannon Lake) | |9th Generation (Cannon Lake) | ||
|*HD | |*HD | ||
|[https:// | |[https://open-iov.org/index.php/GPU_Driver_Internals#i915 i915] | ||
| | | | ||
|Yes | |Yes | ||
|Yes ([https://community.intel.com/t5/Graphics/Graphics-Driver-Support-Update-for-10th-Generation-and-Older/m-p/1403969/thread-id/108899 Legacy]) | |||
|Yes | |Yes | ||
|No | |No | ||
|No | |No | ||
|No | |No | ||
|Yes | |Yes | ||
|- | |- | ||
Line 94: | Line 82: | ||
|10th Generation (Ice Lake) | |10th Generation (Ice Lake) | ||
| | | | ||
|[https:// | |[https://open-iov.org/index.php/GPU_Driver_Internals#i915 i915] | ||
| | | | ||
|No | |No | ||
|Yes ([https://community.intel.com/t5/Graphics/Graphics-Driver-Support-Update-for-10th-Generation-and-Older/m-p/1403969/thread-id/108899 Legacy]) | |||
|No | |No | ||
|No | |No | ||
|No | |No | ||
|No | |No | ||
|No | |No | ||
|- | |- | ||
Line 110: | Line 95: | ||
|11th Generation (Tiger Lake) | |11th Generation (Tiger Lake) | ||
|Xe* | |Xe* | ||
| | |[https://open-iov.org/index.php/GPU_Driver_Internals#i915 i915] | ||
| | | | ||
|Yes | |||
|Yes | |Yes | ||
| | | | ||
|[https://archive.ph/0McAE#selection-4883.0-4943.35 Yes] | |[https://archive.ph/0McAE#selection-4883.0-4943.35 Yes] | ||
| | | | ||
| | |Yes | ||
|Yes | |Yes | ||
|- | |- | ||
Line 126: | Line 108: | ||
|12th Generation (Alder Lake) | |12th Generation (Alder Lake) | ||
|Xe* | |Xe* | ||
| | |[https://open-iov.org/index.php/GPU_Driver_Internals#i915 i915] | ||
| | | | ||
|Yes | |||
|Yes | |Yes | ||
| | | | ||
|[https://archive.ph/0McAE#selection-4883.0-4943.35 Yes] | |[https://archive.ph/0McAE#selection-4883.0-4943.35 Yes] | ||
|In Development | |In Development | ||
|Yes | |Yes | ||
|Yes | |Yes | ||
|- | |- | ||
|Nvidia | |Nvidia | ||
|Maxell | |Maxell | ||
|*(see exceptions below) | |*(see exceptions below) | ||
|[https://nouveau.freedesktop.org/ Nouveau], [https://open-iov.org/index.php/GPU_Driver_Internals#OpenRM OpenRM] | |||
|[https://nouveau.freedesktop.org/ Nouveau], [https://open-iov.org/index.php/OpenRM OpenRM] | |RM | ||
| | |Yes | ||
|Yes | |Yes | ||
|Yes | |Yes | ||
|No | |No | ||
|No | |No | ||
|Yes | |Yes | ||
|Yes | |Yes | ||
|- | |- | ||
Line 174: | Line 134: | ||
|Pascal | |Pascal | ||
|* | |* | ||
|[https://nouveau.freedesktop.org/ Nouveau], [https://open-iov.org/index.php/GPU_Driver_Internals#OpenRM OpenRM] | |||
|[https://nouveau.freedesktop.org/ Nouveau], [https://open-iov.org/index.php/OpenRM OpenRM] | |RM | ||
| | |Yes | ||
|Yes | |Yes | ||
|Yes | |Yes | ||
|No | |No | ||
|No | |No | ||
|Yes | |Yes | ||
|Yes | |Yes | ||
|- | |- | ||
Line 190: | Line 147: | ||
|Turing | |Turing | ||
|* | |* | ||
|[https://nouveau.freedesktop.org/ Nouveau], [https://open-iov.org/index.php/GPU_Driver_Internals#OpenRM OpenRM] | |||
|[https://nouveau.freedesktop.org/ Nouveau], [https:// | |RM | ||
| | |Yes | ||
|Yes | |Yes | ||
|Yes | |Yes | ||
|No | |No | ||
|No | |No | ||
|Yes | |Yes | ||
|Yes | |Yes | ||
|- | |- | ||
Line 206: | Line 160: | ||
|Ampere | |Ampere | ||
|A* | |A* | ||
|[https://nouveau.freedesktop.org/ Nouveau] ([https://nouveau.freedesktop.org/FeatureMatrix.html Development]), [https://open-iov.org/index.php/GPU_Driver_Internals#OpenRM OpenRM] | |||
|[https://nouveau.freedesktop.org/ Nouveau] ([https://nouveau.freedesktop.org/FeatureMatrix.html | |RM | ||
| | |No ([https://developer.nvidia.com/displaymodeselector DisplayModeSelector]) | ||
| | |||
|Yes | |Yes | ||
|Yes | |Yes | ||
|Yes | |Yes | ||
|In Development | |In Development | ||
| | |Yes | ||
|Yes | |Yes | ||
|- | |- | ||
Line 222: | Line 173: | ||
|Ampere | |Ampere | ||
|RTX 30** | |RTX 30** | ||
|[https://nouveau.freedesktop.org/ Nouveau] ([https://nouveau.freedesktop.org/FeatureMatrix.html Development]), [https://open-iov.org/index.php/GPU_Driver_Internals#OpenRM OpenRM] | |||
|[https://nouveau.freedesktop.org/ Nouveau] ([https://nouveau.freedesktop.org/FeatureMatrix.html | |RM | ||
| | |||
|In Development | |In Development | ||
|Yes | |||
|Yes | |Yes | ||
|No | |No | ||
|In Development | |In Development | ||
|In Development | |In Development | ||
Line 238: | Line 186: | ||
|Ada Lovelace | |Ada Lovelace | ||
|* | |* | ||
|[https://open-iov.org/index.php/GPU_Driver_Internals#OpenRM OpenRM] | |||
|[https://open-iov.org/index.php/OpenRM OpenRM] | |||
| | | | ||
| | | | ||
|Yes | |||
|Yes | |||
|Yes | |||
|In Development | |||
|In Development | |In Development | ||
|In Development | |In Development | ||
|- | |- | ||
|Nvidia | |Nvidia | ||
|Hopper | |Hopper | ||
|* | |* | ||
|[https://open-iov.org/index.php/GPU_Driver_Internals#OpenRM OpenRM] | |||
|[https://open-iov.org/index.php/OpenRM OpenRM] | |||
| | | | ||
| | | | ||
|Yes | |||
|Yes | |||
|Yes | |||
|In Development | |||
| | | | ||
| | | | ||
|- | |- | ||
Line 270: | Line 212: | ||
|Tonga | |Tonga | ||
|W7100 | |W7100 | ||
|AMDGPU, [https://github.com/GPUOpen-LibrariesAndSDKs/MxGPU-Virtualization MxGPU / GIM / GPU-IOV Module (Deprecated)] | |||
|AMDGPU, [https://github.com/GPUOpen-LibrariesAndSDKs/MxGPU-Virtualization GPU-IOV Module (Deprecated)] | |||
|AMDGPU-Pro | |AMDGPU-Pro | ||
|No | |No | ||
|[[wikipedia:End-of-life_product|EOL]] (2017) | |||
|No | |No | ||
|Yes | |Yes | ||
([https://forum.level1techs.com/t/how-to-sr-iov-mod-the-w7100-gpu/164186 patched]) | ([https://forum.level1techs.com/t/how-to-sr-iov-mod-the-w7100-gpu/164186 patched]) | ||
|No | |No | ||
|No | |No | ||
|Yes | |Yes | ||
|- | |- | ||
Line 287: | Line 226: | ||
|Tonga | |Tonga | ||
|S71** | |S71** | ||
|AMDGPU, [https://github.com/GPUOpen-LibrariesAndSDKs/MxGPU-Virtualization MxGPU / GIM / GPU-IOV Module (Deprecated)] | |||
|AMDGPU, [https://github.com/GPUOpen-LibrariesAndSDKs/MxGPU-Virtualization GPU-IOV Module (Deprecated)] | |||
|AMDGPU-Pro | |AMDGPU-Pro | ||
|No | |No | ||
|[[wikipedia:End-of-life_product|EOL]] (2017) | |||
|No | |No | ||
|Yes | |Yes | ||
|No | |No | ||
|No | |No | ||
|Yes | |Yes | ||
|- | |- | ||
Line 303: | Line 239: | ||
|Vega | |Vega | ||
|* | |* | ||
|AMDGPU | |AMDGPU | ||
|AMDGPU-Pro | |AMDGPU-Pro | ||
|No | |No | ||
|[https://open-iov.org/index.php/GPU_Support#Drivers_3 Partial] | |||
|Unknown | |Unknown | ||
|Yes | |Yes | ||
|No | |No | ||
|No | |No | ||
|No | |No | ||
|- | |- | ||
|AMD | |AMD | ||
|Navi | |Navi | ||
|* | |* | ||
|AMDGPU | |AMDGPU | ||
|AMDGPU-Pro | |AMDGPU-Pro | ||
|No | |No | ||
|[https://open-iov.org/index.php/GPU_Support#Drivers_3 Partial] | |||
|Unknown | |Unknown | ||
|Yes | |Yes | ||
|No | |No | ||
|No | |No | ||
|No | |No | ||
|} | |} | ||
Line 355: | Line 285: | ||
==== Drivers ==== | ==== Drivers ==== | ||
Drivers with GVT-g support: | Drivers with GVT-g support (6th - 9th gen): | ||
* '''[https://github.com/torvalds/linux/tree/master/drivers/gpu/drm/i915 i915 mainline]''' | * '''[https://github.com/torvalds/linux/tree/master/drivers/gpu/drm/i915 i915 mainline]''' | ||
Drivers with SR-IOV support: | Drivers with SR-IOV support (11th gen+): | ||
* '''[https://github.com/intel/linux-intel-lts/commit/41ef979f0894326c202473807a56b599a2f3d04e i915 upstreaming]''' | * '''[https://github.com/intel/linux-intel-lts/commit/41ef979f0894326c202473807a56b599a2f3d04e i915 upstreaming]''' | ||
Line 419: | Line 349: | ||
==== Drivers ==== | ==== Drivers ==== | ||
The open source [https://github.com/GPUOpen-LibrariesAndSDKs/MxGPU-Virtualization GIM driver] supports AMD Tonga architecture GPUs such as the FirePro S7150 | The open source [https://github.com/GPUOpen-LibrariesAndSDKs/MxGPU-Virtualization GIM driver] supports AMD Tonga architecture GPUs such as the FirePro S7150 & W7100. | ||
To make use of the MxGPU GIM driver on modern kernel versions you will require [https://github.com/GPUOpen-LibrariesAndSDKs/MxGPU-Virtualization/pull/24 pull request 24]. | |||
Virtualization driver support beyond AMD's Tonga architecture is unclear. |
Latest revision as of 21:54, 13 March 2023
Abstract
The following document will detail GPU Support for SR-IOV, SIOV, and VFIO-Mdev functionality.
An absence of critical technical documentation has historically slowed growth and adoption of developer ecosystems for GPU virtualization.
This CC-BY-4.0 licensed content can either be used with attribution, or used as inspiration for new documentation, created by GPU vendors for public commercial distribution as developer documentation.
Where possible, this documentation will clearly label dates and versions of observed-but-not-guaranteed behaviour vs. vendor-documented stable interfaces/behaviour with guarantees of forward or backward compatibility.
Overview
Hardware | Drivers | Virtual I/O Assistance Modes | Userspace | ||||||||
---|---|---|---|---|---|---|---|---|---|---|---|
Vendor | Architecture | Model | Libre Drivers | Blob Drivers | 'Merged' Host+Guest Acceleration | Manufacturer Support | Mdev Support | SR-IOV Support | SIOV Support | GVM-cli Support | LibVF.IO Support |
Intel | 6th Generation (Skylake) | *HD | i915 | Yes | Yes (Legacy) | Yes | No | No | No | Yes | |
Intel | 7th Generation (Kaby Lake) | *HD | i915 | Yes | Yes (Legacy) | Yes | No | No | No | Yes | |
Intel | 8th Generation (Coffee Lake) | *HD | i915 | Yes | Yes (Legacy) | Yes | No | No | No | Yes | |
Intel | 9th Generation (Cannon Lake) | *HD | i915 | Yes | Yes (Legacy) | Yes | No | No | No | Yes | |
Intel | 10th Generation (Ice Lake) | i915 | No | Yes (Legacy) | No | No | No | No | No | ||
Intel | 11th Generation (Tiger Lake) | Xe* | i915 | Yes | Yes | Yes | Yes | Yes | |||
Intel | 12th Generation (Alder Lake) | Xe* | i915 | Yes | Yes | Yes | In Development | Yes | Yes | ||
Nvidia | Maxell | *(see exceptions below) | Nouveau, OpenRM | RM | Yes | Yes | Yes | No | No | Yes | Yes |
Nvidia | Pascal | * | Nouveau, OpenRM | RM | Yes | Yes | Yes | No | No | Yes | Yes |
Nvidia | Turing | * | Nouveau, OpenRM | RM | Yes | Yes | Yes | No | No | Yes | Yes |
Nvidia | Ampere | A* | Nouveau (Development), OpenRM | RM | No (DisplayModeSelector) | Yes | Yes | Yes | In Development | Yes | Yes |
Nvidia | Ampere | RTX 30** | Nouveau (Development), OpenRM | RM | In Development | Yes | Yes | No | In Development | In Development | In Development |
Nvidia | Ada Lovelace | * | OpenRM | Yes | Yes | Yes | In Development | In Development | In Development | ||
Nvidia | Hopper | * | OpenRM | Yes | Yes | Yes | In Development | ||||
AMD | Tonga | W7100 | AMDGPU, MxGPU / GIM / GPU-IOV Module (Deprecated) | AMDGPU-Pro | No | EOL (2017) | No | Yes
(patched) |
No | No | Yes |
AMD | Tonga | S71** | AMDGPU, MxGPU / GIM / GPU-IOV Module (Deprecated) | AMDGPU-Pro | No | EOL (2017) | No | Yes | No | No | Yes |
AMD | Vega | * | AMDGPU | AMDGPU-Pro | No | Partial | Unknown | Yes | No | No | No |
AMD | Navi | * | AMDGPU | AMDGPU-Pro | No | Partial | Unknown | Yes | No | No | No |
Intel
This section will cover Mdev support on Intel GPUs.
Hardware
GPUs supporting GVT-g (software based mediation)[1]:
- Intel HD Graphics 5500
- Intel HD Graphics 6000
- Intel HD Graphics 510
- Intel HD Graphics 520
- Intel HD Graphics 530
- Intel HD Graphics 620
- Intel HD Graphics 630
- Intel UHD Graphics 620
- Intel UHD Graphics 630
GPUs supporting SR-IOV (hardware based mediation):
Drivers
Drivers with GVT-g support (6th - 9th gen):
Drivers with SR-IOV support (11th gen+):
Nvidia
This section will cover Mdev support on Nvidia GPUs.
Hardware
- All(?) Volta based GPUs (V-series)
- All(?) Turing based GPUs (20-series)
- All(?) Pascal based GPUs (10-series)
- Most Maxwell based GPUs (9-series) with the following notable exceptions:
- GTX 970 (mixed FB regions)
- Quadro M2000 (mixed FB regions)
- Tesla M40 [4]
- NVIDIA A100 HGX 80GB
- NVIDIA A100 PCIe 80GB (SR-IOV: 20VF [5])
- NVIDIA A100X
- NVIDIA A100 HGX 40GB
- NVIDIA A100 PCIe 40GB (SR-IOV: 16VF [6])
- NVIDIA A40 (SR-IOV: 32VF [7])
- NVIDIA A30 (SR-IOV: 8VF [8])
- NVIDIA A30X
- NVIDIA A16 (SR-IOV: 16VF [9])
- NVIDIA A10 (SR-IOV: 32VF [10])
- NVIDIA A2 (SR-IOV: 16VF [11])
- NVIDIA RTX A6000
- NVIDIA RTX A5000
- Quadro RTX 8000 (SR-IOV: 24 VF [12])
- Quadro RTX 8000 passive
- Quadro RTX 6000 (SR-IOV: 24 VF [13])
- Quadro RTX 6000 passive
- Tesla V100
- Tesla T4 (SR-IOV: 16VF [14])
- Tesla P100
- Tesla P40
- Tesla P6
- Tesla P4
- Tesla M60
- Tesla M10
- Tesla M6
- GRID K2 (Citrix XenServer and VMware ESXi only)
- GRID K1 (Citrix XenServer and VMware ESXi only)
Drivers
The most widely used method of enabling Mdev functionality on Nvidia GPUs is via the use of Nvidia's proprietary driver package.
Nvidia's proprietary Mdev driver supports both SR-IOV as well as software based mediation. The appropriate mode is chosen between the two methods based on hardware architecture.
AMD
This section will cover Mdev support on AMD GPUs.
Hardware
Supported GPUs:
Drivers
The open source GIM driver supports AMD Tonga architecture GPUs such as the FirePro S7150 & W7100.
To make use of the MxGPU GIM driver on modern kernel versions you will require pull request 24.
Virtualization driver support beyond AMD's Tonga architecture is unclear.