Difference between revisions of "GPU Firmware"

From Open-IOV
Jump to navigation Jump to search
 
(14 intermediate revisions by the same user not shown)
Line 1: Line 1:


GPUs have become highly complex systems containing a number of different embedded controllers. This page will attempt to document embedded GPU firmware and support for IO virtualization through various firmware functions.
GPUs have become highly complex systems containing a number of different embedded controllers. This page will attempt to document embedded GPU firmware and support for IO virtualization through various firmware functions.<blockquote>An absence of critical technical documentation has historically slowed growth and adoption of developer ecosystems for GPU virtualization.
 
This [https://creativecommons.org/licenses/by/4.0/ CC-BY-4.0] licensed content can either be used with attribution, or used as inspiration for new documentation, created by GPU vendors for public commercial distribution as developer documentation.
 
Where possible, this documentation will clearly label dates and versions of observed-but-not-guaranteed behaviour vs. vendor-documented stable interfaces/behaviour with guarantees of forward or backward compatibility.</blockquote>


== Intel ==
== Intel ==


=== Firmware Images ===
=== Firmware Images ===
[[File:Screen Shot 2022-11-18 at 4.14.32 PM.png|alt=Figure 1: The FSP Binary Layout from Intel® Firmware Support Package External Architecture Specification.|thumb|Figure 1: The FSP Binary Layout from Intel® Firmware Support Package External Architecture Specification. [https://cdrdv2.intel.com/v1/dl/getContent/736809 Source]]]
This section will cover firmware images used in Intel GPUs.[[File:Screen Shot 2022-11-18 at 4.14.32 PM.png|alt=Figure 1: The FSP Binary Layout from Intel® Firmware Support Package External Architecture Specification.|thumb|Figure 1: The FSP Binary Layout from Intel® Firmware Support Package External Architecture Specification. [https://cdrdv2.intel.com/v1/dl/getContent/736809 Source]]]


==== Intel Firmware Support Package (FSP) ====
==== Intel Firmware Support Package (FSP) ====
Line 73: Line 77:
The FSP's binary layout is detailed within the [https://cdrdv2.intel.com/v1/dl/getContent/736809 Intel® FSP External Architecture Specification v2.4] on page 14.
The FSP's binary layout is detailed within the [https://cdrdv2.intel.com/v1/dl/getContent/736809 Intel® FSP External Architecture Specification v2.4] on page 14.


===== Known FSP Variations =====
==== Known Firmware Package Variations ====
Some firmware packages may include an **End of Manufacturing Flash Protection Mode** status of Protected or Unprotected.
[[File:Figure 2- Firmware status information for an Intel DG2 device..png|alt=Figure 2: Firmware status information for an Intel DG2 device.|thumb|Figure 2: Firmware status information for an Intel DG2 device.]]
Similar [https://eclypsium.com/2022/09/19/firmware-security-realizations-part-3-spi-write-protections/ SPI Write Protection] functionality is made available through Intel CPUs under [[wikipedia:System_Management_Mode|System Management Mode (SMM)]].


=== Embedded Controllers ===
=== Embedded Controllers ===
This section will cover firmware images as they apply to various embedded controllers within the Intel GPU.


==== GuC ====
==== GuC ====
The Graphics micro (µ) Controller (GuC) is an embedded controller contained within Intel's Discrete Graphics (DG*) series GPUs.
The Graphics micro (µ) Controller (GuC) is an embedded controller contained within Intel's embedded and discrete graphics (DG*) series GPUs.


===== Hardware Architecture =====
===== Hardware Architecture =====
According to igor-blue.github.io (see reference 1, 2):<blockquote>The GuC - an embedded i486 core that supports graphics scheduling, power management and firmware attestation.</blockquote>


''The following section is supported by [https://igor-blue.github.io/2021/02/10/graphics-part1.html igor-blue.github.io] (see reference [https://open-iov.org/index.php/GPU_Firmware#References_(Talks_&_Reading_Material) 1], [https://open-iov.org/index.php/GPU_Firmware#References_(Talks_&_Reading_Material) 2]):''
''"The GuC - an embedded [https://www.wikiwand.com/en/I486 i486] core that supports graphics scheduling, power management and firmware attestation."''
===== Software Architecture =====
===== Software Architecture =====
According to igor-blue.github.io (see reference 1, 2):<blockquote>The μOS kernel runs in 32-bit protected mode, with no paging and old-style segments model (CS, DS, etc’). All code run in ring0. The OS handles HW/SW exceptions and crashes, and supplies debugging and logging services.</blockquote><blockquote>It runs a single process - which initializes the system and then waits for interrupts/events in a loop.</blockquote><blockquote>The bootrom verifies the firmware with a digital signature using a SHA256 hash + PKCSv2.1 RSA signature, and if the test passes copies it to SRAM and starts executing.</blockquote>


''The following section is supported by [https://igor-blue.github.io/2021/02/10/graphics-part1.html igor-blue.github.io] (see reference [https://open-iov.org/index.php/GPU_Firmware#References_(Talks_&_Reading_Material) 1], [https://open-iov.org/index.php/GPU_Firmware#References_(Talks_&_Reading_Material) 2]):''
''"The μOS kernel runs in 32-bit protected mode, with no paging and old-style segments model (CS, DS, etc’). All code run in ring0. The OS handles HW/SW exceptions and crashes, and supplies debugging and logging services."''
''"It runs a single process - which initializes the system and then waits for interrupts/events in a loop."''
====== GuC Blob Checksum & Code Signing ======
''"The bootrom verifies the firmware with a digital signature using a SHA256 hash + PKCSv2.1 RSA signature, and if the test passes copies it to SRAM and starts executing."''
== Nvidia ==
== Nvidia ==


=== Firmware Images ===
=== Firmware Images ===
This section will cover firmware images used in Nvidia GPUs.


=== Embedded Controllers ===
=== Embedded Controllers ===
This section will cover firmware images as they apply to various embedded controllers within the Nvidia GPU.
==== Falcon / NV-RISCV ====
The Fast Logic CONtroller (Falcon) and Nvidia RISC-V ([https://riscv.org/wp-content/uploads/2016/07/Tue1100_Nvidia_RISCV_Story_V2.pdf NV-RISCV]) processors run the NvOS.
==== GSP ====
The GPU System Processor (GSP) is an embedded controller used for offload of the RM Core.
The GSP runs [https://lwn.net/Articles/637658/ Library Operating System (LibOS)].


==== Falcon ====
===== GSP Initialization & Offload =====
The [https://open-iov.org/index.php/GPU_Driver_Internals#Initialization_3 GSP is initialized multiple times] during the system's bring up and runtime.


==== NvRISC ====
GSP offload may occur during:


==== GSP ====
* Hardware bring up when a cached version of the RM Core is loaded from SPI flash
 
* During host driver bring up when the RM Core is offloaded by the [https://open-iov.org/index.php/OpenRM OpenRM driver].
* During guest driver bring up when the RM Core is offloaded.
 
{| class="wikitable"
|+Possible GSP Offloads
!Load Source
!Payload
!Notes
|-
|[https://wiki.segger.com/SPI_Flash SPI Flash]
|Cached RM Core
|Used as a fallback in case of no rm offload.
|-
|OpenRM
|RM Core
|This is the RM Core which was traditionally contained in the proprietary RM driver.
|-
|VGX Guest
|Guest RM Core
|Future OpenRM guests may accomplish RM offload via GSP stubs (controlled via [https://github.com/NVIDIA/open-gpu-kernel-modules/blob/758b4ee8189c5198504cb1c3c5bc29027a9118a3/src/common/sdk/nvidia/inc/ctrl/ctrla081.h#L102 gspHeapSize]?).
|}


==== CMU ====
==== CMU ====
Line 108: Line 158:
# [https://igor-blue.github.io/2021/02/10/graphics-part1.html Security of the Intel Graphics Stack - Part 1 - Introduction]
# [https://igor-blue.github.io/2021/02/10/graphics-part1.html Security of the Intel Graphics Stack - Part 1 - Introduction]
# [https://igor-blue.github.io/2021/02/24/graphics-part2.html Security of the Intel Graphics Stack - Part 2 - FW <-> GuC]
# [https://igor-blue.github.io/2021/02/24/graphics-part2.html Security of the Intel Graphics Stack - Part 2 - FW <-> GuC]
#[https://eclypsium.com/2022/09/19/firmware-security-realizations-part-3-spi-write-protections/ Firmware Security Realizations Part 3: SPI Write Protections]
#[https://www.intel.com/content/www/us/en/intelligent-systems/intel-firmware-support-package/fsp-firmware-solutions-iot-video.html Intel® FSP: Firmware Solutions for the Internet of Things]

Latest revision as of 17:13, 9 April 2023

GPUs have become highly complex systems containing a number of different embedded controllers. This page will attempt to document embedded GPU firmware and support for IO virtualization through various firmware functions.

An absence of critical technical documentation has historically slowed growth and adoption of developer ecosystems for GPU virtualization.

This CC-BY-4.0 licensed content can either be used with attribution, or used as inspiration for new documentation, created by GPU vendors for public commercial distribution as developer documentation.

Where possible, this documentation will clearly label dates and versions of observed-but-not-guaranteed behaviour vs. vendor-documented stable interfaces/behaviour with guarantees of forward or backward compatibility.

Intel

Firmware Images

This section will cover firmware images used in Intel GPUs.

Figure 1: The FSP Binary Layout from Intel® Firmware Support Package External Architecture Specification.
Figure 1: The FSP Binary Layout from Intel® Firmware Support Package External Architecture Specification. Source

Intel Firmware Support Package (FSP)

Much like CPUs Intel's GPUs also contain a Firmware Support Package (FSP).

The Coreboot project provides public domain information on the FSP here.

FSP Configuration

In the context of GPUs the FSP configures several functions of the device.

Those functions are as follows:

GPU Firmware Support Package
FSP Parameter Possible Values
GFSP Status 0x00
FIVR SSC Value *.*%
FIVR RFI Value *.*MHz
GT Subsystem Vendor ID 0x8086
GT Subsystem Device ID 0x**
HDA Subsystem Vendor ID 0x0000
HDA Subsystem Device ID 0x0000
P2SB Enable Yes/No
LMEBAR Max
GTMMADDR Prefetch Capability Prefetch Enabled
Display Present Enabled/Disabled
I2C For Third Party Devices Enabled/Disabled
I2C Device Address 1 0x0000
I2C Device Address 2 0x0000
I2C Bus Speed Standard mode (0 to 100Kbps)
Editing FSP Configuration

The FSP configuration editor can be downloaded here and it's user manual is available here.

FSP Binary Format

The FSP's binary layout is detailed within the Intel® FSP External Architecture Specification v2.4 on page 14.

Known Firmware Package Variations

Some firmware packages may include an **End of Manufacturing Flash Protection Mode** status of Protected or Unprotected.

Figure 2: Firmware status information for an Intel DG2 device.
Figure 2: Firmware status information for an Intel DG2 device.

Similar SPI Write Protection functionality is made available through Intel CPUs under System Management Mode (SMM).

Embedded Controllers

This section will cover firmware images as they apply to various embedded controllers within the Intel GPU.

GuC

The Graphics micro (µ) Controller (GuC) is an embedded controller contained within Intel's embedded and discrete graphics (DG*) series GPUs.

Hardware Architecture

The following section is supported by igor-blue.github.io (see reference 1, 2):

"The GuC - an embedded i486 core that supports graphics scheduling, power management and firmware attestation."

Software Architecture

The following section is supported by igor-blue.github.io (see reference 1, 2):

"The μOS kernel runs in 32-bit protected mode, with no paging and old-style segments model (CS, DS, etc’). All code run in ring0. The OS handles HW/SW exceptions and crashes, and supplies debugging and logging services."

"It runs a single process - which initializes the system and then waits for interrupts/events in a loop."

GuC Blob Checksum & Code Signing

"The bootrom verifies the firmware with a digital signature using a SHA256 hash + PKCSv2.1 RSA signature, and if the test passes copies it to SRAM and starts executing."

Nvidia

Firmware Images

This section will cover firmware images used in Nvidia GPUs.

Embedded Controllers

This section will cover firmware images as they apply to various embedded controllers within the Nvidia GPU.

Falcon / NV-RISCV

The Fast Logic CONtroller (Falcon) and Nvidia RISC-V (NV-RISCV) processors run the NvOS.

GSP

The GPU System Processor (GSP) is an embedded controller used for offload of the RM Core.

The GSP runs Library Operating System (LibOS).

GSP Initialization & Offload

The GSP is initialized multiple times during the system's bring up and runtime.

GSP offload may occur during:

  • Hardware bring up when a cached version of the RM Core is loaded from SPI flash
  • During host driver bring up when the RM Core is offloaded by the OpenRM driver.
  • During guest driver bring up when the RM Core is offloaded.
Possible GSP Offloads
Load Source Payload Notes
SPI Flash Cached RM Core Used as a fallback in case of no rm offload.
OpenRM RM Core This is the RM Core which was traditionally contained in the proprietary RM driver.
VGX Guest Guest RM Core Future OpenRM guests may accomplish RM offload via GSP stubs (controlled via gspHeapSize?).

CMU

AMD

References (Talks & Reading Material)

  1. Security of the Intel Graphics Stack - Part 1 - Introduction
  2. Security of the Intel Graphics Stack - Part 2 - FW <-> GuC
  3. Firmware Security Realizations Part 3: SPI Write Protections
  4. Intel® FSP: Firmware Solutions for the Internet of Things