| 1 |
.. SPDX-License-Identifier: GPL-2.0 |
| 2 |
|
| 3 |
:Author: Kishon Vijay Abraham I <[email protected]> |
| 4 |
|
| 5 |
This document is a guide to use the PCI Endpoint Framework in order to create |
| 6 |
endpoint controller driver, endpoint function driver, and using configfs |
| 7 |
interface to bind the function driver to the controller driver. |
| 8 |
|
| 9 |
Introduction |
| 10 |
============ |
| 11 |
|
| 12 |
Linux has a comprehensive PCI subsystem to support PCI controllers that |
| 13 |
operates in Root Complex mode. The subsystem has capability to scan PCI bus, |
| 14 |
assign memory resources and IRQ resources, load PCI driver (based on |
| 15 |
vendor ID, device ID), support other services like hot-plug, power management, |
| 16 |
advanced error reporting and virtual channels. |
| 17 |
|
| 18 |
However the PCI controller IP integrated in some SoCs is capable of operating |
| 19 |
either in Root Complex mode or Endpoint mode. PCI Endpoint Framework will |
| 20 |
add endpoint mode support in Linux. This will help to run Linux in an |
| 21 |
EP system which can have a wide variety of use cases from testing or |
| 22 |
validation, co-processor accelerator, etc. |
| 23 |
|
| 24 |
PCI Endpoint Core |
| 25 |
================= |
| 26 |
|
| 27 |
The PCI Endpoint Core layer comprises 3 components: the Endpoint Controller |
| 28 |
library, the Endpoint Function library, and the configfs layer to bind the |
| 29 |
endpoint function with the endpoint controller. |
| 30 |
|
| 31 |
PCI Endpoint Controller(EPC) Library |
| 32 |
------------------------------------ |
| 33 |
|
| 34 |
The EPC library provides APIs to be used by the controller that can operate |
| 35 |
in endpoint mode. It also provides APIs to be used by function driver/library |
| 36 |
in order to implement a particular endpoint function. |
| 37 |
|
| 38 |
APIs for the PCI controller Driver |
| 39 |
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| 40 |
|
| 41 |
This section lists the APIs that the PCI Endpoint core provides to be used |
| 42 |
by the PCI controller driver. |
| 43 |
|
| 44 |
* devm_pci_epc_create()/pci_epc_create() |
| 45 |
|
| 46 |
The PCI controller driver should implement the following ops: |
| 47 |
|
| 48 |
* write_header: ops to populate configuration space header |
| 49 |
* set_bar: ops to configure the BAR |
| 50 |
* clear_bar: ops to reset the BAR |
| 51 |
* alloc_addr_space: ops to allocate in PCI controller address space |
| 52 |
* free_addr_space: ops to free the allocated address space |
| 53 |
* raise_irq: ops to raise a legacy, MSI or MSI-X interrupt |
| 54 |
* start: ops to start the PCI link |
| 55 |
* stop: ops to stop the PCI link |
| 56 |
|
| 57 |
The PCI controller driver can then create a new EPC device by invoking |
| 58 |
devm_pci_epc_create()/pci_epc_create(). |
| 59 |
|
| 60 |
* pci_epc_destroy() |
| 61 |
|
| 62 |
The PCI controller driver can destroy the EPC device created by |
| 63 |
pci_epc_create() using pci_epc_destroy(). |
| 64 |
|
| 65 |
* pci_epc_linkup() |
| 66 |
|
| 67 |
In order to notify all the function devices that the EPC device to which |
| 68 |
they are linked has established a link with the host, the PCI controller |
| 69 |
driver should invoke pci_epc_linkup(). |
| 70 |
|
| 71 |
* pci_epc_mem_init() |
| 72 |
|
| 73 |
Initialize the pci_epc_mem structure used for allocating EPC addr space. |
| 74 |
|
| 75 |
* pci_epc_mem_exit() |
| 76 |
|
| 77 |
Cleanup the pci_epc_mem structure allocated during pci_epc_mem_init(). |
| 78 |
|
| 79 |
|
| 80 |
EPC APIs for the PCI Endpoint Function Driver |
| 81 |
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| 82 |
|
| 83 |
This section lists the APIs that the PCI Endpoint core provides to be used |
| 84 |
by the PCI endpoint function driver. |
| 85 |
|
| 86 |
* pci_epc_write_header() |
| 87 |
|
| 88 |
The PCI endpoint function driver should use pci_epc_write_header() to |
| 89 |
write the standard configuration header to the endpoint controller. |
| 90 |
|
| 91 |
* pci_epc_set_bar() |
| 92 |
|
| 93 |
The PCI endpoint function driver should use pci_epc_set_bar() to configure |
| 94 |
the Base Address Register in order for the host to assign PCI addr space. |
| 95 |
Register space of the function driver is usually configured |
| 96 |
using this API. |
| 97 |
|
| 98 |
Some endpoint controllers also support calling pci_epc_set_bar() again |
| 99 |
for the same BAR (without calling pci_epc_clear_bar()) to update inbound |
| 100 |
address translations after the host has programmed the BAR base address. |
| 101 |
Endpoint function drivers can check this capability via the |
| 102 |
dynamic_inbound_mapping EPC feature bit. |
| 103 |
|
| 104 |
When pci_epf_bar.num_submap is non-zero, the endpoint function driver is |
| 105 |
requesting BAR subrange mapping using pci_epf_bar.submap. This requires |
| 106 |
the EPC to advertise support via the subrange_mapping EPC feature bit. |
| 107 |
|
| 108 |
When an EPF driver wants to make use of the inbound subrange mapping |
| 109 |
feature, it requires that the BAR base address has been programmed by |
| 110 |
the host during enumeration. Thus, it needs to call pci_epc_set_bar() |
| 111 |
twice for the same BAR (requires dynamic_inbound_mapping): first with |
| 112 |
num_submap set to zero and configuring the BAR size, then after the PCIe |
| 113 |
link is up and the host enumerates the endpoint and programs the BAR |
| 114 |
base address, again with num_submap set to non-zero value. |
| 115 |
|
| 116 |
Note that when making use of the inbound subrange mapping feature, the |
| 117 |
EPF driver must not call pci_epc_clear_bar() between the two |
| 118 |
pci_epc_set_bar() calls, because clearing the BAR can clear/disable the |
| 119 |
BAR register or BAR decode on the endpoint while the host still expects |
| 120 |
the assigned BAR address to remain valid. |
| 121 |
|
| 122 |
* pci_epc_clear_bar() |
| 123 |
|
| 124 |
The PCI endpoint function driver should use pci_epc_clear_bar() to reset |
| 125 |
the BAR. |
| 126 |
|
| 127 |
* pci_epc_raise_irq() |
| 128 |
|
| 129 |
The PCI endpoint function driver should use pci_epc_raise_irq() to raise |
| 130 |
Legacy Interrupt, MSI or MSI-X Interrupt. |
| 131 |
|
| 132 |
* pci_epc_mem_alloc_addr() |
| 133 |
|
| 134 |
The PCI endpoint function driver should use pci_epc_mem_alloc_addr(), to |
| 135 |
allocate memory address from EPC addr space which is required to access |
| 136 |
RC's buffer |
| 137 |
|
| 138 |
* pci_epc_mem_free_addr() |
| 139 |
|
| 140 |
The PCI endpoint function driver should use pci_epc_mem_free_addr() to |
| 141 |
free the memory space allocated using pci_epc_mem_alloc_addr(). |
| 142 |
|
| 143 |
* pci_epc_map_addr() |
| 144 |
|
| 145 |
A PCI endpoint function driver should use pci_epc_map_addr() to map to a RC |
| 146 |
PCI address the CPU address of local memory obtained with |
| 147 |
pci_epc_mem_alloc_addr(). |
| 148 |
|
| 149 |
* pci_epc_unmap_addr() |
| 150 |
|
| 151 |
A PCI endpoint function driver should use pci_epc_unmap_addr() to unmap the |
| 152 |
CPU address of local memory mapped to a RC address with pci_epc_map_addr(). |
| 153 |
|
| 154 |
* pci_epc_mem_map() |
| 155 |
|
| 156 |
A PCI endpoint controller may impose constraints on the RC PCI addresses that |
| 157 |
can be mapped. The function pci_epc_mem_map() allows endpoint function |
| 158 |
drivers to allocate and map controller memory while handling such |
| 159 |
constraints. This function will determine the size of the memory that must be |
| 160 |
allocated with pci_epc_mem_alloc_addr() for successfully mapping a RC PCI |
| 161 |
address range. This function will also indicate the size of the PCI address |
| 162 |
range that was actually mapped, which can be less than the requested size, as |
| 163 |
well as the offset into the allocated memory to use for accessing the mapped |
| 164 |
RC PCI address range. |
| 165 |
|
| 166 |
* pci_epc_mem_unmap() |
| 167 |
|
| 168 |
A PCI endpoint function driver can use pci_epc_mem_unmap() to unmap and free |
| 169 |
controller memory that was allocated and mapped using pci_epc_mem_map(). |
| 170 |
|
| 171 |
|
| 172 |
Other EPC APIs |
| 173 |
~~~~~~~~~~~~~~ |
| 174 |
|
| 175 |
There are other APIs provided by the EPC library. These are used for binding |
| 176 |
the EPF device with EPC device. pci-ep-cfs.c can be used as reference for |
| 177 |
using these APIs. |
| 178 |
|
| 179 |
* pci_epc_get() |
| 180 |
|
| 181 |
Get a reference to the PCI endpoint controller based on the device name of |
| 182 |
the controller. |
| 183 |
|
| 184 |
* pci_epc_put() |
| 185 |
|
| 186 |
Release the reference to the PCI endpoint controller obtained using |
| 187 |
pci_epc_get() |
| 188 |
|
| 189 |
* pci_epc_add_epf() |
| 190 |
|
| 191 |
Add a PCI endpoint function to a PCI endpoint controller. A PCIe device |
| 192 |
can have up to 8 functions according to the specification. |
| 193 |
|
| 194 |
* pci_epc_remove_epf() |
| 195 |
|
| 196 |
Remove the PCI endpoint function from PCI endpoint controller. |
| 197 |
|
| 198 |
* pci_epc_start() |
| 199 |
|
| 200 |
The PCI endpoint function driver should invoke pci_epc_start() once it |
| 201 |
has configured the endpoint function and wants to start the PCI link. |
| 202 |
|
| 203 |
* pci_epc_stop() |
| 204 |
|
| 205 |
The PCI endpoint function driver should invoke pci_epc_stop() to stop |
| 206 |
the PCI LINK. |
| 207 |
|
| 208 |
|
| 209 |
PCI Endpoint Function(EPF) Library |
| 210 |
---------------------------------- |
| 211 |
|
| 212 |
The EPF library provides APIs to be used by the function driver and the EPC |
| 213 |
library to provide endpoint mode functionality. |
| 214 |
|
| 215 |
EPF APIs for the PCI Endpoint Function Driver |
| 216 |
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| 217 |
|
| 218 |
This section lists the APIs that the PCI Endpoint core provides to be used |
| 219 |
by the PCI endpoint function driver. |
| 220 |
|
| 221 |
* pci_epf_register_driver() |
| 222 |
|
| 223 |
The PCI Endpoint Function driver should implement the following ops: |
| 224 |
* bind: ops to perform when an EPC device has been bound to EPF device |
| 225 |
* unbind: ops to perform when a binding has been lost between an EPC |
| 226 |
device and EPF device |
| 227 |
* add_cfs: optional ops to create function specific configfs |
| 228 |
attributes |
| 229 |
|
| 230 |
The PCI Function driver can then register the PCI EPF driver by using |
| 231 |
pci_epf_register_driver(). |
| 232 |
|
| 233 |
* pci_epf_unregister_driver() |
| 234 |
|
| 235 |
The PCI Function driver can unregister the PCI EPF driver by using |
| 236 |
pci_epf_unregister_driver(). |
| 237 |
|
| 238 |
* pci_epf_alloc_space() |
| 239 |
|
| 240 |
The PCI Function driver can allocate space for a particular BAR using |
| 241 |
pci_epf_alloc_space(). |
| 242 |
|
| 243 |
* pci_epf_free_space() |
| 244 |
|
| 245 |
The PCI Function driver can free the allocated space |
| 246 |
(using pci_epf_alloc_space) by invoking pci_epf_free_space(). |
| 247 |
|
| 248 |
APIs for the PCI Endpoint Controller Library |
| 249 |
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| 250 |
|
| 251 |
This section lists the APIs that the PCI Endpoint core provides to be used |
| 252 |
by the PCI endpoint controller library. |
| 253 |
|
| 254 |
* pci_epf_linkup() |
| 255 |
|
| 256 |
The PCI endpoint controller library invokes pci_epf_linkup() when the |
| 257 |
EPC device has established the connection to the host. |
| 258 |
|
| 259 |
Other EPF APIs |
| 260 |
~~~~~~~~~~~~~~ |
| 261 |
|
| 262 |
There are other APIs provided by the EPF library. These are used to notify |
| 263 |
the function driver when the EPF device is bound to the EPC device. |
| 264 |
pci-ep-cfs.c can be used as reference for using these APIs. |
| 265 |
|
| 266 |
* pci_epf_create() |
| 267 |
|
| 268 |
Create a new PCI EPF device by passing the name of the PCI EPF device. |
| 269 |
This name will be used to bind the EPF device to a EPF driver. |
| 270 |
|
| 271 |
* pci_epf_destroy() |
| 272 |
|
| 273 |
Destroy the created PCI EPF device. |
| 274 |
|
| 275 |
* pci_epf_bind() |
| 276 |
|
| 277 |
pci_epf_bind() should be invoked when the EPF device has been bound to |
| 278 |
an EPC device. |
| 279 |
|
| 280 |
* pci_epf_unbind() |
| 281 |
|
| 282 |
pci_epf_unbind() should be invoked when the binding between EPC device |
| 283 |
and EPF device is lost. |