| 1 |
========================== |
| 2 |
Remote Processor Framework |
| 3 |
========================== |
| 4 |
|
| 5 |
Introduction |
| 6 |
============ |
| 7 |
|
| 8 |
Modern SoCs typically have heterogeneous remote processor devices in asymmetric |
| 9 |
multiprocessing (AMP) configurations, which may be running different instances |
| 10 |
of operating system, whether it's Linux or any other flavor of real-time OS. |
| 11 |
|
| 12 |
OMAP4, for example, has dual Cortex-A9, dual Cortex-M3 and a C64x+ DSP. |
| 13 |
In a typical configuration, the dual cortex-A9 is running Linux in a SMP |
| 14 |
configuration, and each of the other three cores (two M3 cores and a DSP) |
| 15 |
is running its own instance of RTOS in an AMP configuration. |
| 16 |
|
| 17 |
The remoteproc framework allows different platforms/architectures to |
| 18 |
control (power on, load firmware, power off) those remote processors while |
| 19 |
abstracting the hardware differences, so the entire driver doesn't need to be |
| 20 |
duplicated. In addition, this framework also adds rpmsg virtio devices |
| 21 |
for remote processors that supports this kind of communication. This way, |
| 22 |
platform-specific remoteproc drivers only need to provide a few low-level |
| 23 |
handlers, and then all rpmsg drivers will then just work |
| 24 |
(for more information about the virtio-based rpmsg bus and its drivers, |
| 25 |
please read Documentation/staging/rpmsg.rst). |
| 26 |
Registration of other types of virtio devices is now also possible. Firmwares |
| 27 |
just need to publish what kind of virtio devices do they support, and then |
| 28 |
remoteproc will add those devices. This makes it possible to reuse the |
| 29 |
existing virtio drivers with remote processor backends at a minimal development |
| 30 |
cost. |
| 31 |
|
| 32 |
User API |
| 33 |
======== |
| 34 |
|
| 35 |
:: |
| 36 |
|
| 37 |
int rproc_boot(struct rproc *rproc) |
| 38 |
|
| 39 |
Boot a remote processor (i.e. load its firmware, power it on, ...). |
| 40 |
|
| 41 |
If the remote processor is already powered on, this function immediately |
| 42 |
returns (successfully). |
| 43 |
|
| 44 |
Returns 0 on success, and an appropriate error value otherwise. |
| 45 |
Note: to use this function you should already have a valid rproc |
| 46 |
handle. There are several ways to achieve that cleanly (devres, pdata, |
| 47 |
the way remoteproc_rpmsg.c does this, or, if this becomes prevalent, we |
| 48 |
might also consider using dev_archdata for this). |
| 49 |
|
| 50 |
:: |
| 51 |
|
| 52 |
int rproc_shutdown(struct rproc *rproc) |
| 53 |
|
| 54 |
Power off a remote processor (previously booted with rproc_boot()). |
| 55 |
In case @rproc is still being used by an additional user(s), then |
| 56 |
this function will just decrement the power refcount and exit, |
| 57 |
without really powering off the device. |
| 58 |
|
| 59 |
Returns 0 on success, and an appropriate error value otherwise. |
| 60 |
Every call to rproc_boot() must (eventually) be accompanied by a call |
| 61 |
to rproc_shutdown(). Calling rproc_shutdown() redundantly is a bug. |
| 62 |
|
| 63 |
.. note:: |
| 64 |
|
| 65 |
we're not decrementing the rproc's refcount, only the power refcount. |
| 66 |
which means that the @rproc handle stays valid even after |
| 67 |
rproc_shutdown() returns, and users can still use it with a subsequent |
| 68 |
rproc_boot(), if needed. |
| 69 |
|
| 70 |
:: |
| 71 |
|
| 72 |
struct rproc *rproc_get_by_phandle(phandle phandle) |
| 73 |
|
| 74 |
Find an rproc handle using a device tree phandle. Returns the rproc |
| 75 |
handle on success, and NULL on failure. This function increments |
| 76 |
the remote processor's refcount, so always use rproc_put() to |
| 77 |
decrement it back once rproc isn't needed anymore. |
| 78 |
|
| 79 |
Typical usage |
| 80 |
============= |
| 81 |
|
| 82 |
:: |
| 83 |
|
| 84 |
#include <linux/remoteproc.h> |
| 85 |
|
| 86 |
/* in case we were given a valid 'rproc' handle */ |
| 87 |
int dummy_rproc_example(struct rproc *my_rproc) |
| 88 |
{ |
| 89 |
int ret; |
| 90 |
|
| 91 |
/* let's power on and boot our remote processor */ |
| 92 |
ret = rproc_boot(my_rproc); |
| 93 |
if (ret) { |
| 94 |
/* |
| 95 |
* something went wrong. handle it and leave. |
| 96 |
*/ |
| 97 |
} |
| 98 |
|
| 99 |
/* |
| 100 |
* our remote processor is now powered on... give it some work |
| 101 |
*/ |
| 102 |
|
| 103 |
/* let's shut it down now */ |
| 104 |
rproc_shutdown(my_rproc); |
| 105 |
} |
| 106 |
|
| 107 |
API for implementers |
| 108 |
==================== |
| 109 |
|
| 110 |
:: |
| 111 |
|
| 112 |
struct rproc *rproc_alloc(struct device *dev, const char *name, |
| 113 |
const struct rproc_ops *ops, |
| 114 |
const char *firmware, int len) |
| 115 |
|
| 116 |
Allocate a new remote processor handle, but don't register |
| 117 |
it yet. Required parameters are the underlying device, the |
| 118 |
name of this remote processor, platform-specific ops handlers, |
| 119 |
the name of the firmware to boot this rproc with, and the |
| 120 |
length of private data needed by the allocating rproc driver (in bytes). |
| 121 |
|
| 122 |
This function should be used by rproc implementations during |
| 123 |
initialization of the remote processor. |
| 124 |
|
| 125 |
After creating an rproc handle using this function, and when ready, |
| 126 |
implementations should then call rproc_add() to complete |
| 127 |
the registration of the remote processor. |
| 128 |
|
| 129 |
On success, the new rproc is returned, and on failure, NULL. |
| 130 |
|
| 131 |
.. note:: |
| 132 |
|
| 133 |
**never** directly deallocate @rproc, even if it was not registered |
| 134 |
yet. Instead, when you need to unroll rproc_alloc(), use rproc_free(). |
| 135 |
|
| 136 |
:: |
| 137 |
|
| 138 |
void rproc_free(struct rproc *rproc) |
| 139 |
|
| 140 |
Free an rproc handle that was allocated by rproc_alloc. |
| 141 |
|
| 142 |
This function essentially unrolls rproc_alloc(), by decrementing the |
| 143 |
rproc's refcount. It doesn't directly free rproc; that would happen |
| 144 |
only if there are no other references to rproc and its refcount now |
| 145 |
dropped to zero. |
| 146 |
|
| 147 |
:: |
| 148 |
|
| 149 |
int rproc_add(struct rproc *rproc) |
| 150 |
|
| 151 |
Register @rproc with the remoteproc framework, after it has been |
| 152 |
allocated with rproc_alloc(). |
| 153 |
|
| 154 |
This is called by the platform-specific rproc implementation, whenever |
| 155 |
a new remote processor device is probed. |
| 156 |
|
| 157 |
Returns 0 on success and an appropriate error code otherwise. |
| 158 |
Note: this function initiates an asynchronous firmware loading |
| 159 |
context, which will look for virtio devices supported by the rproc's |
| 160 |
firmware. |
| 161 |
|
| 162 |
If found, those virtio devices will be created and added, so as a result |
| 163 |
of registering this remote processor, additional virtio drivers might get |
| 164 |
probed. |
| 165 |
|
| 166 |
:: |
| 167 |
|
| 168 |
int rproc_del(struct rproc *rproc) |
| 169 |
|
| 170 |
Unroll rproc_add(). |
| 171 |
|
| 172 |
This function should be called when the platform specific rproc |
| 173 |
implementation decides to remove the rproc device. it should |
| 174 |
_only_ be called if a previous invocation of rproc_add() |
| 175 |
has completed successfully. |
| 176 |
|
| 177 |
After rproc_del() returns, @rproc is still valid, and its |
| 178 |
last refcount should be decremented by calling rproc_free(). |
| 179 |
|
| 180 |
Returns 0 on success and -EINVAL if @rproc isn't valid. |
| 181 |
|
| 182 |
:: |
| 183 |
|
| 184 |
void rproc_report_crash(struct rproc *rproc, enum rproc_crash_type type) |
| 185 |
|
| 186 |
Report a crash in a remoteproc |
| 187 |
|
| 188 |
This function must be called every time a crash is detected by the |
| 189 |
platform specific rproc implementation. This should not be called from a |
| 190 |
non-remoteproc driver. This function can be called from atomic/interrupt |
| 191 |
context. |
| 192 |
|
| 193 |
Implementation callbacks |
| 194 |
======================== |
| 195 |
|
| 196 |
These callbacks should be provided by platform-specific remoteproc |
| 197 |
drivers:: |
| 198 |
|
| 199 |
/** |
| 200 |
* struct rproc_ops - platform-specific device handlers |
| 201 |
* @start: power on the device and boot it |
| 202 |
* @stop: power off the device |
| 203 |
* @kick: kick a virtqueue (virtqueue id given as a parameter) |
| 204 |
*/ |
| 205 |
struct rproc_ops { |
| 206 |
int (*start)(struct rproc *rproc); |
| 207 |
int (*stop)(struct rproc *rproc); |
| 208 |
void (*kick)(struct rproc *rproc, int vqid); |
| 209 |
}; |
| 210 |
|
| 211 |
Every remoteproc implementation should at least provide the ->start and ->stop |
| 212 |
handlers. If rpmsg/virtio functionality is also desired, then the ->kick handler |
| 213 |
should be provided as well. |
| 214 |
|
| 215 |
The ->start() handler takes an rproc handle and should then power on the |
| 216 |
device and boot it (use rproc->priv to access platform-specific private data). |
| 217 |
The boot address, in case needed, can be found in rproc->bootaddr (remoteproc |
| 218 |
core puts there the ELF entry point). |
| 219 |
On success, 0 should be returned, and on failure, an appropriate error code. |
| 220 |
|
| 221 |
The ->stop() handler takes an rproc handle and powers the device down. |
| 222 |
On success, 0 is returned, and on failure, an appropriate error code. |
| 223 |
|
| 224 |
The ->kick() handler takes an rproc handle, and an index of a virtqueue |
| 225 |
where new message was placed in. Implementations should interrupt the remote |
| 226 |
processor and let it know it has pending messages. Notifying remote processors |
| 227 |
the exact virtqueue index to look in is optional: it is easy (and not |
| 228 |
too expensive) to go through the existing virtqueues and look for new buffers |
| 229 |
in the used rings. |
| 230 |
|
| 231 |
Binary Firmware Structure |
| 232 |
========================= |
| 233 |
|
| 234 |
At this point remoteproc supports ELF32 and ELF64 firmware binaries. However, |
| 235 |
it is quite expected that other platforms/devices which we'd want to |
| 236 |
support with this framework will be based on different binary formats. |
| 237 |
|
| 238 |
When those use cases show up, we will have to decouple the binary format |
| 239 |
from the framework core, so we can support several binary formats without |
| 240 |
duplicating common code. |
| 241 |
|
| 242 |
When the firmware is parsed, its various segments are loaded to memory |
| 243 |
according to the specified device address (might be a physical address |
| 244 |
if the remote processor is accessing memory directly). |
| 245 |
|
| 246 |
In addition to the standard ELF segments, most remote processors would |
| 247 |
also include a special section which we call "the resource table". |
| 248 |
|
| 249 |
The resource table contains system resources that the remote processor |
| 250 |
requires before it should be powered on, such as allocation of physically |
| 251 |
contiguous memory, or iommu mapping of certain on-chip peripherals. |
| 252 |
Remotecore will only power up the device after all the resource table's |
| 253 |
requirement are met. |
| 254 |
|
| 255 |
In addition to system resources, the resource table may also contain |
| 256 |
resource entries that publish the existence of supported features |
| 257 |
or configurations by the remote processor, such as trace buffers and |
| 258 |
supported virtio devices (and their configurations). |
| 259 |
|
| 260 |
The resource table begins with this header:: |
| 261 |
|
| 262 |
/** |
| 263 |
* struct resource_table - firmware resource table header |
| 264 |
* @ver: version number |
| 265 |
* @num: number of resource entries |
| 266 |
* @reserved: reserved (must be zero) |
| 267 |
* @offset: array of offsets pointing at the various resource entries |
| 268 |
* |
| 269 |
* The header of the resource table, as expressed by this structure, |
| 270 |
* contains a version number (should we need to change this format in the |
| 271 |
* future), the number of available resource entries, and their offsets |
| 272 |
* in the table. |
| 273 |
*/ |
| 274 |
struct resource_table { |
| 275 |
u32 ver; |
| 276 |
u32 num; |
| 277 |
u32 reserved[2]; |
| 278 |
u32 offset[0]; |
| 279 |
} __packed; |
| 280 |
|
| 281 |
Immediately following this header are the resource entries themselves, |
| 282 |
each of which begins with the following resource entry header:: |
| 283 |
|
| 284 |
/** |
| 285 |
* struct fw_rsc_hdr - firmware resource entry header |
| 286 |
* @type: resource type |
| 287 |
* @data: resource data |
| 288 |
* |
| 289 |
* Every resource entry begins with a 'struct fw_rsc_hdr' header providing |
| 290 |
* its @type. The content of the entry itself will immediately follow |
| 291 |
* this header, and it should be parsed according to the resource type. |
| 292 |
*/ |
| 293 |
struct fw_rsc_hdr { |
| 294 |
u32 type; |
| 295 |
u8 data[0]; |
| 296 |
} __packed; |
| 297 |
|
| 298 |
Some resources entries are mere announcements, where the host is informed |
| 299 |
of specific remoteproc configuration. Other entries require the host to |
| 300 |
do something (e.g. allocate a system resource). Sometimes a negotiation |
| 301 |
is expected, where the firmware requests a resource, and once allocated, |
| 302 |
the host should provide back its details (e.g. address of an allocated |
| 303 |
memory region). |
| 304 |
|
| 305 |
Here are the various resource types that are currently supported:: |
| 306 |
|
| 307 |
/** |
| 308 |
* enum fw_resource_type - types of resource entries |
| 309 |
* |
| 310 |
* @RSC_CARVEOUT: request for allocation of a physically contiguous |
| 311 |
* memory region. |
| 312 |
* @RSC_DEVMEM: request to iommu_map a memory-based peripheral. |
| 313 |
* @RSC_TRACE: announces the availability of a trace buffer into which |
| 314 |
* the remote processor will be writing logs. |
| 315 |
* @RSC_VDEV: declare support for a virtio device, and serve as its |
| 316 |
* virtio header. |
| 317 |
* @RSC_LAST: just keep this one at the end |
| 318 |
* @RSC_VENDOR_START: start of the vendor specific resource types range |
| 319 |
* @RSC_VENDOR_END: end of the vendor specific resource types range |
| 320 |
* |
| 321 |
* Please note that these values are used as indices to the rproc_handle_rsc |
| 322 |
* lookup table, so please keep them sane. Moreover, @RSC_LAST is used to |
| 323 |
* check the validity of an index before the lookup table is accessed, so |
| 324 |
* please update it as needed. |
| 325 |
*/ |
| 326 |
enum fw_resource_type { |
| 327 |
RSC_CARVEOUT = 0, |
| 328 |
RSC_DEVMEM = 1, |
| 329 |
RSC_TRACE = 2, |
| 330 |
RSC_VDEV = 3, |
| 331 |
RSC_LAST = 4, |
| 332 |
RSC_VENDOR_START = 128, |
| 333 |
RSC_VENDOR_END = 512, |
| 334 |
}; |
| 335 |
|
| 336 |
For more details regarding a specific resource type, please see its |
| 337 |
dedicated structure in include/linux/remoteproc.h. |
| 338 |
|
| 339 |
We also expect that platform-specific resource entries will show up |
| 340 |
at some point. When that happens, we could easily add a new RSC_PLATFORM |
| 341 |
type, and hand those resources to the platform-specific rproc driver to handle. |
| 342 |
|
| 343 |
Virtio and remoteproc |
| 344 |
===================== |
| 345 |
|
| 346 |
The firmware should provide remoteproc information about virtio devices |
| 347 |
that it supports, and their configurations: a RSC_VDEV resource entry |
| 348 |
should specify the virtio device id (as in virtio_ids.h), virtio features, |
| 349 |
virtio config space, vrings information, etc. |
| 350 |
|
| 351 |
When a new remote processor is registered, the remoteproc framework |
| 352 |
will look for its resource table and will register the virtio devices |
| 353 |
it supports. A firmware may support any number of virtio devices, and |
| 354 |
of any type (a single remote processor can also easily support several |
| 355 |
rpmsg virtio devices this way, if desired). |
| 356 |
|
| 357 |
Of course, RSC_VDEV resource entries are only good enough for static |
| 358 |
allocation of virtio devices. Dynamic allocations will also be made possible |
| 359 |
using the rpmsg bus (similar to how we already do dynamic allocations of |
| 360 |
rpmsg channels; read more about it in rpmsg.txt). |