| 1 |
.. SPDX-License-Identifier: GPL-2.0 |
| 2 |
|
| 3 |
================== |
| 4 |
Intel IPU6 Driver |
| 5 |
================== |
| 6 |
|
| 7 |
Author: Bingbu Cao <[email protected]> |
| 8 |
|
| 9 |
Overview |
| 10 |
========= |
| 11 |
|
| 12 |
Intel IPU6 is the sixth generation of Intel Image Processing Unit used in some |
| 13 |
Intel Chipsets such as Tiger Lake, Jasper Lake, Alder Lake, Raptor Lake and |
| 14 |
Meteor Lake. IPU6 consists of two major systems: Input System (ISYS) and |
| 15 |
Processing System (PSYS). IPU6 are visible on the PCI bus as a single device, it |
| 16 |
can be found by ``lspci``: |
| 17 |
|
| 18 |
``0000:00:05.0 Multimedia controller: Intel Corporation Device xxxx (rev xx)`` |
| 19 |
|
| 20 |
IPU6 has a 16 MB BAR in PCI configuration Space for MMIO registers which is |
| 21 |
visible for driver. |
| 22 |
|
| 23 |
Buttress |
| 24 |
========= |
| 25 |
|
| 26 |
The IPU6 is connecting to the system fabric with Buttress which is enabling host |
| 27 |
driver to control the IPU6, it also allows IPU6 access the system memory to |
| 28 |
store and load frame pixel streams and any other metadata. |
| 29 |
|
| 30 |
Buttress mainly manages several system functionalities: power management, |
| 31 |
interrupt handling, firmware authentication and global timer sync. |
| 32 |
|
| 33 |
ISYS and PSYS Power flow |
| 34 |
------------------------ |
| 35 |
|
| 36 |
IPU6 driver initialize the ISYS and PSYS power up or down request by setting the |
| 37 |
Buttress frequency control register for ISYS and PSYS |
| 38 |
(``IPU6_BUTTRESS_REG_IS_FREQ_CTL`` and ``IPU6_BUTTRESS_REG_PS_FREQ_CTL``) in |
| 39 |
function: |
| 40 |
|
| 41 |
.. c:function:: int ipu6_buttress_power(...) |
| 42 |
|
| 43 |
Buttress forwards the request to Punit, after Punit execute the power up flow, |
| 44 |
Buttress indicates driver that ISYS or PSYS is powered up by updating the power |
| 45 |
status registers. |
| 46 |
|
| 47 |
.. Note:: ISYS power up needs take place prior to PSYS power up, ISYS power down |
| 48 |
needs take place after PSYS power down due to hardware limitation. |
| 49 |
|
| 50 |
Interrupt |
| 51 |
--------- |
| 52 |
|
| 53 |
IPU6 interrupt can be generated as MSI or INTA, interrupt will be triggered when |
| 54 |
ISYS, PSYS, Buttress event or error happen, driver can get the interrupt cause |
| 55 |
by reading the interrupt status register ``BUTTRESS_REG_ISR_STATUS``, driver |
| 56 |
clears the irq status and then calls specific ISYS or PSYS irq handler. |
| 57 |
|
| 58 |
.. c:function:: irqreturn_t ipu6_buttress_isr(int irq, ...) |
| 59 |
|
| 60 |
Security and firmware authentication |
| 61 |
------------------------------------- |
| 62 |
|
| 63 |
To address the IPU6 firmware security concerns, the IPU6 firmware needs to |
| 64 |
undergo an authentication process before it is allowed to executed on the IPU6 |
| 65 |
internal processors. The IPU6 driver will work with Converged Security Engine |
| 66 |
(CSE) to complete authentication process. The CSE is responsible of |
| 67 |
authenticating the IPU6 firmware. The authenticated firmware binary is copied |
| 68 |
into an isolated memory region. Firmware authentication process is implemented |
| 69 |
by CSE following an IPC handshake with the IPU6 driver. There are some Buttress |
| 70 |
registers used by the CSE and the IPU6 driver to communicate with each other via |
| 71 |
IPC. |
| 72 |
|
| 73 |
.. c:function:: int ipu6_buttress_authenticate(...) |
| 74 |
|
| 75 |
Global timer sync |
| 76 |
----------------- |
| 77 |
|
| 78 |
The IPU6 driver initiates a Hammock Harbor synchronization flow each time it |
| 79 |
starts camera operation. The IPU6 will synchronizes an internal counter in the |
| 80 |
Buttress with a copy of the SoC time, this counter maintains the up-to-date time |
| 81 |
until camera operation is stopped. The IPU6 driver can use this time counter to |
| 82 |
calibrate the timestamp based on the timestamp in response event from firmware. |
| 83 |
|
| 84 |
.. c:function:: int ipu6_buttress_start_tsc_sync(...) |
| 85 |
|
| 86 |
DMA and MMU |
| 87 |
============ |
| 88 |
|
| 89 |
The IPU6 has its own scalar processor where the firmware run at and an internal |
| 90 |
32-bit virtual address space. The IPU6 has MMU address translation hardware to |
| 91 |
allow that scalar processors to access the internal memory and external system |
| 92 |
memory through IPU6 virtual address. The address translation is based on two |
| 93 |
levels of page lookup tables stored in system memory which are maintained by the |
| 94 |
IPU6 driver. The IPU6 driver sets the level-1 page table base address to MMU |
| 95 |
register and allows MMU to perform page table lookups. |
| 96 |
|
| 97 |
The IPU6 driver exports its own DMA operations. The IPU6 driver will update the |
| 98 |
page table entries for each DMA operation and invalidate the MMU TLB after each |
| 99 |
unmap and free. |
| 100 |
|
| 101 |
Firmware file format |
| 102 |
==================== |
| 103 |
|
| 104 |
The IPU6 firmware is in Code Partition Directory (CPD) file format. The CPD |
| 105 |
firmware contains a CPD header, several CPD entries and components. The CPD |
| 106 |
component includes 3 entries - manifest, metadata and module data. Manifest and |
| 107 |
metadata are defined by CSE and used by CSE for authentication. Module data is |
| 108 |
specific to IPU6 which holds the binary data of firmware called package |
| 109 |
directory. The IPU6 driver (``ipu6-cpd.c`` in particular) parses and validates |
| 110 |
the CPD firmware file and gets the package directory binary data of the IPU6 |
| 111 |
firmware, copies it to specific DMA buffer and sets its base address to Buttress |
| 112 |
``FW_SOURCE_BASE`` register. Finally the CSE will do authentication for this |
| 113 |
firmware binary. |
| 114 |
|
| 115 |
|
| 116 |
Syscom interface |
| 117 |
================ |
| 118 |
|
| 119 |
The IPU6 driver communicates with firmware via the Syscom ABI. Syscom is an |
| 120 |
inter-processor communication mechanism between the IPU scalar processors and |
| 121 |
the CPU. There are a number of resources shared between firmware and software. |
| 122 |
A system memory region where the message queues reside, firmware can access the |
| 123 |
memory region via the IPU MMU. The Syscom queues are FIFO fixed depth queues |
| 124 |
with a configurable number of tokens (messages). There are also common IPU6 MMIO |
| 125 |
registers where the queue read and write indices reside. Software and firmware |
| 126 |
function as producer and consumer of tokens in the queues and update the write |
| 127 |
and read indices separately when sending or receiving each message. |
| 128 |
|
| 129 |
The IPU6 driver must prepare and configure the number of input and output |
| 130 |
queues, configure the count of tokens per queue and the size of per token before |
| 131 |
initiating and starting the communication with firmware. Firmware and software |
| 132 |
must use same configurations. The IPU6 Buttress has a number of firmware boot |
| 133 |
parameter registers which can be used to store the address of configuration and |
| 134 |
initialise the Syscom state, then driver can request firmware to start and run via |
| 135 |
setting the scalar processor control status register. |
| 136 |
|
| 137 |
Input System |
| 138 |
============ |
| 139 |
|
| 140 |
IPU6 input system consists of MIPI D-PHY and several CSI-2 receivers. It can |
| 141 |
capture image pixel data from camera sensors or other MIPI CSI-2 output devices. |
| 142 |
|
| 143 |
D-PHYs and CSI-2 ports lane mapping |
| 144 |
----------------------------------- |
| 145 |
|
| 146 |
The IPU6 integrates different D-PHY IPs on different SoCs, on Tiger Lake and |
| 147 |
Alder Lake, IPU6 integrates MCD10 D-PHY, IPU6SE on Jasper Lake integrates JSL |
| 148 |
D-PHY and IPU6EP on Meteor Lake integrates a Synopsys DWC D-PHY. There is an |
| 149 |
adaptional layer between D-PHY and CSI-2 receiver controller which includes port |
| 150 |
configuration, PHY wrapper or private test interfaces for D-PHY. There are 3 |
| 151 |
D-PHY drivers ``ipu6-isys-mcd-phy.c``, ``ipu6-isys-jsl-phy.c`` and |
| 152 |
``ipu6-isys-dwc-phy.c`` program the above 3 D-PHYs in IPU6. |
| 153 |
|
| 154 |
Different IPU6 versions have different D-PHY lanes mappings, On Tiger Lake, |
| 155 |
there are 12 data lanes and 8 clock lanes, IPU6 support maximum 8 CSI-2 ports, |
| 156 |
see the PPI mmapping in ``ipu6-isys-mcd-phy.c`` for more information. On Jasper |
| 157 |
Lake and Alder Lake, D-PHY has 8 data lanes and 4 clock lanes, the IPU6 supports |
| 158 |
maximum 4 CSI-2 ports. For Meteor Lake, D-PHY has 12 data lanes and 6 clock |
| 159 |
lanes so IPU6 support maximum 6 CSI-2 ports. |
| 160 |
|
| 161 |
.. Note:: Each pair of CSI-2 two ports is a single unit that can share the data |
| 162 |
lanes. For example, for CSI-2 port 0 and 1, CSI-2 port 0 support |
| 163 |
maximum 4 data lanes, CSI-2 port 1 support maximum 2 data lanes, CSI-2 |
| 164 |
port 0 with 2 data lanes can work together with CSI-2 port 1 with 2 |
| 165 |
data lanes. If trying to use CSI-2 port 0 with 4 lanes, CSI-2 port 1 |
| 166 |
will not be available as the 4 data lanes are shared by CSI-2 port 0 |
| 167 |
and 1. The same applies to CSI ports 2/3, 4/5 and 7/8. |
| 168 |
|
| 169 |
ISYS firmware ABIs |
| 170 |
------------------ |
| 171 |
|
| 172 |
The IPU6 firmware implements a series of ABIs for software access. In general, |
| 173 |
software firstly prepares the stream configuration ``struct |
| 174 |
ipu6_fw_isys_stream_cfg_data_abi`` and sends the configuration to firmware via |
| 175 |
sending ``STREAM_OPEN`` command. Stream configuration includes input pins and |
| 176 |
output pins, input pin ``struct ipu6_fw_isys_input_pin_info_abi`` defines the |
| 177 |
resolution and data type of input source, output pin ``struct |
| 178 |
ipu6_fw_isys_output_pin_info_abi`` defines the output resolution, stride and |
| 179 |
frame format, etc. |
| 180 |
|
| 181 |
Once the driver gets the interrupt from firmware that indicates stream open |
| 182 |
successfully, the driver will send the ``STREAM_START`` and ``STREAM_CAPTURE`` |
| 183 |
command to request firmware to start capturing image frames. ``STREAM_CAPTURE`` |
| 184 |
command queues the buffers to firmware with ``struct |
| 185 |
ipu6_fw_isys_frame_buff_set``, software then waits for the interrupt and |
| 186 |
response from firmware, ``PIN_DATA_READY`` means a buffer is ready on a specific |
| 187 |
output pin and then software can return the buffer to user. |
| 188 |
|
| 189 |
.. Note:: See :ref:`Examples<ipu6_isys_capture_examples>` about how to do |
| 190 |
capture by IPU6 ISYS driver. |