| 1 |
.. SPDX-License-Identifier: GPL-2.0 |
| 2 |
|
| 3 |
.. include:: <isonum.txt> |
| 4 |
|
| 5 |
======================================================== |
| 6 |
Intel Image Processing Unit 6 (IPU6) Input System driver |
| 7 |
======================================================== |
| 8 |
|
| 9 |
Copyright |copy| 2023--2024 Intel Corporation |
| 10 |
|
| 11 |
Introduction |
| 12 |
============ |
| 13 |
|
| 14 |
This file documents the Intel IPU6 (6th generation Image Processing Unit) |
| 15 |
Input System (MIPI CSI2 receiver) drivers located under |
| 16 |
drivers/media/pci/intel/ipu6. |
| 17 |
|
| 18 |
The Intel IPU6 can be found in certain Intel SoCs but not in all SKUs: |
| 19 |
|
| 20 |
* Tiger Lake |
| 21 |
* Jasper Lake |
| 22 |
* Alder Lake |
| 23 |
* Raptor Lake |
| 24 |
* Meteor Lake |
| 25 |
|
| 26 |
Intel IPU6 is made up of two components - Input System (ISYS) and Processing |
| 27 |
System (PSYS). |
| 28 |
|
| 29 |
The Input System mainly works as MIPI CSI-2 receiver which receives and |
| 30 |
processes the image data from the sensors and outputs the frames to memory. |
| 31 |
|
| 32 |
There are 2 driver modules - intel-ipu6 and intel-ipu6-isys. intel-ipu6 is an |
| 33 |
IPU6 common driver which does PCI configuration, firmware loading and parsing, |
| 34 |
firmware authentication, DMA mapping and IPU-MMU (internal Memory mapping Unit) |
| 35 |
configuration. intel_ipu6_isys implements V4L2, Media Controller and V4L2 |
| 36 |
sub-device interfaces. The IPU6 ISYS driver supports camera sensors connected |
| 37 |
to the IPU6 ISYS through V4L2 sub-device sensor drivers. |
| 38 |
|
| 39 |
.. Note:: See Documentation/driver-api/media/drivers/ipu6.rst for more |
| 40 |
information about the IPU6 hardware. |
| 41 |
|
| 42 |
Input system driver |
| 43 |
=================== |
| 44 |
|
| 45 |
The Input System driver mainly configures CSI-2 D-PHY, constructs the firmware |
| 46 |
stream configuration, sends commands to firmware, gets response from hardware |
| 47 |
and firmware and then returns buffers to user. The ISYS is represented as |
| 48 |
several V4L2 sub-devices as well as video nodes. |
| 49 |
|
| 50 |
.. kernel-figure:: ipu6_isys_graph.svg |
| 51 |
:alt: ipu6 isys media graph with multiple streams support |
| 52 |
|
| 53 |
IPU6 ISYS media graph with multiple streams support |
| 54 |
|
| 55 |
The graph has been produced using the following command: |
| 56 |
|
| 57 |
.. code-block:: none |
| 58 |
|
| 59 |
fdp -Gsplines=true -Tsvg < dot > dot.svg |
| 60 |
|
| 61 |
Capturing frames with IPU6 ISYS |
| 62 |
------------------------------- |
| 63 |
|
| 64 |
IPU6 ISYS is used to capture frames from the camera sensors connected to the |
| 65 |
CSI2 ports. The supported input formats of ISYS are listed in table below: |
| 66 |
|
| 67 |
.. tabularcolumns:: |p{0.8cm}|p{4.0cm}|p{4.0cm}| |
| 68 |
|
| 69 |
.. flat-table:: |
| 70 |
:header-rows: 1 |
| 71 |
|
| 72 |
* - IPU6 ISYS supported input formats |
| 73 |
|
| 74 |
* - RGB565, RGB888 |
| 75 |
|
| 76 |
* - UYVY8, YUYV8 |
| 77 |
|
| 78 |
* - RAW8, RAW10, RAW12 |
| 79 |
|
| 80 |
.. _ipu6_isys_capture_examples: |
| 81 |
|
| 82 |
Examples |
| 83 |
~~~~~~~~ |
| 84 |
|
| 85 |
Here is an example of IPU6 ISYS raw capture on Dell XPS 9315 laptop. On this |
| 86 |
machine, ov01a10 sensor is connected to IPU ISYS CSI-2 port 2, which can |
| 87 |
generate images at sBGGR10 with resolution 1280x800. |
| 88 |
|
| 89 |
Using the media controller APIs, we can configure ov01a10 sensor by |
| 90 |
media-ctl [#f1]_ and yavta [#f2]_ to transmit frames to IPU6 ISYS. |
| 91 |
|
| 92 |
.. code-block:: none |
| 93 |
|
| 94 |
# Example 1 capture frame from ov01a10 camera sensor |
| 95 |
# This example assumes /dev/media0 as the IPU ISYS media device |
| 96 |
export MDEV=/dev/media0 |
| 97 |
|
| 98 |
# Establish the link for the media devices using media-ctl |
| 99 |
media-ctl -d $MDEV -l "\"ov01a10 3-0036\":0 -> \"Intel IPU6 CSI2 2\":0[1]" |
| 100 |
|
| 101 |
# Set the format for the media devices |
| 102 |
media-ctl -d $MDEV -V "ov01a10:0 [fmt:SBGGR10/1280x800]" |
| 103 |
media-ctl -d $MDEV -V "Intel IPU6 CSI2 2:0 [fmt:SBGGR10/1280x800]" |
| 104 |
media-ctl -d $MDEV -V "Intel IPU6 CSI2 2:1 [fmt:SBGGR10/1280x800]" |
| 105 |
|
| 106 |
Once the media pipeline is configured, desired sensor specific settings |
| 107 |
(such as exposure and gain settings) can be set, using the yavta tool. |
| 108 |
|
| 109 |
e.g |
| 110 |
|
| 111 |
.. code-block:: none |
| 112 |
|
| 113 |
# and that ov01a10 sensor is connected to i2c bus 3 with address 0x36 |
| 114 |
export SDEV=$(media-ctl -d $MDEV -e "ov01a10 3-0036") |
| 115 |
|
| 116 |
yavta -w 0x009e0903 400 $SDEV |
| 117 |
yavta -w 0x009e0913 1000 $SDEV |
| 118 |
yavta -w 0x009e0911 2000 $SDEV |
| 119 |
|
| 120 |
Once the desired sensor settings are set, frame captures can be done as below. |
| 121 |
|
| 122 |
e.g |
| 123 |
|
| 124 |
.. code-block:: none |
| 125 |
|
| 126 |
yavta --data-prefix -u -c10 -n5 -I -s 1280x800 --file=/tmp/frame-#.bin \ |
| 127 |
-f SBGGR10 $(media-ctl -d $MDEV -e "Intel IPU6 ISYS Capture 0") |
| 128 |
|
| 129 |
With the above command, 10 frames are captured at 1280x800 resolution with |
| 130 |
sBGGR10 format. The captured frames are available as /tmp/frame-#.bin files. |
| 131 |
|
| 132 |
Here is another example of IPU6 ISYS RAW and metadata capture from camera |
| 133 |
sensor ov2740 on Lenovo X1 Yoga laptop. |
| 134 |
|
| 135 |
.. code-block:: none |
| 136 |
|
| 137 |
media-ctl -l "\"ov2740 14-0036\":0 -> \"Intel IPU6 CSI2 1\":0[1]" |
| 138 |
media-ctl -l "\"Intel IPU6 CSI2 1\":1 -> \"Intel IPU6 ISYS Capture 0\":0[1]" |
| 139 |
media-ctl -l "\"Intel IPU6 CSI2 1\":2 -> \"Intel IPU6 ISYS Capture 1\":0[1]" |
| 140 |
|
| 141 |
# set routing |
| 142 |
media-ctl -R "\"Intel IPU6 CSI2 1\" [0/0->1/0[1],0/1->2/1[1]]" |
| 143 |
|
| 144 |
media-ctl -V "\"Intel IPU6 CSI2 1\":0/0 [fmt:SGRBG10/1932x1092]" |
| 145 |
media-ctl -V "\"Intel IPU6 CSI2 1\":0/1 [fmt:GENERIC_8/97x1]" |
| 146 |
media-ctl -V "\"Intel IPU6 CSI2 1\":1/0 [fmt:SGRBG10/1932x1092]" |
| 147 |
media-ctl -V "\"Intel IPU6 CSI2 1\":2/1 [fmt:GENERIC_8/97x1]" |
| 148 |
|
| 149 |
CAPTURE_DEV=$(media-ctl -e "Intel IPU6 ISYS Capture 0") |
| 150 |
./yavta --data-prefix -c100 -n5 -I -s1932x1092 --file=/tmp/frame-#.bin \ |
| 151 |
-f SGRBG10 ${CAPTURE_DEV} |
| 152 |
|
| 153 |
CAPTURE_META=$(media-ctl -e "Intel IPU6 ISYS Capture 1") |
| 154 |
./yavta --data-prefix -c100 -n5 -I -s97x1 -B meta-capture \ |
| 155 |
--file=/tmp/meta-#.bin -f GENERIC_8 ${CAPTURE_META} |
| 156 |
|
| 157 |
References |
| 158 |
========== |
| 159 |
|
| 160 |
.. [#f1] https://git.ideasonboard.org/media-ctl.git |
| 161 |
.. [#f2] https://git.ideasonboard.org/yavta.git |