| 1 |
.. SPDX-License-Identifier: GPL-2.0 |
| 2 |
|
| 3 |
.. _media_writing_camera_sensor_drivers: |
| 4 |
|
| 5 |
Writing camera sensor drivers |
| 6 |
============================= |
| 7 |
|
| 8 |
This document covers the in-kernel APIs only. For the best practices on |
| 9 |
userspace API implementation in camera sensor drivers, please see |
| 10 |
:ref:`media_using_camera_sensor_drivers`. |
| 11 |
|
| 12 |
CSI-2, parallel and BT.656 buses |
| 13 |
-------------------------------- |
| 14 |
|
| 15 |
Please see :ref:`transmitter-receiver`. |
| 16 |
|
| 17 |
Handling clocks |
| 18 |
--------------- |
| 19 |
|
| 20 |
Camera sensors have an internal clock tree including a PLL and a number of |
| 21 |
divisors. The clock tree is generally configured by the driver based on a few |
| 22 |
input parameters that are specific to the hardware: the external clock frequency |
| 23 |
and the link frequency. The two parameters generally are obtained from system |
| 24 |
firmware. **No other frequencies should be used in any circumstances.** |
| 25 |
|
| 26 |
The reason why the clock frequencies are so important is that the clock signals |
| 27 |
come out of the SoC, and in many cases a specific frequency is designed to be |
| 28 |
used in the system. Using another frequency may cause harmful effects |
| 29 |
elsewhere. Therefore only the pre-determined frequencies are configurable by the |
| 30 |
user. |
| 31 |
|
| 32 |
The external clock frequency shall be retrieved by obtaining the external clock |
| 33 |
using the ``devm_v4l2_sensor_clk_get()`` helper function, and then getting its |
| 34 |
frequency with ``clk_get_rate()``. Usage of the helper function guarantees |
| 35 |
correct behaviour regardless of whether the sensor is integrated in a DT-based |
| 36 |
or ACPI-based system. |
| 37 |
|
| 38 |
ACPI |
| 39 |
~~~~ |
| 40 |
|
| 41 |
ACPI-based systems typically don't register the sensor external clock with the |
| 42 |
kernel, but specify the external clock frequency in the ``clock-frequency`` |
| 43 |
_DSD property. The ``devm_v4l2_sensor_clk_get()`` helper creates and returns a |
| 44 |
fixed clock set at that rate. |
| 45 |
|
| 46 |
Devicetree |
| 47 |
~~~~~~~~~~ |
| 48 |
|
| 49 |
Devicetree-based systems declare the sensor external clock in the device tree |
| 50 |
and reference it from the sensor node. The preferred way to select the external |
| 51 |
clock frequency is to use the ``assigned-clocks``, ``assigned-clock-parents`` |
| 52 |
and ``assigned-clock-rates`` properties in the sensor node to set the clock |
| 53 |
rate. See the `clock device tree bindings |
| 54 |
<https://github.com/devicetree-org/dt-schema/blob/main/dtschema/schemas/clock/clock.yaml>`_ |
| 55 |
for more information. The ``devm_v4l2_sensor_clk_get()`` helper retrieves and |
| 56 |
returns that clock. |
| 57 |
|
| 58 |
This approach has the drawback that there's no guarantee that the frequency |
| 59 |
hasn't been modified directly or indirectly by another driver, or supported by |
| 60 |
the board's clock tree to begin with. Changes to the Common Clock Framework API |
| 61 |
are required to ensure reliability. |
| 62 |
|
| 63 |
Power management |
| 64 |
---------------- |
| 65 |
|
| 66 |
Camera sensors are used in conjunction with other devices to form a camera |
| 67 |
pipeline. They must obey the rules listed herein to ensure coherent power |
| 68 |
management over the pipeline. |
| 69 |
|
| 70 |
Camera sensor drivers are responsible for controlling the power state of the |
| 71 |
device they otherwise control as well. They shall use runtime PM to manage |
| 72 |
power states. Runtime PM shall be enabled at probe time and disabled at remove |
| 73 |
time. Drivers should enable runtime PM autosuspend. Also see |
| 74 |
:ref:`async sub-device registration <media-registering-async-subdevs>`. |
| 75 |
|
| 76 |
The runtime PM handlers shall handle clocks, regulators, GPIOs, and other |
| 77 |
system resources required to power the sensor up and down. For drivers that |
| 78 |
don't use any of those resources (such as drivers that support ACPI systems |
| 79 |
only), the runtime PM handlers may be left unimplemented. |
| 80 |
|
| 81 |
In general, the device shall be powered on at least when its registers are |
| 82 |
being accessed and when it is streaming. Drivers should use |
| 83 |
``pm_runtime_resume_and_get()`` when starting streaming and |
| 84 |
``pm_runtime_put()`` or ``pm_runtime_put_autosuspend()`` when stopping |
| 85 |
streaming. They may power the device up at probe time (for example to read |
| 86 |
identification registers), but should not keep it powered unconditionally after |
| 87 |
probe. |
| 88 |
|
| 89 |
At system suspend time, the whole camera pipeline must stop streaming, and |
| 90 |
restart when the system is resumed. This requires coordination between the |
| 91 |
camera sensor and the rest of the camera pipeline. Bridge drivers are |
| 92 |
responsible for this coordination, and instruct camera sensors to stop and |
| 93 |
restart streaming by calling the appropriate subdev operations |
| 94 |
(``.enable_streams()`` or ``.disable_streams()``). Camera sensor drivers shall |
| 95 |
therefore **not** keep track of the streaming state to stop streaming in the PM |
| 96 |
suspend handler and restart it in the resume handler. Drivers should in general |
| 97 |
not implement the system PM handlers. |
| 98 |
|
| 99 |
Camera sensor drivers shall **not** implement the subdev ``.s_power()`` |
| 100 |
operation, as it is deprecated. While this operation is implemented in some |
| 101 |
existing drivers as they predate the deprecation, new drivers shall use runtime |
| 102 |
PM instead. If you feel you need to begin calling ``.s_power()`` from an ISP or |
| 103 |
a bridge driver, instead add runtime PM support to the sensor driver you are |
| 104 |
using and drop its ``.s_power()`` handler. |
| 105 |
|
| 106 |
Please also see :ref:`examples <media-camera-sensor-examples>`. |
| 107 |
|
| 108 |
Control framework |
| 109 |
~~~~~~~~~~~~~~~~~ |
| 110 |
|
| 111 |
``v4l2_ctrl_handler_setup()`` function may not be used in the device's runtime |
| 112 |
PM ``runtime_resume`` callback, as it has no way to figure out the power state |
| 113 |
of the device. This is because the power state of the device is only changed |
| 114 |
after the power state transition has taken place. The ``s_ctrl`` callback can be |
| 115 |
used to obtain device's power state after the power state transition: |
| 116 |
|
| 117 |
.. c:function:: int pm_runtime_get_if_in_use(struct device *dev); |
| 118 |
|
| 119 |
The function returns a non-zero value if it succeeded getting the power count or |
| 120 |
runtime PM was disabled, in either of which cases the driver may proceed to |
| 121 |
access the device. |
| 122 |
|
| 123 |
Rotation, orientation and flipping |
| 124 |
---------------------------------- |
| 125 |
|
| 126 |
Use ``v4l2_fwnode_device_parse()`` to obtain rotation and orientation |
| 127 |
information from system firmware and ``v4l2_ctrl_new_fwnode_properties()`` to |
| 128 |
register the appropriate controls. |
| 129 |
|
| 130 |
.. _media-camera-sensor-examples: |
| 131 |
|
| 132 |
Example drivers |
| 133 |
--------------- |
| 134 |
|
| 135 |
Features implemented by sensor drivers vary, and depending on the set of |
| 136 |
supported features and other qualities, particular sensor drivers better serve |
| 137 |
the purpose of an example. The following drivers are known to be good examples: |
| 138 |
|
| 139 |
.. flat-table:: Example sensor drivers |
| 140 |
:header-rows: 0 |
| 141 |
:widths: 1 1 1 2 |
| 142 |
|
| 143 |
* - Driver name |
| 144 |
- File(s) |
| 145 |
- Driver type |
| 146 |
- Example topic |
| 147 |
* - CCS |
| 148 |
- ``drivers/media/i2c/ccs/`` |
| 149 |
- Freely configurable |
| 150 |
- Power management (ACPI and DT), UAPI |
| 151 |
* - imx219 |
| 152 |
- ``drivers/media/i2c/imx219.c`` |
| 153 |
- Register list based |
| 154 |
- Power management (DT), UAPI, mode selection |
| 155 |
* - imx319 |
| 156 |
- ``drivers/media/i2c/imx319.c`` |
| 157 |
- Register list based |
| 158 |
- Power management (ACPI and DT) |