| 1 |
.. SPDX-License-Identifier: GPL-2.0 OR GFDL-1.1-no-invariants-or-later |
| 2 |
.. c:namespace:: MC |
| 3 |
|
| 4 |
.. _media-request-api: |
| 5 |
|
| 6 |
Request API |
| 7 |
=========== |
| 8 |
|
| 9 |
The Request API has been designed to allow V4L2 to deal with requirements of |
| 10 |
modern devices (stateless codecs, complex camera pipelines, ...) and APIs |
| 11 |
(Android Codec v2). One such requirement is the ability for devices belonging to |
| 12 |
the same pipeline to reconfigure and collaborate closely on a per-frame basis. |
| 13 |
Another is support of stateless codecs, which require controls to be applied |
| 14 |
to specific frames (aka 'per-frame controls') in order to be used efficiently. |
| 15 |
|
| 16 |
While the initial use-case was V4L2, it can be extended to other subsystems |
| 17 |
as well, as long as they use the media controller. |
| 18 |
|
| 19 |
Supporting these features without the Request API is not always possible and if |
| 20 |
it is, it is terribly inefficient: user-space would have to flush all activity |
| 21 |
on the media pipeline, reconfigure it for the next frame, queue the buffers to |
| 22 |
be processed with that configuration, and wait until they are all available for |
| 23 |
dequeuing before considering the next frame. This defeats the purpose of having |
| 24 |
buffer queues since in practice only one buffer would be queued at a time. |
| 25 |
|
| 26 |
The Request API allows a specific configuration of the pipeline (media |
| 27 |
controller topology + configuration for each media entity) to be associated with |
| 28 |
specific buffers. This allows user-space to schedule several tasks ("requests") |
| 29 |
with different configurations in advance, knowing that the configuration will be |
| 30 |
applied when needed to get the expected result. Configuration values at the time |
| 31 |
of request completion are also available for reading. |
| 32 |
|
| 33 |
General Usage |
| 34 |
------------- |
| 35 |
|
| 36 |
The Request API extends the Media Controller API and cooperates with |
| 37 |
subsystem-specific APIs to support request usage. At the Media Controller |
| 38 |
level, requests are allocated from the supporting Media Controller device |
| 39 |
node. Their life cycle is then managed through the request file descriptors in |
| 40 |
an opaque way. Configuration data, buffer handles and processing results |
| 41 |
stored in requests are accessed through subsystem-specific APIs extended for |
| 42 |
request support, such as V4L2 APIs that take an explicit ``request_fd`` |
| 43 |
parameter. |
| 44 |
|
| 45 |
Request Allocation |
| 46 |
------------------ |
| 47 |
|
| 48 |
User-space allocates requests using :ref:`MEDIA_IOC_REQUEST_ALLOC` |
| 49 |
for the media device node. This returns a file descriptor representing the |
| 50 |
request. Typically, several such requests will be allocated. |
| 51 |
|
| 52 |
Request Preparation |
| 53 |
------------------- |
| 54 |
|
| 55 |
Standard V4L2 ioctls can then receive a request file descriptor to express the |
| 56 |
fact that the ioctl is part of said request, and is not to be applied |
| 57 |
immediately. See :ref:`MEDIA_IOC_REQUEST_ALLOC` for a list of ioctls that |
| 58 |
support this. Configurations set with a ``request_fd`` parameter are stored |
| 59 |
instead of being immediately applied, and buffers queued to a request do not |
| 60 |
enter the regular buffer queue until the request itself is queued. |
| 61 |
|
| 62 |
Request Submission |
| 63 |
------------------ |
| 64 |
|
| 65 |
Once the configuration and buffers of the request are specified, it can be |
| 66 |
queued by calling :ref:`MEDIA_REQUEST_IOC_QUEUE` on the request file descriptor. |
| 67 |
A request must contain at least one buffer, otherwise ``ENOENT`` is returned. |
| 68 |
A queued request cannot be modified anymore. |
| 69 |
|
| 70 |
.. caution:: |
| 71 |
For :ref:`memory-to-memory devices <mem2mem>` you can use requests only for |
| 72 |
output buffers, not for capture buffers. Attempting to add a capture buffer |
| 73 |
to a request will result in an ``EBADR`` error. |
| 74 |
|
| 75 |
If the request contains configurations for multiple entities, individual drivers |
| 76 |
may synchronize so the requested pipeline's topology is applied before the |
| 77 |
buffers are processed. Media controller drivers do a best effort implementation |
| 78 |
since perfect atomicity may not be possible due to hardware limitations. |
| 79 |
|
| 80 |
.. caution:: |
| 81 |
|
| 82 |
It is not allowed to mix queuing requests with directly queuing buffers: |
| 83 |
whichever method is used first locks this in place until |
| 84 |
:ref:`VIDIOC_STREAMOFF <VIDIOC_STREAMON>` is called or the device is |
| 85 |
:ref:`closed <func-close>`. Attempts to directly queue a buffer when earlier |
| 86 |
a buffer was queued via a request or vice versa will result in an ``EBUSY`` |
| 87 |
error. |
| 88 |
|
| 89 |
Controls can still be set without a request and are applied immediately, |
| 90 |
regardless of whether a request is in use or not. |
| 91 |
|
| 92 |
.. caution:: |
| 93 |
|
| 94 |
Setting the same control through a request and also directly can lead to |
| 95 |
undefined behavior! |
| 96 |
|
| 97 |
User-space can :c:func:`poll()` a request file descriptor in |
| 98 |
order to wait until the request completes. A request is considered complete |
| 99 |
once all its associated buffers are available for dequeuing and all the |
| 100 |
associated controls have been updated with the values at the time of completion. |
| 101 |
Note that user-space does not need to wait for the request to complete to |
| 102 |
dequeue its buffers: buffers that are available halfway through a request can |
| 103 |
be dequeued independently of the request's state. |
| 104 |
|
| 105 |
A completed request contains the state of the device after the request was |
| 106 |
executed. User-space can query that state by calling |
| 107 |
:ref:`ioctl VIDIOC_G_EXT_CTRLS <VIDIOC_G_EXT_CTRLS>` with the request file |
| 108 |
descriptor. Calling :ref:`ioctl VIDIOC_G_EXT_CTRLS <VIDIOC_G_EXT_CTRLS>` for a |
| 109 |
request that has been queued but not yet completed will return ``EBUSY`` |
| 110 |
since the control values might be changed at any time by the driver while the |
| 111 |
request is in flight. |
| 112 |
|
| 113 |
.. _media-request-life-time: |
| 114 |
|
| 115 |
Recycling and Destruction |
| 116 |
------------------------- |
| 117 |
|
| 118 |
Finally, a completed request can either be discarded or be reused. Calling |
| 119 |
:c:func:`close()` on a request file descriptor will make |
| 120 |
that file descriptor unusable and the request will be freed once it is no |
| 121 |
longer in use by the kernel. That is, if the request is queued and then the |
| 122 |
file descriptor is closed, then it won't be freed until the driver completed |
| 123 |
the request. |
| 124 |
|
| 125 |
The :ref:`MEDIA_REQUEST_IOC_REINIT` will clear a request's state and make it |
| 126 |
available again. No state is retained by this operation: the request is as |
| 127 |
if it had just been allocated. |
| 128 |
|
| 129 |
Example for a Codec Device |
| 130 |
-------------------------- |
| 131 |
|
| 132 |
For use-cases such as :ref:`codecs <mem2mem>`, the request API can be used |
| 133 |
to associate specific controls to |
| 134 |
be applied by the driver for the OUTPUT buffer, allowing user-space |
| 135 |
to queue many such buffers in advance. It can also take advantage of requests' |
| 136 |
ability to capture the state of controls when the request completes to read back |
| 137 |
information that may be subject to change. |
| 138 |
|
| 139 |
Put into code, after obtaining a request, user-space can assign controls and one |
| 140 |
OUTPUT buffer to it: |
| 141 |
|
| 142 |
.. code-block:: c |
| 143 |
|
| 144 |
struct v4l2_buffer buf; |
| 145 |
struct v4l2_ext_controls ctrls; |
| 146 |
int req_fd; |
| 147 |
... |
| 148 |
if (ioctl(media_fd, MEDIA_IOC_REQUEST_ALLOC, &req_fd)) |
| 149 |
return errno; |
| 150 |
... |
| 151 |
ctrls.which = V4L2_CTRL_WHICH_REQUEST_VAL; |
| 152 |
ctrls.request_fd = req_fd; |
| 153 |
if (ioctl(codec_fd, VIDIOC_S_EXT_CTRLS, &ctrls)) |
| 154 |
return errno; |
| 155 |
... |
| 156 |
buf.type = V4L2_BUF_TYPE_VIDEO_OUTPUT; |
| 157 |
buf.flags |= V4L2_BUF_FLAG_REQUEST_FD; |
| 158 |
buf.request_fd = req_fd; |
| 159 |
if (ioctl(codec_fd, VIDIOC_QBUF, &buf)) |
| 160 |
return errno; |
| 161 |
|
| 162 |
Note that it is not allowed to use the Request API for CAPTURE buffers |
| 163 |
since there are no per-frame settings to report there. |
| 164 |
|
| 165 |
Once the request is fully prepared, it can be queued to the driver: |
| 166 |
|
| 167 |
.. code-block:: c |
| 168 |
|
| 169 |
if (ioctl(req_fd, MEDIA_REQUEST_IOC_QUEUE)) |
| 170 |
return errno; |
| 171 |
|
| 172 |
User-space can then either wait for the request to complete by calling poll() on |
| 173 |
its file descriptor, or start dequeuing CAPTURE buffers. Most likely, it will |
| 174 |
want to get CAPTURE buffers as soon as possible and this can be done using a |
| 175 |
regular :ref:`VIDIOC_DQBUF <VIDIOC_QBUF>`: |
| 176 |
|
| 177 |
.. code-block:: c |
| 178 |
|
| 179 |
struct v4l2_buffer buf; |
| 180 |
|
| 181 |
memset(&buf, 0, sizeof(buf)); |
| 182 |
buf.type = V4L2_BUF_TYPE_VIDEO_CAPTURE; |
| 183 |
if (ioctl(codec_fd, VIDIOC_DQBUF, &buf)) |
| 184 |
return errno; |
| 185 |
|
| 186 |
Note that this example assumes for simplicity that for every OUTPUT buffer |
| 187 |
there will be one CAPTURE buffer, but this does not have to be the case. |
| 188 |
|
| 189 |
We can then, after ensuring that the request is completed via polling the |
| 190 |
request file descriptor, query control values at the time of its completion via |
| 191 |
a call to :ref:`VIDIOC_G_EXT_CTRLS <VIDIOC_G_EXT_CTRLS>`. |
| 192 |
This is particularly useful for volatile controls for which we want to |
| 193 |
query values as soon as the capture buffer is produced. |
| 194 |
|
| 195 |
.. code-block:: c |
| 196 |
|
| 197 |
struct pollfd pfd = { .events = POLLPRI, .fd = req_fd }; |
| 198 |
poll(&pfd, 1, -1); |
| 199 |
... |
| 200 |
ctrls.which = V4L2_CTRL_WHICH_REQUEST_VAL; |
| 201 |
ctrls.request_fd = req_fd; |
| 202 |
if (ioctl(codec_fd, VIDIOC_G_EXT_CTRLS, &ctrls)) |
| 203 |
return errno; |
| 204 |
|
| 205 |
Once we don't need the request anymore, we can either recycle it for reuse with |
| 206 |
:ref:`MEDIA_REQUEST_IOC_REINIT`... |
| 207 |
|
| 208 |
.. code-block:: c |
| 209 |
|
| 210 |
if (ioctl(req_fd, MEDIA_REQUEST_IOC_REINIT)) |
| 211 |
return errno; |
| 212 |
|
| 213 |
... or close its file descriptor to completely dispose of it. |
| 214 |
|
| 215 |
.. code-block:: c |
| 216 |
|
| 217 |
close(req_fd); |
| 218 |
|
| 219 |
Example for a Simple Capture Device |
| 220 |
----------------------------------- |
| 221 |
|
| 222 |
With a simple capture device, requests can be used to specify controls to apply |
| 223 |
for a given CAPTURE buffer. |
| 224 |
|
| 225 |
.. code-block:: c |
| 226 |
|
| 227 |
struct v4l2_buffer buf; |
| 228 |
struct v4l2_ext_controls ctrls; |
| 229 |
int req_fd; |
| 230 |
... |
| 231 |
if (ioctl(media_fd, MEDIA_IOC_REQUEST_ALLOC, &req_fd)) |
| 232 |
return errno; |
| 233 |
... |
| 234 |
ctrls.which = V4L2_CTRL_WHICH_REQUEST_VAL; |
| 235 |
ctrls.request_fd = req_fd; |
| 236 |
if (ioctl(camera_fd, VIDIOC_S_EXT_CTRLS, &ctrls)) |
| 237 |
return errno; |
| 238 |
... |
| 239 |
buf.type = V4L2_BUF_TYPE_VIDEO_CAPTURE; |
| 240 |
buf.flags |= V4L2_BUF_FLAG_REQUEST_FD; |
| 241 |
buf.request_fd = req_fd; |
| 242 |
if (ioctl(camera_fd, VIDIOC_QBUF, &buf)) |
| 243 |
return errno; |
| 244 |
|
| 245 |
Once the request is fully prepared, it can be queued to the driver: |
| 246 |
|
| 247 |
.. code-block:: c |
| 248 |
|
| 249 |
if (ioctl(req_fd, MEDIA_REQUEST_IOC_QUEUE)) |
| 250 |
return errno; |
| 251 |
|
| 252 |
User-space can then dequeue buffers, wait for the request completion, query |
| 253 |
controls and recycle the request as in the M2M example above. |