| 1 |
.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later |
| 2 |
.. c:namespace:: V4L |
| 3 |
|
| 4 |
.. _crop: |
| 5 |
|
| 6 |
***************************************************** |
| 7 |
Image Cropping, Insertion and Scaling -- the CROP API |
| 8 |
***************************************************** |
| 9 |
|
| 10 |
.. note:: |
| 11 |
|
| 12 |
The CROP API is mostly superseded by the newer :ref:`SELECTION API |
| 13 |
<selection-api>`. The new API should be preferred in most cases, |
| 14 |
with the exception of pixel aspect ratio detection, which is |
| 15 |
implemented by :ref:`VIDIOC_CROPCAP <VIDIOC_CROPCAP>` and has no |
| 16 |
equivalent in the SELECTION API. See :ref:`selection-vs-crop` for a |
| 17 |
comparison of the two APIs. |
| 18 |
|
| 19 |
Some video capture devices can sample a subsection of the picture and |
| 20 |
shrink or enlarge it to an image of arbitrary size. We call these |
| 21 |
abilities cropping and scaling. Some video output devices can scale an |
| 22 |
image up or down and insert it at an arbitrary scan line and horizontal |
| 23 |
offset into a video signal. |
| 24 |
|
| 25 |
Applications can use the following API to select an area in the video |
| 26 |
signal, query the default area and the hardware limits. |
| 27 |
|
| 28 |
.. note:: |
| 29 |
|
| 30 |
Despite their name, the :ref:`VIDIOC_CROPCAP <VIDIOC_CROPCAP>`, |
| 31 |
:ref:`VIDIOC_G_CROP <VIDIOC_G_CROP>` and :ref:`VIDIOC_S_CROP |
| 32 |
<VIDIOC_G_CROP>` ioctls apply to input as well as output devices. |
| 33 |
|
| 34 |
Scaling requires a source and a target. On a video capture or overlay |
| 35 |
device the source is the video signal, and the cropping ioctls determine |
| 36 |
the area actually sampled. The target are images read by the application |
| 37 |
or overlaid onto the graphics screen. Their size (and position for an |
| 38 |
overlay) is negotiated with the :ref:`VIDIOC_G_FMT <VIDIOC_G_FMT>` |
| 39 |
and :ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>` ioctls. |
| 40 |
|
| 41 |
On a video output device the source are the images passed in by the |
| 42 |
application, and their size is again negotiated with the |
| 43 |
:ref:`VIDIOC_G_FMT <VIDIOC_G_FMT>` and :ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>` |
| 44 |
ioctls, or may be encoded in a compressed video stream. The target is |
| 45 |
the video signal, and the cropping ioctls determine the area where the |
| 46 |
images are inserted. |
| 47 |
|
| 48 |
Source and target rectangles are defined even if the device does not |
| 49 |
support scaling or the :ref:`VIDIOC_G_CROP <VIDIOC_G_CROP>` and |
| 50 |
:ref:`VIDIOC_S_CROP <VIDIOC_G_CROP>` ioctls. Their size (and position |
| 51 |
where applicable) will be fixed in this case. |
| 52 |
|
| 53 |
.. note:: |
| 54 |
|
| 55 |
All capture and output devices that support the CROP or SELECTION |
| 56 |
API will also support the :ref:`VIDIOC_CROPCAP <VIDIOC_CROPCAP>` |
| 57 |
ioctl. |
| 58 |
|
| 59 |
Cropping Structures |
| 60 |
=================== |
| 61 |
|
| 62 |
|
| 63 |
.. _crop-scale: |
| 64 |
|
| 65 |
.. kernel-figure:: crop.svg |
| 66 |
:alt: crop.svg |
| 67 |
:align: center |
| 68 |
|
| 69 |
Image Cropping, Insertion and Scaling |
| 70 |
|
| 71 |
The cropping, insertion and scaling process |
| 72 |
|
| 73 |
|
| 74 |
|
| 75 |
For capture devices the coordinates of the top left corner, width and |
| 76 |
height of the area which can be sampled is given by the ``bounds`` |
| 77 |
substructure of the struct :c:type:`v4l2_cropcap` returned |
| 78 |
by the :ref:`VIDIOC_CROPCAP <VIDIOC_CROPCAP>` ioctl. To support a wide |
| 79 |
range of hardware this specification does not define an origin or units. |
| 80 |
However by convention drivers should horizontally count unscaled samples |
| 81 |
relative to 0H (the leading edge of the horizontal sync pulse, see |
| 82 |
:ref:`vbi-hsync`). Vertically ITU-R line numbers of the first field |
| 83 |
(see ITU R-525 line numbering for :ref:`525 lines <vbi-525>` and for |
| 84 |
:ref:`625 lines <vbi-625>`), multiplied by two if the driver |
| 85 |
can capture both fields. |
| 86 |
|
| 87 |
The top left corner, width and height of the source rectangle, that is |
| 88 |
the area actually sampled, is given by struct |
| 89 |
:c:type:`v4l2_crop` using the same coordinate system as |
| 90 |
struct :c:type:`v4l2_cropcap`. Applications can use the |
| 91 |
:ref:`VIDIOC_G_CROP <VIDIOC_G_CROP>` and :ref:`VIDIOC_S_CROP <VIDIOC_G_CROP>` |
| 92 |
ioctls to get and set this rectangle. It must lie completely within the |
| 93 |
capture boundaries and the driver may further adjust the requested size |
| 94 |
and/or position according to hardware limitations. |
| 95 |
|
| 96 |
Each capture device has a default source rectangle, given by the |
| 97 |
``defrect`` substructure of struct |
| 98 |
:c:type:`v4l2_cropcap`. The center of this rectangle |
| 99 |
shall align with the center of the active picture area of the video |
| 100 |
signal, and cover what the driver writer considers the complete picture. |
| 101 |
Drivers shall reset the source rectangle to the default when the driver |
| 102 |
is first loaded, but not later. |
| 103 |
|
| 104 |
For output devices these structures and ioctls are used accordingly, |
| 105 |
defining the *target* rectangle where the images will be inserted into |
| 106 |
the video signal. |
| 107 |
|
| 108 |
|
| 109 |
Scaling Adjustments |
| 110 |
=================== |
| 111 |
|
| 112 |
Video hardware can have various cropping, insertion and scaling |
| 113 |
limitations. It may only scale up or down, support only discrete scaling |
| 114 |
factors, or have different scaling abilities in horizontal and vertical |
| 115 |
direction. Also it may not support scaling at all. At the same time the |
| 116 |
struct :c:type:`v4l2_crop` rectangle may have to be aligned, |
| 117 |
and both the source and target rectangles may have arbitrary upper and |
| 118 |
lower size limits. In particular the maximum ``width`` and ``height`` in |
| 119 |
struct :c:type:`v4l2_crop` may be smaller than the struct |
| 120 |
:c:type:`v4l2_cropcap`. ``bounds`` area. Therefore, as |
| 121 |
usual, drivers are expected to adjust the requested parameters and |
| 122 |
return the actual values selected. |
| 123 |
|
| 124 |
Applications can change the source or the target rectangle first, as |
| 125 |
they may prefer a particular image size or a certain area in the video |
| 126 |
signal. If the driver has to adjust both to satisfy hardware |
| 127 |
limitations, the last requested rectangle shall take priority, and the |
| 128 |
driver should preferably adjust the opposite one. The |
| 129 |
:ref:`VIDIOC_TRY_FMT <VIDIOC_G_FMT>` ioctl however shall not change |
| 130 |
the driver state and therefore only adjust the requested rectangle. |
| 131 |
|
| 132 |
Suppose scaling on a video capture device is restricted to a factor 1:1 |
| 133 |
or 2:1 in either direction and the target image size must be a multiple |
| 134 |
of 16 × 16 pixels. The source cropping rectangle is set to defaults, |
| 135 |
which are also the upper limit in this example, of 640 × 400 pixels at |
| 136 |
offset 0, 0. An application requests an image size of 300 × 225 pixels, |
| 137 |
assuming video will be scaled down from the "full picture" accordingly. |
| 138 |
The driver sets the image size to the closest possible values 304 × 224, |
| 139 |
then chooses the cropping rectangle closest to the requested size, that |
| 140 |
is 608 × 224 (224 × 2:1 would exceed the limit 400). The offset 0, 0 is |
| 141 |
still valid, thus unmodified. Given the default cropping rectangle |
| 142 |
reported by :ref:`VIDIOC_CROPCAP <VIDIOC_CROPCAP>` the application can |
| 143 |
easily propose another offset to center the cropping rectangle. |
| 144 |
|
| 145 |
Now the application may insist on covering an area using a picture |
| 146 |
aspect ratio closer to the original request, so it asks for a cropping |
| 147 |
rectangle of 608 × 456 pixels. The present scaling factors limit |
| 148 |
cropping to 640 × 384, so the driver returns the cropping size 608 × 384 |
| 149 |
and adjusts the image size to closest possible 304 × 192. |
| 150 |
|
| 151 |
|
| 152 |
Examples |
| 153 |
======== |
| 154 |
|
| 155 |
Source and target rectangles shall remain unchanged across closing and |
| 156 |
reopening a device, such that piping data into or out of a device will |
| 157 |
work without special preparations. More advanced applications should |
| 158 |
ensure the parameters are suitable before starting I/O. |
| 159 |
|
| 160 |
.. note:: |
| 161 |
|
| 162 |
On the next two examples, a video capture device is assumed; |
| 163 |
change ``V4L2_BUF_TYPE_VIDEO_CAPTURE`` for other types of device. |
| 164 |
|
| 165 |
Example: Resetting the cropping parameters |
| 166 |
========================================== |
| 167 |
|
| 168 |
.. code-block:: c |
| 169 |
|
| 170 |
struct v4l2_cropcap cropcap; |
| 171 |
struct v4l2_crop crop; |
| 172 |
|
| 173 |
memset (&cropcap, 0, sizeof (cropcap)); |
| 174 |
cropcap.type = V4L2_BUF_TYPE_VIDEO_CAPTURE; |
| 175 |
|
| 176 |
if (-1 == ioctl (fd, VIDIOC_CROPCAP, &cropcap)) { |
| 177 |
perror ("VIDIOC_CROPCAP"); |
| 178 |
exit (EXIT_FAILURE); |
| 179 |
} |
| 180 |
|
| 181 |
memset (&crop, 0, sizeof (crop)); |
| 182 |
crop.type = V4L2_BUF_TYPE_VIDEO_CAPTURE; |
| 183 |
crop.c = cropcap.defrect; |
| 184 |
|
| 185 |
/* Ignore if cropping is not supported (EINVAL). */ |
| 186 |
|
| 187 |
if (-1 == ioctl (fd, VIDIOC_S_CROP, &crop) |
| 188 |
&& errno != EINVAL) { |
| 189 |
perror ("VIDIOC_S_CROP"); |
| 190 |
exit (EXIT_FAILURE); |
| 191 |
} |
| 192 |
|
| 193 |
|
| 194 |
Example: Simple downscaling |
| 195 |
=========================== |
| 196 |
|
| 197 |
.. code-block:: c |
| 198 |
|
| 199 |
struct v4l2_cropcap cropcap; |
| 200 |
struct v4l2_format format; |
| 201 |
|
| 202 |
reset_cropping_parameters (); |
| 203 |
|
| 204 |
/* Scale down to 1/4 size of full picture. */ |
| 205 |
|
| 206 |
memset (&format, 0, sizeof (format)); /* defaults */ |
| 207 |
|
| 208 |
format.type = V4L2_BUF_TYPE_VIDEO_CAPTURE; |
| 209 |
|
| 210 |
format.fmt.pix.width = cropcap.defrect.width >> 1; |
| 211 |
format.fmt.pix.height = cropcap.defrect.height >> 1; |
| 212 |
format.fmt.pix.pixelformat = V4L2_PIX_FMT_YUYV; |
| 213 |
|
| 214 |
if (-1 == ioctl (fd, VIDIOC_S_FMT, &format)) { |
| 215 |
perror ("VIDIOC_S_FORMAT"); |
| 216 |
exit (EXIT_FAILURE); |
| 217 |
} |
| 218 |
|
| 219 |
/* We could check the actual image size now, the actual scaling factor |
| 220 |
or if the driver can scale at all. */ |
| 221 |
|
| 222 |
Example: Selecting an output area |
| 223 |
================================= |
| 224 |
|
| 225 |
.. note:: This example assumes an output device. |
| 226 |
|
| 227 |
.. code-block:: c |
| 228 |
|
| 229 |
struct v4l2_cropcap cropcap; |
| 230 |
struct v4l2_crop crop; |
| 231 |
|
| 232 |
memset (&cropcap, 0, sizeof (cropcap)); |
| 233 |
cropcap.type = V4L2_BUF_TYPE_VIDEO_OUTPUT; |
| 234 |
|
| 235 |
if (-1 == ioctl (fd, VIDIOC_CROPCAP;, &cropcap)) { |
| 236 |
perror ("VIDIOC_CROPCAP"); |
| 237 |
exit (EXIT_FAILURE); |
| 238 |
} |
| 239 |
|
| 240 |
memset (&crop, 0, sizeof (crop)); |
| 241 |
|
| 242 |
crop.type = V4L2_BUF_TYPE_VIDEO_OUTPUT; |
| 243 |
crop.c = cropcap.defrect; |
| 244 |
|
| 245 |
/* Scale the width and height to 50 % of their original size |
| 246 |
and center the output. */ |
| 247 |
|
| 248 |
crop.c.width /= 2; |
| 249 |
crop.c.height /= 2; |
| 250 |
crop.c.left += crop.c.width / 2; |
| 251 |
crop.c.top += crop.c.height / 2; |
| 252 |
|
| 253 |
/* Ignore if cropping is not supported (EINVAL). */ |
| 254 |
|
| 255 |
if (-1 == ioctl (fd, VIDIOC_S_CROP, &crop) |
| 256 |
&& errno != EINVAL) { |
| 257 |
perror ("VIDIOC_S_CROP"); |
| 258 |
exit (EXIT_FAILURE); |
| 259 |
} |
| 260 |
|
| 261 |
Example: Current scaling factor and pixel aspect |
| 262 |
================================================ |
| 263 |
|
| 264 |
.. note:: This example assumes a video capture device. |
| 265 |
|
| 266 |
.. code-block:: c |
| 267 |
|
| 268 |
struct v4l2_cropcap cropcap; |
| 269 |
struct v4l2_crop crop; |
| 270 |
struct v4l2_format format; |
| 271 |
double hscale, vscale; |
| 272 |
double aspect; |
| 273 |
int dwidth, dheight; |
| 274 |
|
| 275 |
memset (&cropcap, 0, sizeof (cropcap)); |
| 276 |
cropcap.type = V4L2_BUF_TYPE_VIDEO_CAPTURE; |
| 277 |
|
| 278 |
if (-1 == ioctl (fd, VIDIOC_CROPCAP, &cropcap)) { |
| 279 |
perror ("VIDIOC_CROPCAP"); |
| 280 |
exit (EXIT_FAILURE); |
| 281 |
} |
| 282 |
|
| 283 |
memset (&crop, 0, sizeof (crop)); |
| 284 |
crop.type = V4L2_BUF_TYPE_VIDEO_CAPTURE; |
| 285 |
|
| 286 |
if (-1 == ioctl (fd, VIDIOC_G_CROP, &crop)) { |
| 287 |
if (errno != EINVAL) { |
| 288 |
perror ("VIDIOC_G_CROP"); |
| 289 |
exit (EXIT_FAILURE); |
| 290 |
} |
| 291 |
|
| 292 |
/* Cropping not supported. */ |
| 293 |
crop.c = cropcap.defrect; |
| 294 |
} |
| 295 |
|
| 296 |
memset (&format, 0, sizeof (format)); |
| 297 |
format.fmt.type = V4L2_BUF_TYPE_VIDEO_CAPTURE; |
| 298 |
|
| 299 |
if (-1 == ioctl (fd, VIDIOC_G_FMT, &format)) { |
| 300 |
perror ("VIDIOC_G_FMT"); |
| 301 |
exit (EXIT_FAILURE); |
| 302 |
} |
| 303 |
|
| 304 |
/* The scaling applied by the driver. */ |
| 305 |
|
| 306 |
hscale = format.fmt.pix.width / (double) crop.c.width; |
| 307 |
vscale = format.fmt.pix.height / (double) crop.c.height; |
| 308 |
|
| 309 |
aspect = cropcap.pixelaspect.numerator / |
| 310 |
(double) cropcap.pixelaspect.denominator; |
| 311 |
aspect = aspect * hscale / vscale; |
| 312 |
|
| 313 |
/* Devices following ITU-R BT.601 do not capture |
| 314 |
square pixels. For playback on a computer monitor |
| 315 |
we should scale the images to this size. */ |
| 316 |
|
| 317 |
dwidth = format.fmt.pix.width / aspect; |
| 318 |
dheight = format.fmt.pix.height; |