| 1 |
.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later |
| 2 |
.. c:namespace:: V4L |
| 3 |
|
| 4 |
.. _overlay: |
| 5 |
|
| 6 |
*********************** |
| 7 |
Video Overlay Interface |
| 8 |
*********************** |
| 9 |
|
| 10 |
**Also known as Framebuffer Overlay or Previewing.** |
| 11 |
|
| 12 |
Video overlay devices have the ability to genlock (TV-)video into the |
| 13 |
(VGA-)video signal of a graphics card, or to store captured images |
| 14 |
directly in video memory of a graphics card, typically with clipping. |
| 15 |
This can be considerable more efficient than capturing images and |
| 16 |
displaying them by other means. In the old days when only nuclear power |
| 17 |
plants needed cooling towers this used to be the only way to put live |
| 18 |
video into a window. |
| 19 |
|
| 20 |
Video overlay devices are accessed through the same character special |
| 21 |
files as :ref:`video capture <capture>` devices. |
| 22 |
|
| 23 |
.. note:: |
| 24 |
|
| 25 |
The default function of a ``/dev/video`` device is video |
| 26 |
capturing. The overlay function is only available after calling |
| 27 |
the :ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>` ioctl. |
| 28 |
|
| 29 |
The driver may support simultaneous overlay and capturing using the |
| 30 |
read/write and streaming I/O methods. If so, operation at the nominal |
| 31 |
frame rate of the video standard is not guaranteed. Frames may be |
| 32 |
directed away from overlay to capture, or one field may be used for |
| 33 |
overlay and the other for capture if the capture parameters permit this. |
| 34 |
|
| 35 |
Applications should use different file descriptors for capturing and |
| 36 |
overlay. This must be supported by all drivers capable of simultaneous |
| 37 |
capturing and overlay. Optionally these drivers may also permit |
| 38 |
capturing and overlay with a single file descriptor for compatibility |
| 39 |
with V4L and earlier versions of V4L2. [#f1]_ |
| 40 |
|
| 41 |
A common application of two file descriptors is the X11 |
| 42 |
:ref:`Xv/V4L <xvideo>` interface driver and a V4L2 application. |
| 43 |
While the X server controls video overlay, the application can take |
| 44 |
advantage of memory mapping and DMA. |
| 45 |
|
| 46 |
Querying Capabilities |
| 47 |
===================== |
| 48 |
|
| 49 |
Devices supporting the video overlay interface set the |
| 50 |
``V4L2_CAP_VIDEO_OVERLAY`` flag in the ``capabilities`` field of struct |
| 51 |
:c:type:`v4l2_capability` returned by the |
| 52 |
:ref:`VIDIOC_QUERYCAP` ioctl. The overlay I/O |
| 53 |
method specified below must be supported. Tuners and audio inputs are |
| 54 |
optional. |
| 55 |
|
| 56 |
|
| 57 |
Supplemental Functions |
| 58 |
====================== |
| 59 |
|
| 60 |
Video overlay devices shall support :ref:`audio input <audio>`, |
| 61 |
:ref:`tuner`, :ref:`controls <control>`, |
| 62 |
:ref:`cropping and scaling <crop>` and |
| 63 |
:ref:`streaming parameter <streaming-par>` ioctls as needed. The |
| 64 |
:ref:`video input <video>` and :ref:`video standard <standard>` |
| 65 |
ioctls must be supported by all video overlay devices. |
| 66 |
|
| 67 |
|
| 68 |
Setup |
| 69 |
===== |
| 70 |
|
| 71 |
*Note: support for this has been removed.* |
| 72 |
Before overlay can commence applications must program the driver with |
| 73 |
frame buffer parameters, namely the address and size of the frame buffer |
| 74 |
and the image format, for example RGB 5:6:5. The |
| 75 |
:ref:`VIDIOC_G_FBUF <VIDIOC_G_FBUF>` and |
| 76 |
:ref:`VIDIOC_S_FBUF <VIDIOC_G_FBUF>` ioctls are available to get and |
| 77 |
set these parameters, respectively. The :ref:`VIDIOC_S_FBUF <VIDIOC_G_FBUF>` ioctl is |
| 78 |
privileged because it allows to set up DMA into physical memory, |
| 79 |
bypassing the memory protection mechanisms of the kernel. Only the |
| 80 |
superuser can change the frame buffer address and size. Users are not |
| 81 |
supposed to run TV applications as root or with SUID bit set. A small |
| 82 |
helper application with suitable privileges should query the graphics |
| 83 |
system and program the V4L2 driver at the appropriate time. |
| 84 |
|
| 85 |
Some devices add the video overlay to the output signal of the graphics |
| 86 |
card. In this case the frame buffer is not modified by the video device, |
| 87 |
and the frame buffer address and pixel format are not needed by the |
| 88 |
driver. The :ref:`VIDIOC_S_FBUF <VIDIOC_G_FBUF>` ioctl is not privileged. An application |
| 89 |
can check for this type of device by calling the :ref:`VIDIOC_G_FBUF <VIDIOC_G_FBUF>` |
| 90 |
ioctl. |
| 91 |
|
| 92 |
A driver may support any (or none) of five clipping/blending methods: |
| 93 |
|
| 94 |
1. Chroma-keying displays the overlaid image only where pixels in the |
| 95 |
primary graphics surface assume a certain color. |
| 96 |
|
| 97 |
2. *Note: support for this has been removed.* |
| 98 |
A bitmap can be specified where each bit corresponds to a pixel in |
| 99 |
the overlaid image. When the bit is set, the corresponding video |
| 100 |
pixel is displayed, otherwise a pixel of the graphics surface. |
| 101 |
|
| 102 |
3. *Note: support for this has been removed.* |
| 103 |
A list of clipping rectangles can be specified. In these regions *no* |
| 104 |
video is displayed, so the graphics surface can be seen here. |
| 105 |
|
| 106 |
4. The framebuffer has an alpha channel that can be used to clip or |
| 107 |
blend the framebuffer with the video. |
| 108 |
|
| 109 |
5. A global alpha value can be specified to blend the framebuffer |
| 110 |
contents with video images. |
| 111 |
|
| 112 |
When simultaneous capturing and overlay is supported and the hardware |
| 113 |
prohibits different image and frame buffer formats, the format requested |
| 114 |
first takes precedence. The attempt to capture |
| 115 |
(:ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>`) or overlay |
| 116 |
(:ref:`VIDIOC_S_FBUF <VIDIOC_G_FBUF>`) may fail with an ``EBUSY`` error |
| 117 |
code or return accordingly modified parameters.. |
| 118 |
|
| 119 |
|
| 120 |
Overlay Window |
| 121 |
============== |
| 122 |
|
| 123 |
The overlaid image is determined by cropping and overlay window |
| 124 |
parameters. The former select an area of the video picture to capture, |
| 125 |
the latter how images are overlaid and clipped. Cropping initialization |
| 126 |
at minimum requires to reset the parameters to defaults. An example is |
| 127 |
given in :ref:`crop`. |
| 128 |
|
| 129 |
The overlay window is described by a struct |
| 130 |
:c:type:`v4l2_window`. It defines the size of the image, |
| 131 |
its position over the graphics surface and the clipping to be applied. |
| 132 |
To get the current parameters applications set the ``type`` field of a |
| 133 |
struct :c:type:`v4l2_format` to |
| 134 |
``V4L2_BUF_TYPE_VIDEO_OVERLAY`` and call the |
| 135 |
:ref:`VIDIOC_G_FMT <VIDIOC_G_FMT>` ioctl. The driver fills the |
| 136 |
struct :c:type:`v4l2_window` substructure named ``win``. It is not |
| 137 |
possible to retrieve a previously programmed clipping list or bitmap. |
| 138 |
|
| 139 |
To program the overlay window applications set the ``type`` field of a |
| 140 |
struct :c:type:`v4l2_format` to |
| 141 |
``V4L2_BUF_TYPE_VIDEO_OVERLAY``, initialize the ``win`` substructure and |
| 142 |
call the :ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>` ioctl. The driver |
| 143 |
adjusts the parameters against hardware limits and returns the actual |
| 144 |
parameters as :ref:`VIDIOC_G_FMT <VIDIOC_G_FMT>` does. Like :ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>`, the |
| 145 |
:ref:`VIDIOC_TRY_FMT <VIDIOC_G_FMT>` ioctl can be used to learn |
| 146 |
about driver capabilities without actually changing driver state. Unlike |
| 147 |
:ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>` this also works after the overlay has been enabled. |
| 148 |
|
| 149 |
The scaling factor of the overlaid image is implied by the width and |
| 150 |
height given in struct :c:type:`v4l2_window` and the size |
| 151 |
of the cropping rectangle. For more information see :ref:`crop`. |
| 152 |
|
| 153 |
When simultaneous capturing and overlay is supported and the hardware |
| 154 |
prohibits different image and window sizes, the size requested first |
| 155 |
takes precedence. The attempt to capture or overlay as well |
| 156 |
(:ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>`) may fail with an ``EBUSY`` error |
| 157 |
code or return accordingly modified parameters. |
| 158 |
|
| 159 |
|
| 160 |
.. c:type:: v4l2_window |
| 161 |
|
| 162 |
struct v4l2_window |
| 163 |
------------------ |
| 164 |
|
| 165 |
``struct v4l2_rect w`` |
| 166 |
Size and position of the window relative to the top, left corner of |
| 167 |
the frame buffer defined with |
| 168 |
:ref:`VIDIOC_S_FBUF <VIDIOC_G_FBUF>`. The window can extend the |
| 169 |
frame buffer width and height, the ``x`` and ``y`` coordinates can |
| 170 |
be negative, and it can lie completely outside the frame buffer. The |
| 171 |
driver clips the window accordingly, or if that is not possible, |
| 172 |
modifies its size and/or position. |
| 173 |
|
| 174 |
``enum v4l2_field field`` |
| 175 |
Applications set this field to determine which video field shall be |
| 176 |
overlaid, typically one of ``V4L2_FIELD_ANY`` (0), |
| 177 |
``V4L2_FIELD_TOP``, ``V4L2_FIELD_BOTTOM`` or |
| 178 |
``V4L2_FIELD_INTERLACED``. Drivers may have to choose a different |
| 179 |
field order and return the actual setting here. |
| 180 |
|
| 181 |
``__u32 chromakey`` |
| 182 |
When chroma-keying has been negotiated with |
| 183 |
:ref:`VIDIOC_S_FBUF <VIDIOC_G_FBUF>` applications set this field |
| 184 |
to the desired pixel value for the chroma key. The format is the |
| 185 |
same as the pixel format of the framebuffer (struct |
| 186 |
:c:type:`v4l2_framebuffer` ``fmt.pixelformat`` |
| 187 |
field), with bytes in host order. E. g. for |
| 188 |
:ref:`V4L2_PIX_FMT_BGR24 <V4L2-PIX-FMT-BGR32>` the value should |
| 189 |
be 0xRRGGBB on a little endian, 0xBBGGRR on a big endian host. |
| 190 |
|
| 191 |
``struct v4l2_clip * clips`` |
| 192 |
*Note: support for this has been removed.* |
| 193 |
When chroma-keying has *not* been negotiated and |
| 194 |
:ref:`VIDIOC_G_FBUF <VIDIOC_G_FBUF>` indicated this capability, |
| 195 |
applications can set this field to point to an array of clipping |
| 196 |
rectangles. |
| 197 |
|
| 198 |
Like the window coordinates w, clipping rectangles are defined |
| 199 |
relative to the top, left corner of the frame buffer. However |
| 200 |
clipping rectangles must not extend the frame buffer width and |
| 201 |
height, and they must not overlap. If possible applications |
| 202 |
should merge adjacent rectangles. Whether this must create |
| 203 |
x-y or y-x bands, or the order of rectangles, is not defined. When |
| 204 |
clip lists are not supported the driver ignores this field. Its |
| 205 |
contents after calling :ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>` |
| 206 |
are undefined. |
| 207 |
|
| 208 |
``__u32 clipcount`` |
| 209 |
*Note: support for this has been removed.* |
| 210 |
When the application set the ``clips`` field, this field must |
| 211 |
contain the number of clipping rectangles in the list. When clip |
| 212 |
lists are not supported the driver ignores this field, its contents |
| 213 |
after calling :ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>` are undefined. When clip lists are |
| 214 |
supported but no clipping is desired this field must be set to zero. |
| 215 |
|
| 216 |
``void * bitmap`` |
| 217 |
*Note: support for this has been removed.* |
| 218 |
When chroma-keying has *not* been negotiated and |
| 219 |
:ref:`VIDIOC_G_FBUF <VIDIOC_G_FBUF>` indicated this capability, |
| 220 |
applications can set this field to point to a clipping bit mask. |
| 221 |
|
| 222 |
It must be of the same size as the window, ``w.width`` and ``w.height``. |
| 223 |
Each bit corresponds to a pixel in the overlaid image, which is |
| 224 |
displayed only when the bit is *set*. Pixel coordinates translate to |
| 225 |
bits like: |
| 226 |
|
| 227 |
|
| 228 |
.. code-block:: c |
| 229 |
|
| 230 |
((__u8 *) bitmap)[w.width * y + x / 8] & (1 << (x & 7)) |
| 231 |
|
| 232 |
where ``0`` ≤ x < ``w.width`` and ``0`` ≤ y <``w.height``. [#f2]_ |
| 233 |
|
| 234 |
When a clipping bit mask is not supported the driver ignores this field, |
| 235 |
its contents after calling :ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>` are |
| 236 |
undefined. When a bit mask is supported but no clipping is desired this |
| 237 |
field must be set to ``NULL``. |
| 238 |
|
| 239 |
Applications need not create a clip list or bit mask. When they pass |
| 240 |
both, or despite negotiating chroma-keying, the results are undefined. |
| 241 |
Regardless of the chosen method, the clipping abilities of the hardware |
| 242 |
may be limited in quantity or quality. The results when these limits are |
| 243 |
exceeded are undefined. [#f3]_ |
| 244 |
|
| 245 |
``__u8 global_alpha`` |
| 246 |
The global alpha value used to blend the framebuffer with video |
| 247 |
images, if global alpha blending has been negotiated |
| 248 |
(``V4L2_FBUF_FLAG_GLOBAL_ALPHA``, see |
| 249 |
:ref:`VIDIOC_S_FBUF <VIDIOC_G_FBUF>`, |
| 250 |
:ref:`framebuffer-flags`). |
| 251 |
|
| 252 |
.. note:: |
| 253 |
|
| 254 |
This field was added in Linux 2.6.23, extending the |
| 255 |
structure. However the :ref:`VIDIOC_[G|S|TRY]_FMT <VIDIOC_G_FMT>` |
| 256 |
ioctls, which take a pointer to a :c:type:`v4l2_format` |
| 257 |
parent structure with padding bytes at the end, are not affected. |
| 258 |
|
| 259 |
|
| 260 |
.. c:type:: v4l2_clip |
| 261 |
|
| 262 |
struct v4l2_clip [#f4]_ |
| 263 |
----------------------- |
| 264 |
|
| 265 |
``struct v4l2_rect c`` |
| 266 |
Coordinates of the clipping rectangle, relative to the top, left |
| 267 |
corner of the frame buffer. Only window pixels *outside* all |
| 268 |
clipping rectangles are displayed. |
| 269 |
|
| 270 |
``struct v4l2_clip * next`` |
| 271 |
Pointer to the next clipping rectangle, ``NULL`` when this is the last |
| 272 |
rectangle. Drivers ignore this field, it cannot be used to pass a |
| 273 |
linked list of clipping rectangles. |
| 274 |
|
| 275 |
|
| 276 |
.. c:type:: v4l2_rect |
| 277 |
|
| 278 |
struct v4l2_rect |
| 279 |
---------------- |
| 280 |
|
| 281 |
``__s32 left`` |
| 282 |
Horizontal offset of the top, left corner of the rectangle, in |
| 283 |
pixels. |
| 284 |
|
| 285 |
``__s32 top`` |
| 286 |
Vertical offset of the top, left corner of the rectangle, in pixels. |
| 287 |
Offsets increase to the right and down. |
| 288 |
|
| 289 |
``__u32 width`` |
| 290 |
Width of the rectangle, in pixels. |
| 291 |
|
| 292 |
``__u32 height`` |
| 293 |
Height of the rectangle, in pixels. |
| 294 |
|
| 295 |
|
| 296 |
Enabling Overlay |
| 297 |
================ |
| 298 |
|
| 299 |
To start or stop the frame buffer overlay applications call the |
| 300 |
:ref:`VIDIOC_OVERLAY` ioctl. |
| 301 |
|
| 302 |
.. [#f1] |
| 303 |
In the opinion of the designers of this API, no driver writer taking |
| 304 |
the efforts to support simultaneous capturing and overlay will |
| 305 |
restrict this ability by requiring a single file descriptor, as in |
| 306 |
V4L and earlier versions of V4L2. Making this optional means |
| 307 |
applications depending on two file descriptors need backup routines |
| 308 |
to be compatible with all drivers, which is considerable more work |
| 309 |
than using two fds in applications which do not. Also two fd's fit |
| 310 |
the general concept of one file descriptor for each logical stream. |
| 311 |
Hence as a complexity trade-off drivers *must* support two file |
| 312 |
descriptors and *may* support single fd operation. |
| 313 |
|
| 314 |
.. [#f2] |
| 315 |
Should we require ``w.width`` to be a multiple of eight? |
| 316 |
|
| 317 |
.. [#f3] |
| 318 |
When the image is written into frame buffer memory it will be |
| 319 |
undesirable if the driver clips out less pixels than expected, |
| 320 |
because the application and graphics system are not aware these |
| 321 |
regions need to be refreshed. The driver should clip out more pixels |
| 322 |
or not write the image at all. |
| 323 |
|
| 324 |
.. [#f4] |
| 325 |
The X Window system defines "regions" which are vectors of ``struct |
| 326 |
BoxRec { short x1, y1, x2, y2; }`` with ``width = x2 - x1`` and |
| 327 |
``height = y2 - y1``, so one cannot pass X11 clip lists directly. |