Debian Codegraph @

Documentation/userspace-api/media/v4l/dev-overlay.rst

raw
1 .. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
2 .. c:namespace:: V4L
3
4 .. _overlay:
5
6 ***********************
7 Video Overlay Interface
8 ***********************
9
10 **Also known as Framebuffer Overlay or Previewing.**
11
12 Video overlay devices have the ability to genlock (TV-)video into the
13 (VGA-)video signal of a graphics card, or to store captured images
14 directly in video memory of a graphics card, typically with clipping.
15 This can be considerable more efficient than capturing images and
16 displaying them by other means. In the old days when only nuclear power
17 plants needed cooling towers this used to be the only way to put live
18 video into a window.
19
20 Video overlay devices are accessed through the same character special
21 files as :ref:`video capture <capture>` devices.
22
23 .. note::
24
25 The default function of a ``/dev/video`` device is video
26 capturing. The overlay function is only available after calling
27 the :ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>` ioctl.
28
29 The driver may support simultaneous overlay and capturing using the
30 read/write and streaming I/O methods. If so, operation at the nominal
31 frame rate of the video standard is not guaranteed. Frames may be
32 directed away from overlay to capture, or one field may be used for
33 overlay and the other for capture if the capture parameters permit this.
34
35 Applications should use different file descriptors for capturing and
36 overlay. This must be supported by all drivers capable of simultaneous
37 capturing and overlay. Optionally these drivers may also permit
38 capturing and overlay with a single file descriptor for compatibility
39 with V4L and earlier versions of V4L2. [#f1]_
40
41 A common application of two file descriptors is the X11
42 :ref:`Xv/V4L <xvideo>` interface driver and a V4L2 application.
43 While the X server controls video overlay, the application can take
44 advantage of memory mapping and DMA.
45
46 Querying Capabilities
47 =====================
48
49 Devices supporting the video overlay interface set the
50 ``V4L2_CAP_VIDEO_OVERLAY`` flag in the ``capabilities`` field of struct
51 :c:type:`v4l2_capability` returned by the
52 :ref:`VIDIOC_QUERYCAP` ioctl. The overlay I/O
53 method specified below must be supported. Tuners and audio inputs are
54 optional.
55
56
57 Supplemental Functions
58 ======================
59
60 Video overlay devices shall support :ref:`audio input <audio>`,
61 :ref:`tuner`, :ref:`controls <control>`,
62 :ref:`cropping and scaling <crop>` and
63 :ref:`streaming parameter <streaming-par>` ioctls as needed. The
64 :ref:`video input <video>` and :ref:`video standard <standard>`
65 ioctls must be supported by all video overlay devices.
66
67
68 Setup
69 =====
70
71 *Note: support for this has been removed.*
72 Before overlay can commence applications must program the driver with
73 frame buffer parameters, namely the address and size of the frame buffer
74 and the image format, for example RGB 5:6:5. The
75 :ref:`VIDIOC_G_FBUF <VIDIOC_G_FBUF>` and
76 :ref:`VIDIOC_S_FBUF <VIDIOC_G_FBUF>` ioctls are available to get and
77 set these parameters, respectively. The :ref:`VIDIOC_S_FBUF <VIDIOC_G_FBUF>` ioctl is
78 privileged because it allows to set up DMA into physical memory,
79 bypassing the memory protection mechanisms of the kernel. Only the
80 superuser can change the frame buffer address and size. Users are not
81 supposed to run TV applications as root or with SUID bit set. A small
82 helper application with suitable privileges should query the graphics
83 system and program the V4L2 driver at the appropriate time.
84
85 Some devices add the video overlay to the output signal of the graphics
86 card. In this case the frame buffer is not modified by the video device,
87 and the frame buffer address and pixel format are not needed by the
88 driver. The :ref:`VIDIOC_S_FBUF <VIDIOC_G_FBUF>` ioctl is not privileged. An application
89 can check for this type of device by calling the :ref:`VIDIOC_G_FBUF <VIDIOC_G_FBUF>`
90 ioctl.
91
92 A driver may support any (or none) of five clipping/blending methods:
93
94 1. Chroma-keying displays the overlaid image only where pixels in the
95 primary graphics surface assume a certain color.
96
97 2. *Note: support for this has been removed.*
98 A bitmap can be specified where each bit corresponds to a pixel in
99 the overlaid image. When the bit is set, the corresponding video
100 pixel is displayed, otherwise a pixel of the graphics surface.
101
102 3. *Note: support for this has been removed.*
103 A list of clipping rectangles can be specified. In these regions *no*
104 video is displayed, so the graphics surface can be seen here.
105
106 4. The framebuffer has an alpha channel that can be used to clip or
107 blend the framebuffer with the video.
108
109 5. A global alpha value can be specified to blend the framebuffer
110 contents with video images.
111
112 When simultaneous capturing and overlay is supported and the hardware
113 prohibits different image and frame buffer formats, the format requested
114 first takes precedence. The attempt to capture
115 (:ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>`) or overlay
116 (:ref:`VIDIOC_S_FBUF <VIDIOC_G_FBUF>`) may fail with an ``EBUSY`` error
117 code or return accordingly modified parameters..
118
119
120 Overlay Window
121 ==============
122
123 The overlaid image is determined by cropping and overlay window
124 parameters. The former select an area of the video picture to capture,
125 the latter how images are overlaid and clipped. Cropping initialization
126 at minimum requires to reset the parameters to defaults. An example is
127 given in :ref:`crop`.
128
129 The overlay window is described by a struct
130 :c:type:`v4l2_window`. It defines the size of the image,
131 its position over the graphics surface and the clipping to be applied.
132 To get the current parameters applications set the ``type`` field of a
133 struct :c:type:`v4l2_format` to
134 ``V4L2_BUF_TYPE_VIDEO_OVERLAY`` and call the
135 :ref:`VIDIOC_G_FMT <VIDIOC_G_FMT>` ioctl. The driver fills the
136 struct :c:type:`v4l2_window` substructure named ``win``. It is not
137 possible to retrieve a previously programmed clipping list or bitmap.
138
139 To program the overlay window applications set the ``type`` field of a
140 struct :c:type:`v4l2_format` to
141 ``V4L2_BUF_TYPE_VIDEO_OVERLAY``, initialize the ``win`` substructure and
142 call the :ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>` ioctl. The driver
143 adjusts the parameters against hardware limits and returns the actual
144 parameters as :ref:`VIDIOC_G_FMT <VIDIOC_G_FMT>` does. Like :ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>`, the
145 :ref:`VIDIOC_TRY_FMT <VIDIOC_G_FMT>` ioctl can be used to learn
146 about driver capabilities without actually changing driver state. Unlike
147 :ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>` this also works after the overlay has been enabled.
148
149 The scaling factor of the overlaid image is implied by the width and
150 height given in struct :c:type:`v4l2_window` and the size
151 of the cropping rectangle. For more information see :ref:`crop`.
152
153 When simultaneous capturing and overlay is supported and the hardware
154 prohibits different image and window sizes, the size requested first
155 takes precedence. The attempt to capture or overlay as well
156 (:ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>`) may fail with an ``EBUSY`` error
157 code or return accordingly modified parameters.
158
159
160 .. c:type:: v4l2_window
161
162 struct v4l2_window
163 ------------------
164
165 ``struct v4l2_rect w``
166 Size and position of the window relative to the top, left corner of
167 the frame buffer defined with
168 :ref:`VIDIOC_S_FBUF <VIDIOC_G_FBUF>`. The window can extend the
169 frame buffer width and height, the ``x`` and ``y`` coordinates can
170 be negative, and it can lie completely outside the frame buffer. The
171 driver clips the window accordingly, or if that is not possible,
172 modifies its size and/or position.
173
174 ``enum v4l2_field field``
175 Applications set this field to determine which video field shall be
176 overlaid, typically one of ``V4L2_FIELD_ANY`` (0),
177 ``V4L2_FIELD_TOP``, ``V4L2_FIELD_BOTTOM`` or
178 ``V4L2_FIELD_INTERLACED``. Drivers may have to choose a different
179 field order and return the actual setting here.
180
181 ``__u32 chromakey``
182 When chroma-keying has been negotiated with
183 :ref:`VIDIOC_S_FBUF <VIDIOC_G_FBUF>` applications set this field
184 to the desired pixel value for the chroma key. The format is the
185 same as the pixel format of the framebuffer (struct
186 :c:type:`v4l2_framebuffer` ``fmt.pixelformat``
187 field), with bytes in host order. E. g. for
188 :ref:`V4L2_PIX_FMT_BGR24 <V4L2-PIX-FMT-BGR32>` the value should
189 be 0xRRGGBB on a little endian, 0xBBGGRR on a big endian host.
190
191 ``struct v4l2_clip * clips``
192 *Note: support for this has been removed.*
193 When chroma-keying has *not* been negotiated and
194 :ref:`VIDIOC_G_FBUF <VIDIOC_G_FBUF>` indicated this capability,
195 applications can set this field to point to an array of clipping
196 rectangles.
197
198 Like the window coordinates w, clipping rectangles are defined
199 relative to the top, left corner of the frame buffer. However
200 clipping rectangles must not extend the frame buffer width and
201 height, and they must not overlap. If possible applications
202 should merge adjacent rectangles. Whether this must create
203 x-y or y-x bands, or the order of rectangles, is not defined. When
204 clip lists are not supported the driver ignores this field. Its
205 contents after calling :ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>`
206 are undefined.
207
208 ``__u32 clipcount``
209 *Note: support for this has been removed.*
210 When the application set the ``clips`` field, this field must
211 contain the number of clipping rectangles in the list. When clip
212 lists are not supported the driver ignores this field, its contents
213 after calling :ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>` are undefined. When clip lists are
214 supported but no clipping is desired this field must be set to zero.
215
216 ``void * bitmap``
217 *Note: support for this has been removed.*
218 When chroma-keying has *not* been negotiated and
219 :ref:`VIDIOC_G_FBUF <VIDIOC_G_FBUF>` indicated this capability,
220 applications can set this field to point to a clipping bit mask.
221
222 It must be of the same size as the window, ``w.width`` and ``w.height``.
223 Each bit corresponds to a pixel in the overlaid image, which is
224 displayed only when the bit is *set*. Pixel coordinates translate to
225 bits like:
226
227
228 .. code-block:: c
229
230 ((__u8 *) bitmap)[w.width * y + x / 8] & (1 << (x & 7))
231
232 where ``0`` ≤ x < ``w.width`` and ``0`` ≤ y <``w.height``. [#f2]_
233
234 When a clipping bit mask is not supported the driver ignores this field,
235 its contents after calling :ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>` are
236 undefined. When a bit mask is supported but no clipping is desired this
237 field must be set to ``NULL``.
238
239 Applications need not create a clip list or bit mask. When they pass
240 both, or despite negotiating chroma-keying, the results are undefined.
241 Regardless of the chosen method, the clipping abilities of the hardware
242 may be limited in quantity or quality. The results when these limits are
243 exceeded are undefined. [#f3]_
244
245 ``__u8 global_alpha``
246 The global alpha value used to blend the framebuffer with video
247 images, if global alpha blending has been negotiated
248 (``V4L2_FBUF_FLAG_GLOBAL_ALPHA``, see
249 :ref:`VIDIOC_S_FBUF <VIDIOC_G_FBUF>`,
250 :ref:`framebuffer-flags`).
251
252 .. note::
253
254 This field was added in Linux 2.6.23, extending the
255 structure. However the :ref:`VIDIOC_[G|S|TRY]_FMT <VIDIOC_G_FMT>`
256 ioctls, which take a pointer to a :c:type:`v4l2_format`
257 parent structure with padding bytes at the end, are not affected.
258
259
260 .. c:type:: v4l2_clip
261
262 struct v4l2_clip [#f4]_
263 -----------------------
264
265 ``struct v4l2_rect c``
266 Coordinates of the clipping rectangle, relative to the top, left
267 corner of the frame buffer. Only window pixels *outside* all
268 clipping rectangles are displayed.
269
270 ``struct v4l2_clip * next``
271 Pointer to the next clipping rectangle, ``NULL`` when this is the last
272 rectangle. Drivers ignore this field, it cannot be used to pass a
273 linked list of clipping rectangles.
274
275
276 .. c:type:: v4l2_rect
277
278 struct v4l2_rect
279 ----------------
280
281 ``__s32 left``
282 Horizontal offset of the top, left corner of the rectangle, in
283 pixels.
284
285 ``__s32 top``
286 Vertical offset of the top, left corner of the rectangle, in pixels.
287 Offsets increase to the right and down.
288
289 ``__u32 width``
290 Width of the rectangle, in pixels.
291
292 ``__u32 height``
293 Height of the rectangle, in pixels.
294
295
296 Enabling Overlay
297 ================
298
299 To start or stop the frame buffer overlay applications call the
300 :ref:`VIDIOC_OVERLAY` ioctl.
301
302 .. [#f1]
303 In the opinion of the designers of this API, no driver writer taking
304 the efforts to support simultaneous capturing and overlay will
305 restrict this ability by requiring a single file descriptor, as in
306 V4L and earlier versions of V4L2. Making this optional means
307 applications depending on two file descriptors need backup routines
308 to be compatible with all drivers, which is considerable more work
309 than using two fds in applications which do not. Also two fd's fit
310 the general concept of one file descriptor for each logical stream.
311 Hence as a complexity trade-off drivers *must* support two file
312 descriptors and *may* support single fd operation.
313
314 .. [#f2]
315 Should we require ``w.width`` to be a multiple of eight?
316
317 .. [#f3]
318 When the image is written into frame buffer memory it will be
319 undesirable if the driver clips out less pixels than expected,
320 because the application and graphics system are not aware these
321 regions need to be refreshed. The driver should clip out more pixels
322 or not write the image at all.
323
324 .. [#f4]
325 The X Window system defines "regions" which are vectors of ``struct
326 BoxRec { short x1, y1, x2, y2; }`` with ``width = x2 - x1`` and
327 ``height = y2 - y1``, so one cannot pass X11 clip lists directly.