Debian Codegraph @

doc/spec/source/programming_guide/VPL_prg_hw.rst

raw
1 .. SPDX-FileCopyrightText: 2019-2020 Intel Corporation
2 ..
3 .. SPDX-License-Identifier: CC-BY-4.0
4 ..
5 Intel® Video Processing Library (Intel® VPL)
6 .. _hw-acceleration:
7
8 =====================
9 Hardware Acceleration
10 =====================
11
12 |vpl_short_name| provides a new model for working with hardware acceleration while
13 continuing to support hardware acceleration in legacy mode.
14
15 --------------------------------------------
16 New Model to Work with Hardware Acceleration
17 --------------------------------------------
18
19 |vpl_short_name| API version 2.0 introduces a new memory model: internal allocation where
20 |vpl_short_name| is responsible for video memory allocation. In this mode,
21 an application is not dependent on a low-level video framework API, such as
22 DirectX\* or the VA-API, and does not need to create and set corresponding
23 low-level |vpl_short_name| primitives such as `ID3D11Device` or `VADisplay`. Instead,
24 |vpl_short_name| creates all required objects to work with hardware acceleration and video
25 surfaces internally. An application can get access to these objects using
26 :cpp:func:`MFXVideoCORE_GetHandle` or with help of the
27 :cpp:struct:`mfxFrameSurfaceInterface` interface.
28
29 This approach simplifies the |vpl_short_name| initialization, making calls to the
30 :cpp:func:`MFXVideoENCODE_QueryIOSurf`, :cpp:func:`MFXVideoDECODE_QueryIOSurf`,
31 or :cpp:func:`MFXVideoVPP_QueryIOSurf` functions optional. See
32 :ref:`Internal Memory Management <internal-mem-manage>`.
33
34 .. note:: Applications can set device handle before session creation through :cpp:func:`MFXSetConfigFilterProperty`
35 like shown in the code below:
36
37 .. literalinclude:: ../snippets/prg_hw.cpp
38 :language: c++
39 :start-after: /*beg3*/
40 :end-before: /*end3*/
41 :lineno-start: 1
42
43
44 ----------------------------------------------
45 Work with Hardware Acceleration in Legacy Mode
46 ----------------------------------------------
47
48 Work with Multiple Media Devices
49 --------------------------------
50
51 If your system has multiple graphics adapters, you may need hints on which
52 adapter is better suited to process a particular workload. The legacy mode of
53 |vpl_short_name| provides a helper API to select the most suitable adapter for your
54 workload based on the provided workload description.
55
56 .. important:: :cpp:func:`MFXQueryAdapters`, :cpp:func:`MFXQueryAdaptersDecode`,
57 and :cpp:func:`MFXQueryAdaptersNumber` are deprecated starting
58 from API 2.9. Applications should use :cpp:func:`MFXEnumImplementations`
59 and :cpp:func:`MFXSetConfigFilterProperty` to query adapter
60 capabilities and to select a suitable adapter for the input workload.
61
62 The following example shows workload initialization on a discrete adapter
63 in legacy mode:
64
65
66 .. literalinclude:: ../snippets/prg_hw.cpp
67 :language: c++
68 :start-after: /*beg1*/
69 :end-before: /*end1*/
70 :lineno-start: 1
71
72 The example shows that after obtaining the adapter list with
73 :cpp:func:`MFXQueryAdapters`, further initialization of :cpp:type:`mfxSession`
74 is performed in the regular way. The specific adapter is selected using
75 the :cpp:enumerator:`MFX_IMPL_HARDWARE`, :cpp:enumerator:`MFX_IMPL_HARDWARE2`,
76 :cpp:enumerator:`MFX_IMPL_HARDWARE3`, or :cpp:enumerator:`MFX_IMPL_HARDWARE4`
77 values of :cpp:type:`mfxIMPL`.
78
79 The following example shows the use of :cpp:func:`MFXQueryAdapters` for querying
80 the most suitable adapter for a particular encode workload:
81
82 .. literalinclude:: ../snippets/prg_hw.cpp
83 :language: c++
84 :start-after: /*beg2*/
85 :end-before: /*end2*/
86 :lineno-start: 1
87
88 See the :cpp:func:`MFXQueryAdapters` description for adapter priority rules.
89
90 Work with Video Memory
91 ----------------------
92
93 To fully utilize the |vpl_short_name| acceleration capability, the application should support
94 OS specific infrastructures. If using Microsoft\* Windows\*, the application
95 should support Microsoft DirectX\*. If using Linux\*, the application should
96 support the VA-API for Linux.
97
98 The hardware acceleration support in an application consists of video memory
99 support and acceleration device support.
100
101 Depending on the usage model, the application can use video memory at different
102 stages in the pipeline. Three major scenarios are shown in the following
103 diagrams:
104
105 .. graphviz::
106
107 digraph {
108 rankdir=LR;
109 labelloc="t";
110 label="Intel® VPL functions interconnection";
111 F1 [shape=octagon label="Intel® VPL Function"];
112 F2 [shape=octagon label="Intel® VPL Function"];
113 F1->F2 [ label="Video Memory" ];
114 }
115
116 |
117
118 .. graphviz::
119
120 digraph {
121 rankdir=LR;
122 labelloc="t";
123 label="Video memory as output";
124 F3 [shape=octagon label="Intel® VPL Function"];
125 F4 [shape=octagon label="Application" fillcolor=lightgrey];
126 F3->F4 [ label="Video Memory" ];
127 }
128
129 |
130
131 .. graphviz::
132
133 digraph {
134 rankdir=LR;
135 labelloc="t";
136 label="Video memory as input";
137 F5 [shape=octagon label="Application"];
138 F6 [shape=octagon label="Intel® VPL Function"];
139 F5->F6 [ label="Video Memory" ];
140 }
141
142 |
143
144 The application must use the :cpp:member:`mfxVideoParam::IOPattern` field to
145 indicate the I/O access pattern during initialization. Subsequent function calls
146 must follow this access pattern. For example, if a function operates on video
147 memory surfaces at both input and output, the
148 application must specify the access pattern `IOPattern` at initialization in
149 :cpp:enumerator:`MFX_IOPATTERN_IN_VIDEO_MEMORY` for input and
150 :cpp:enumerator:`MFX_IOPATTERN_OUT_VIDEO_MEMORY` for output. This particular I/O
151 access pattern must not change inside the **Init** **-** **Close** sequence.
152
153 Initialization of any hardware accelerated |vpl_short_name| component requires the
154 acceleration device handle. This handle is also used by the |vpl_short_name| component to
155 query hardware capabilities. The application can share its device with |vpl_short_name|
156 by passing the device handle through the :cpp:func:`MFXVideoCORE_SetHandle`
157 function. It is recommended to share the handle before any actual usage of |vpl_short_name|.
158 The shared handle must remain valid until after the application calls :cpp:func:`MFXClose`.
159
160 .. _work_ms_directx_app:
161
162 Work with Microsoft DirectX\* Applications
163 ------------------------------------------
164
165 |vpl_short_name| supports two different infrastructures for hardware acceleration on the
166 Microsoft Windows OS: the Direct3D\* 9 DXVA2 and Direct3D 11 Video API. If Direct3D 9
167 DXVA2 is used for hardware acceleration, the application should use the
168 `IDirect3DDeviceManager9` interface as the acceleration device handle. If the
169 Direct3D 11 Video API is used for hardware acceleration, the application should
170 use the `ID3D11Device` interface as the acceleration device handle.
171
172 The application should share one of these interfaces with |vpl_short_name| through the
173 :cpp:func:`MFXVideoCORE_SetHandle` function. If the application does not provide
174 the interface, then |vpl_short_name| creates its own internal acceleration device. As a result,
175 |vpl_short_name| input and output will be limited to system memory only for the external allocation mode, which will reduce |vpl_short_name| performance. If |vpl_short_name| fails to create a valid acceleration device,
176 then |vpl_short_name| cannot proceed with hardware acceleration and returns an error status to the
177 application.
178
179 .. note:: It is recommended to work in the internal allocation mode if the application does not provide the `IDirect3DDeviceManager9` or `ID3D11Device` interface.
180
181 The application must create the Direct3D 9 device with the flag
182 ``D3DCREATE_MULTITHREADED``. The flag ``D3DCREATE_FPU_PRESERVE`` is also
183 recommended. This influences floating-point calculations, including PTS values.
184
185 The application must also set multi-threading mode for the Direct3D 11 device.
186 The following example shows how to set multi-threading mode for a Direct3D 11
187 device:
188
189 .. code-block:: c++
190 :lineno-start: 1
191
192 ID3D11Device *pD11Device;
193 ID3D11DeviceContext *pD11Context;
194 ID3D10Multithread *pD10Multithread;
195
196 pD11Device->GetImmediateContext(&pD11Context);
197 pD11Context->QueryInterface(IID_ID3D10Multithread, &pD10Multithread);
198 pD10Multithread->SetMultithreadProtected(true);
199
200 During hardware acceleration, if a Direct3D “device lost” event occurs, the |vpl_short_name|
201 operation terminates with the :cpp:enumerator:`mfxStatus::MFX_ERR_DEVICE_LOST`
202 return status. If the application provided the Direct3D device handle, the
203 application must reset the Direct3D device.
204
205 When the |vpl_short_name| decoder creates auxiliary devices for hardware acceleration, it
206 must allocate the list of Direct3D surfaces for I/O access, also known as the
207 surface chain, and pass the surface chain as part of the device creation command.
208 In most cases, the surface chain is the frame surface pool mentioned in the
209 :ref:`Frame Surface Locking <frame-surface-manag>` section.
210
211 The application passes the surface chain to the |vpl_short_name| component **Init** function
212 through a |vpl_short_name| external allocator callback. See the
213 :ref:`Memory Allocation and External Allocators <mem-alloc-ext-alloc>` section
214 for details.
215
216 Only the decoder **Init** function requests the external surface chain from the
217 application and uses it for auxiliary device creation. Encoder and VPP **Init**
218 functions may only request internal surfaces. See the
219 :ref:`ExtMemFrameType enumerator <extmemframetype>` for more details about
220 different memory types.
221
222 Depending on configuration parameters, |vpl_short_name| requires different surface types.
223 It is strongly recommended to call the :cpp:func:`MFXVideoENCODE_QueryIOSurf`
224 function, the :cpp:func:`MFXVideoDECODE_QueryIOSurf` function, or the
225 :cpp:func:`MFXVideoVPP_QueryIOSurf` function to determine the appropriate type in
226 the external allocation mode.
227
228 Work with VA-API Applications
229 -----------------------------
230
231 |vpl_short_name| supports the VA-API infrastructure for hardware acceleration on Linux.
232 The application should use the `VADisplay` interface as the acceleration device
233 handle for this infrastructure and share it with |vpl_short_name| through the
234 :cpp:func:`MFXVideoCORE_SetHandle` function.
235
236 The following example shows how to obtain the VA display from the X Window System:
237
238 .. code-block::
239 :lineno-start: 1
240
241 Display *x11_display;
242 VADisplay va_display;
243
244 x11_display = XOpenDisplay(current_display);
245 va_display = vaGetDisplay(x11_display);
246
247 MFXVideoCORE_SetHandle(session, MFX_HANDLE_VA_DISPLAY, (mfxHDL) va_display);
248
249 The following example shows how to obtain the VA display from the Direct
250 Rendering Manager:
251
252 .. code-block::
253 :lineno-start: 1
254
255 int card;
256 VADisplay va_display;
257
258 card = open("/dev/dri/card0", O_RDWR); /* primary card */
259 va_display = vaGetDisplayDRM(card);
260 vaInitialize(va_display, &major_version, &minor_version);
261
262 MFXVideoCORE_SetHandle(session, MFX_HANDLE_VA_DISPLAY, (mfxHDL) va_display);
263
264 When the |vpl_short_name| decoder creates a hardware acceleration device, it must allocate
265 the list of video memory surfaces for I/O access, also known as the surface chain,
266 and pass the surface chain as part of the device creation command. The
267 application passes the surface chain to the |vpl_short_name| component **Init** function
268 through a |vpl_short_name| external allocator callback. See the
269 :ref:`Memory Allocation and External Allocators <mem-alloc-ext-alloc>` section
270 for details.
271 Starting from |vpl_short_name| API version 2.0, |vpl_short_name| creates its own surface chain if an
272 external allocator is not set. See the :ref`New Model to work with Hardware Acceleration <hw-acceleration>`
273 section for details.
274
275
276 .. note:: The VA-API does not define any surface types and the application can
277 use either :cpp:enumerator:`MFX_MEMTYPE_VIDEO_MEMORY_DECODER_TARGET`
278 or :cpp:enumerator:`MFX_MEMTYPE_VIDEO_MEMORY_PROCESSOR_TARGET` to indicate data in video
279 memory.
280