| 1 |
.. SPDX-FileCopyrightText: 2019-2020 Intel Corporation |
| 2 |
.. |
| 3 |
.. SPDX-License-Identifier: CC-BY-4.0 |
| 4 |
.. |
| 5 |
Intel® Video Processing Library (Intel® VPL) |
| 6 |
.. _hw-acceleration: |
| 7 |
|
| 8 |
===================== |
| 9 |
Hardware Acceleration |
| 10 |
===================== |
| 11 |
|
| 12 |
|vpl_short_name| provides a new model for working with hardware acceleration while |
| 13 |
continuing to support hardware acceleration in legacy mode. |
| 14 |
|
| 15 |
-------------------------------------------- |
| 16 |
New Model to Work with Hardware Acceleration |
| 17 |
-------------------------------------------- |
| 18 |
|
| 19 |
|vpl_short_name| API version 2.0 introduces a new memory model: internal allocation where |
| 20 |
|vpl_short_name| is responsible for video memory allocation. In this mode, |
| 21 |
an application is not dependent on a low-level video framework API, such as |
| 22 |
DirectX\* or the VA-API, and does not need to create and set corresponding |
| 23 |
low-level |vpl_short_name| primitives such as `ID3D11Device` or `VADisplay`. Instead, |
| 24 |
|vpl_short_name| creates all required objects to work with hardware acceleration and video |
| 25 |
surfaces internally. An application can get access to these objects using |
| 26 |
:cpp:func:`MFXVideoCORE_GetHandle` or with help of the |
| 27 |
:cpp:struct:`mfxFrameSurfaceInterface` interface. |
| 28 |
|
| 29 |
This approach simplifies the |vpl_short_name| initialization, making calls to the |
| 30 |
:cpp:func:`MFXVideoENCODE_QueryIOSurf`, :cpp:func:`MFXVideoDECODE_QueryIOSurf`, |
| 31 |
or :cpp:func:`MFXVideoVPP_QueryIOSurf` functions optional. See |
| 32 |
:ref:`Internal Memory Management <internal-mem-manage>`. |
| 33 |
|
| 34 |
.. note:: Applications can set device handle before session creation through :cpp:func:`MFXSetConfigFilterProperty` |
| 35 |
like shown in the code below: |
| 36 |
|
| 37 |
.. literalinclude:: ../snippets/prg_hw.cpp |
| 38 |
:language: c++ |
| 39 |
:start-after: /*beg3*/ |
| 40 |
:end-before: /*end3*/ |
| 41 |
:lineno-start: 1 |
| 42 |
|
| 43 |
|
| 44 |
---------------------------------------------- |
| 45 |
Work with Hardware Acceleration in Legacy Mode |
| 46 |
---------------------------------------------- |
| 47 |
|
| 48 |
Work with Multiple Media Devices |
| 49 |
-------------------------------- |
| 50 |
|
| 51 |
If your system has multiple graphics adapters, you may need hints on which |
| 52 |
adapter is better suited to process a particular workload. The legacy mode of |
| 53 |
|vpl_short_name| provides a helper API to select the most suitable adapter for your |
| 54 |
workload based on the provided workload description. |
| 55 |
|
| 56 |
.. important:: :cpp:func:`MFXQueryAdapters`, :cpp:func:`MFXQueryAdaptersDecode`, |
| 57 |
and :cpp:func:`MFXQueryAdaptersNumber` are deprecated starting |
| 58 |
from API 2.9. Applications should use :cpp:func:`MFXEnumImplementations` |
| 59 |
and :cpp:func:`MFXSetConfigFilterProperty` to query adapter |
| 60 |
capabilities and to select a suitable adapter for the input workload. |
| 61 |
|
| 62 |
The following example shows workload initialization on a discrete adapter |
| 63 |
in legacy mode: |
| 64 |
|
| 65 |
|
| 66 |
.. literalinclude:: ../snippets/prg_hw.cpp |
| 67 |
:language: c++ |
| 68 |
:start-after: /*beg1*/ |
| 69 |
:end-before: /*end1*/ |
| 70 |
:lineno-start: 1 |
| 71 |
|
| 72 |
The example shows that after obtaining the adapter list with |
| 73 |
:cpp:func:`MFXQueryAdapters`, further initialization of :cpp:type:`mfxSession` |
| 74 |
is performed in the regular way. The specific adapter is selected using |
| 75 |
the :cpp:enumerator:`MFX_IMPL_HARDWARE`, :cpp:enumerator:`MFX_IMPL_HARDWARE2`, |
| 76 |
:cpp:enumerator:`MFX_IMPL_HARDWARE3`, or :cpp:enumerator:`MFX_IMPL_HARDWARE4` |
| 77 |
values of :cpp:type:`mfxIMPL`. |
| 78 |
|
| 79 |
The following example shows the use of :cpp:func:`MFXQueryAdapters` for querying |
| 80 |
the most suitable adapter for a particular encode workload: |
| 81 |
|
| 82 |
.. literalinclude:: ../snippets/prg_hw.cpp |
| 83 |
:language: c++ |
| 84 |
:start-after: /*beg2*/ |
| 85 |
:end-before: /*end2*/ |
| 86 |
:lineno-start: 1 |
| 87 |
|
| 88 |
See the :cpp:func:`MFXQueryAdapters` description for adapter priority rules. |
| 89 |
|
| 90 |
Work with Video Memory |
| 91 |
---------------------- |
| 92 |
|
| 93 |
To fully utilize the |vpl_short_name| acceleration capability, the application should support |
| 94 |
OS specific infrastructures. If using Microsoft\* Windows\*, the application |
| 95 |
should support Microsoft DirectX\*. If using Linux\*, the application should |
| 96 |
support the VA-API for Linux. |
| 97 |
|
| 98 |
The hardware acceleration support in an application consists of video memory |
| 99 |
support and acceleration device support. |
| 100 |
|
| 101 |
Depending on the usage model, the application can use video memory at different |
| 102 |
stages in the pipeline. Three major scenarios are shown in the following |
| 103 |
diagrams: |
| 104 |
|
| 105 |
.. graphviz:: |
| 106 |
|
| 107 |
digraph { |
| 108 |
rankdir=LR; |
| 109 |
labelloc="t"; |
| 110 |
label="Intel® VPL functions interconnection"; |
| 111 |
F1 [shape=octagon label="Intel® VPL Function"]; |
| 112 |
F2 [shape=octagon label="Intel® VPL Function"]; |
| 113 |
F1->F2 [ label="Video Memory" ]; |
| 114 |
} |
| 115 |
|
| 116 |
| |
| 117 |
|
| 118 |
.. graphviz:: |
| 119 |
|
| 120 |
digraph { |
| 121 |
rankdir=LR; |
| 122 |
labelloc="t"; |
| 123 |
label="Video memory as output"; |
| 124 |
F3 [shape=octagon label="Intel® VPL Function"]; |
| 125 |
F4 [shape=octagon label="Application" fillcolor=lightgrey]; |
| 126 |
F3->F4 [ label="Video Memory" ]; |
| 127 |
} |
| 128 |
|
| 129 |
| |
| 130 |
|
| 131 |
.. graphviz:: |
| 132 |
|
| 133 |
digraph { |
| 134 |
rankdir=LR; |
| 135 |
labelloc="t"; |
| 136 |
label="Video memory as input"; |
| 137 |
F5 [shape=octagon label="Application"]; |
| 138 |
F6 [shape=octagon label="Intel® VPL Function"]; |
| 139 |
F5->F6 [ label="Video Memory" ]; |
| 140 |
} |
| 141 |
|
| 142 |
| |
| 143 |
|
| 144 |
The application must use the :cpp:member:`mfxVideoParam::IOPattern` field to |
| 145 |
indicate the I/O access pattern during initialization. Subsequent function calls |
| 146 |
must follow this access pattern. For example, if a function operates on video |
| 147 |
memory surfaces at both input and output, the |
| 148 |
application must specify the access pattern `IOPattern` at initialization in |
| 149 |
:cpp:enumerator:`MFX_IOPATTERN_IN_VIDEO_MEMORY` for input and |
| 150 |
:cpp:enumerator:`MFX_IOPATTERN_OUT_VIDEO_MEMORY` for output. This particular I/O |
| 151 |
access pattern must not change inside the **Init** **-** **Close** sequence. |
| 152 |
|
| 153 |
Initialization of any hardware accelerated |vpl_short_name| component requires the |
| 154 |
acceleration device handle. This handle is also used by the |vpl_short_name| component to |
| 155 |
query hardware capabilities. The application can share its device with |vpl_short_name| |
| 156 |
by passing the device handle through the :cpp:func:`MFXVideoCORE_SetHandle` |
| 157 |
function. It is recommended to share the handle before any actual usage of |vpl_short_name|. |
| 158 |
The shared handle must remain valid until after the application calls :cpp:func:`MFXClose`. |
| 159 |
|
| 160 |
.. _work_ms_directx_app: |
| 161 |
|
| 162 |
Work with Microsoft DirectX\* Applications |
| 163 |
------------------------------------------ |
| 164 |
|
| 165 |
|vpl_short_name| supports two different infrastructures for hardware acceleration on the |
| 166 |
Microsoft Windows OS: the Direct3D\* 9 DXVA2 and Direct3D 11 Video API. If Direct3D 9 |
| 167 |
DXVA2 is used for hardware acceleration, the application should use the |
| 168 |
`IDirect3DDeviceManager9` interface as the acceleration device handle. If the |
| 169 |
Direct3D 11 Video API is used for hardware acceleration, the application should |
| 170 |
use the `ID3D11Device` interface as the acceleration device handle. |
| 171 |
|
| 172 |
The application should share one of these interfaces with |vpl_short_name| through the |
| 173 |
:cpp:func:`MFXVideoCORE_SetHandle` function. If the application does not provide |
| 174 |
the interface, then |vpl_short_name| creates its own internal acceleration device. As a result, |
| 175 |
|vpl_short_name| input and output will be limited to system memory only for the external allocation mode, which will reduce |vpl_short_name| performance. If |vpl_short_name| fails to create a valid acceleration device, |
| 176 |
then |vpl_short_name| cannot proceed with hardware acceleration and returns an error status to the |
| 177 |
application. |
| 178 |
|
| 179 |
.. note:: It is recommended to work in the internal allocation mode if the application does not provide the `IDirect3DDeviceManager9` or `ID3D11Device` interface. |
| 180 |
|
| 181 |
The application must create the Direct3D 9 device with the flag |
| 182 |
``D3DCREATE_MULTITHREADED``. The flag ``D3DCREATE_FPU_PRESERVE`` is also |
| 183 |
recommended. This influences floating-point calculations, including PTS values. |
| 184 |
|
| 185 |
The application must also set multi-threading mode for the Direct3D 11 device. |
| 186 |
The following example shows how to set multi-threading mode for a Direct3D 11 |
| 187 |
device: |
| 188 |
|
| 189 |
.. code-block:: c++ |
| 190 |
:lineno-start: 1 |
| 191 |
|
| 192 |
ID3D11Device *pD11Device; |
| 193 |
ID3D11DeviceContext *pD11Context; |
| 194 |
ID3D10Multithread *pD10Multithread; |
| 195 |
|
| 196 |
pD11Device->GetImmediateContext(&pD11Context); |
| 197 |
pD11Context->QueryInterface(IID_ID3D10Multithread, &pD10Multithread); |
| 198 |
pD10Multithread->SetMultithreadProtected(true); |
| 199 |
|
| 200 |
During hardware acceleration, if a Direct3D “device lost” event occurs, the |vpl_short_name| |
| 201 |
operation terminates with the :cpp:enumerator:`mfxStatus::MFX_ERR_DEVICE_LOST` |
| 202 |
return status. If the application provided the Direct3D device handle, the |
| 203 |
application must reset the Direct3D device. |
| 204 |
|
| 205 |
When the |vpl_short_name| decoder creates auxiliary devices for hardware acceleration, it |
| 206 |
must allocate the list of Direct3D surfaces for I/O access, also known as the |
| 207 |
surface chain, and pass the surface chain as part of the device creation command. |
| 208 |
In most cases, the surface chain is the frame surface pool mentioned in the |
| 209 |
:ref:`Frame Surface Locking <frame-surface-manag>` section. |
| 210 |
|
| 211 |
The application passes the surface chain to the |vpl_short_name| component **Init** function |
| 212 |
through a |vpl_short_name| external allocator callback. See the |
| 213 |
:ref:`Memory Allocation and External Allocators <mem-alloc-ext-alloc>` section |
| 214 |
for details. |
| 215 |
|
| 216 |
Only the decoder **Init** function requests the external surface chain from the |
| 217 |
application and uses it for auxiliary device creation. Encoder and VPP **Init** |
| 218 |
functions may only request internal surfaces. See the |
| 219 |
:ref:`ExtMemFrameType enumerator <extmemframetype>` for more details about |
| 220 |
different memory types. |
| 221 |
|
| 222 |
Depending on configuration parameters, |vpl_short_name| requires different surface types. |
| 223 |
It is strongly recommended to call the :cpp:func:`MFXVideoENCODE_QueryIOSurf` |
| 224 |
function, the :cpp:func:`MFXVideoDECODE_QueryIOSurf` function, or the |
| 225 |
:cpp:func:`MFXVideoVPP_QueryIOSurf` function to determine the appropriate type in |
| 226 |
the external allocation mode. |
| 227 |
|
| 228 |
Work with VA-API Applications |
| 229 |
----------------------------- |
| 230 |
|
| 231 |
|vpl_short_name| supports the VA-API infrastructure for hardware acceleration on Linux. |
| 232 |
The application should use the `VADisplay` interface as the acceleration device |
| 233 |
handle for this infrastructure and share it with |vpl_short_name| through the |
| 234 |
:cpp:func:`MFXVideoCORE_SetHandle` function. |
| 235 |
|
| 236 |
The following example shows how to obtain the VA display from the X Window System: |
| 237 |
|
| 238 |
.. code-block:: |
| 239 |
:lineno-start: 1 |
| 240 |
|
| 241 |
Display *x11_display; |
| 242 |
VADisplay va_display; |
| 243 |
|
| 244 |
x11_display = XOpenDisplay(current_display); |
| 245 |
va_display = vaGetDisplay(x11_display); |
| 246 |
|
| 247 |
MFXVideoCORE_SetHandle(session, MFX_HANDLE_VA_DISPLAY, (mfxHDL) va_display); |
| 248 |
|
| 249 |
The following example shows how to obtain the VA display from the Direct |
| 250 |
Rendering Manager: |
| 251 |
|
| 252 |
.. code-block:: |
| 253 |
:lineno-start: 1 |
| 254 |
|
| 255 |
int card; |
| 256 |
VADisplay va_display; |
| 257 |
|
| 258 |
card = open("/dev/dri/card0", O_RDWR); /* primary card */ |
| 259 |
va_display = vaGetDisplayDRM(card); |
| 260 |
vaInitialize(va_display, &major_version, &minor_version); |
| 261 |
|
| 262 |
MFXVideoCORE_SetHandle(session, MFX_HANDLE_VA_DISPLAY, (mfxHDL) va_display); |
| 263 |
|
| 264 |
When the |vpl_short_name| decoder creates a hardware acceleration device, it must allocate |
| 265 |
the list of video memory surfaces for I/O access, also known as the surface chain, |
| 266 |
and pass the surface chain as part of the device creation command. The |
| 267 |
application passes the surface chain to the |vpl_short_name| component **Init** function |
| 268 |
through a |vpl_short_name| external allocator callback. See the |
| 269 |
:ref:`Memory Allocation and External Allocators <mem-alloc-ext-alloc>` section |
| 270 |
for details. |
| 271 |
Starting from |vpl_short_name| API version 2.0, |vpl_short_name| creates its own surface chain if an |
| 272 |
external allocator is not set. See the :ref`New Model to work with Hardware Acceleration <hw-acceleration>` |
| 273 |
section for details. |
| 274 |
|
| 275 |
|
| 276 |
.. note:: The VA-API does not define any surface types and the application can |
| 277 |
use either :cpp:enumerator:`MFX_MEMTYPE_VIDEO_MEMORY_DECODER_TARGET` |
| 278 |
or :cpp:enumerator:`MFX_MEMTYPE_VIDEO_MEMORY_PROCESSOR_TARGET` to indicate data in video |
| 279 |
memory. |
| 280 |
|