| 1 |
.. SPDX-FileCopyrightText: 2019-2020 Intel Corporation |
| 2 |
.. |
| 3 |
.. SPDX-License-Identifier: CC-BY-4.0 |
| 4 |
|
| 5 |
.. _decoding-proc: |
| 6 |
|
| 7 |
=================== |
| 8 |
Decoding Procedures |
| 9 |
=================== |
| 10 |
|
| 11 |
There are several approaches to decode video frames. The first one is based on |
| 12 |
the internal allocation mechanism presented here: |
| 13 |
|
| 14 |
.. literalinclude:: ../snippets/prg_decoding.c |
| 15 |
:language: c++ |
| 16 |
:start-after: /*beg1*/ |
| 17 |
:end-before: /*end1*/ |
| 18 |
:lineno-start: 1 |
| 19 |
|
| 20 |
Note the following key points about the example: |
| 21 |
|
| 22 |
- The application calls the :cpp:func:`MFXVideoDECODE_DecodeFrameAsync` function |
| 23 |
for a decoding operation with the bitstream buffer (bits), frame surface is |
| 24 |
allocated internally by the library. |
| 25 |
|
| 26 |
.. attention:: As shown in the example above starting with API version 2.0, |
| 27 |
the application can provide NULL |
| 28 |
as the working frame surface that leads to internal memory |
| 29 |
allocation. |
| 30 |
|
| 31 |
- If decoding output is not available, the function returns a status code |
| 32 |
requesting additional bitstream input as follows: |
| 33 |
|
| 34 |
- :cpp:enumerator:`mfxStatus::MFX_ERR_MORE_DATA`: The function needs |
| 35 |
additional bitstream input. The existing buffer contains less than a frame's |
| 36 |
worth of bitstream data. |
| 37 |
|
| 38 |
- Upon successful decoding, the :cpp:func:`MFXVideoDECODE_DecodeFrameAsync` |
| 39 |
function returns :cpp:enumerator:`mfxStatus::MFX_ERR_NONE`. However, the |
| 40 |
decoded frame data (identified by the surface_out pointer) is not yet |
| 41 |
available because the :cpp:func:`MFXVideoDECODE_DecodeFrameAsync` function is |
| 42 |
asynchronous. The application must use the |
| 43 |
:cpp:func:`MFXVideoCORE_SyncOperation` or |
| 44 |
:cpp:member:`mfxFrameSurfaceInterface::Synchronize` to synchronize the |
| 45 |
decoding operation before retrieving the decoded frame data. |
| 46 |
|
| 47 |
- At the end of the bitstream, the application continuously calls the |
| 48 |
:cpp:func:`MFXVideoDECODE_DecodeFrameAsync` function with a NULL bitstream |
| 49 |
pointer to drain any remaining frames cached within the decoder until the |
| 50 |
function returns :cpp:enumerator:`mfxStatus::MFX_ERR_MORE_DATA`. |
| 51 |
|
| 52 |
- When application completes the work with frame surface, it must call release |
| 53 |
to avoid memory leaks. |
| 54 |
|
| 55 |
The next example demonstrates how applications can use internally pre-allocated |
| 56 |
chunk of video surfaces: |
| 57 |
|
| 58 |
.. literalinclude:: ../snippets/prg_decoding.c |
| 59 |
:language: c++ |
| 60 |
:start-after: /*beg2*/ |
| 61 |
:end-before: /*end2*/ |
| 62 |
:lineno-start: 1 |
| 63 |
|
| 64 |
Here the application should use the :cpp:func:`MFXVideoDECODE_QueryIOSurf` |
| 65 |
function to obtain the number of working frame surfaces required to reorder |
| 66 |
output frames. It is also required that |
| 67 |
:cpp:func:`MFXMemory_GetSurfaceForDecode` call is done after decoder |
| 68 |
initialization. In the :cpp:func:`MFXVideoDECODE_DecodeFrameAsync` the |vpl_short_name| |
| 69 |
library increments reference counter of incoming surface frame so it is required |
| 70 |
that the application releases frame surface after the call. |
| 71 |
|
| 72 |
Another approach to decode frames is to allocate video frames on-fly with help |
| 73 |
of :cpp:func:`MFXMemory_GetSurfaceForDecode` function, feed the library and |
| 74 |
release working surface after :cpp:func:`MFXVideoDECODE_DecodeFrameAsync` call. |
| 75 |
|
| 76 |
.. attention:: Please pay attention on two release calls for surfaces: |
| 77 |
after :cpp:func:`MFXVideoDECODE_DecodeFrameAsync` to decrease |
| 78 |
reference counter of working surface returned by |
| 79 |
:cpp:func:`MFXMemory_GetSurfaceForDecode`. |
| 80 |
After :cpp:func:`MFXVideoCORE_SyncOperation` to decrease |
| 81 |
reference counter of output surface returned by |
| 82 |
:cpp:func:`MFXVideoDECODE_DecodeFrameAsync`. |
| 83 |
|
| 84 |
.. literalinclude:: ../snippets/prg_decoding.c |
| 85 |
:language: c++ |
| 86 |
:start-after: /*beg6*/ |
| 87 |
:end-before: /*end6*/ |
| 88 |
:lineno-start: 1 |
| 89 |
|
| 90 |
|
| 91 |
The following pseudo code shows the decoding procedure according to the legacy |
| 92 |
mode with external video frames allocation: |
| 93 |
|
| 94 |
.. literalinclude:: ../snippets/prg_decoding.c |
| 95 |
:language: c++ |
| 96 |
:start-after: /*beg3*/ |
| 97 |
:end-before: /*end3*/ |
| 98 |
:lineno-start: 1 |
| 99 |
|
| 100 |
Note the following key points about the example: |
| 101 |
|
| 102 |
|
| 103 |
- The application can use the :cpp:func:`MFXVideoDECODE_DecodeHeader` function |
| 104 |
to retrieve decoding initialization parameters from the bitstream. This step |
| 105 |
is optional if the data is retrievable from other sources such as an |
| 106 |
audio/video splitter. |
| 107 |
|
| 108 |
- The :cpp:func:`MFXVideoDECODE_DecodeFrameAsync` function can return following |
| 109 |
status codes in addition to the described above: |
| 110 |
|
| 111 |
- :cpp:enumerator:`mfxStatus::MFX_ERR_MORE_SURFACE`: The function needs one |
| 112 |
more frame surface to produce any output. |
| 113 |
- :cpp:enumerator:`mfxStatus::MFX_ERR_REALLOC_SURFACE`: Dynamic resolution |
| 114 |
change case - the function needs a bigger working frame surface (work). |
| 115 |
|
| 116 |
The following pseudo code shows the simplified decoding procedure: |
| 117 |
|
| 118 |
.. literalinclude:: ../snippets/prg_decoding.c |
| 119 |
:language: c++ |
| 120 |
:start-after: /*beg4*/ |
| 121 |
:end-before: /*end4*/ |
| 122 |
:lineno-start: 1 |
| 123 |
|
| 124 |
.. _simplified-decoding-procedure: |
| 125 |
|
| 126 |
|vpl_short_name| API version 2.0 introduces a new decoding approach. For simple use cases, |
| 127 |
when the user wants to decode a stream and does not want to set additional |
| 128 |
parameters, a simplified procedure for the decoder's initialization has been |
| 129 |
proposed. In this scenario it is possible to skip explicit stages of a |
| 130 |
stream's header decoding and the decoder's initialization and instead to perform |
| 131 |
these steps implicitly during decoding of the first frame. This change also |
| 132 |
requires setting the additional field :cpp:member:`mfxBitstream::CodecId` to |
| 133 |
indicate codec type. In this mode the decoder allocates |
| 134 |
:cpp:struct:`mfxFrameSurface1` internally, so users should set the input surface |
| 135 |
to zero. |
| 136 |
|
| 137 |
----------------------- |
| 138 |
Bitstream Repositioning |
| 139 |
----------------------- |
| 140 |
|
| 141 |
The application can use the following procedure for bitstream reposition during |
| 142 |
decoding: |
| 143 |
|
| 144 |
#. Use the :cpp:func:`MFXVideoDECODE_Reset` function to reset the |vpl_short_name| |
| 145 |
decoder. |
| 146 |
#. Optional: If the application maintains a sequence header that correctly |
| 147 |
decodes the bitstream at the new position, the application may insert the |
| 148 |
sequence header to the bitstream buffer. |
| 149 |
#. Append the bitstream from the new location to the bitstream buffer. |
| 150 |
#. Resume the decoding procedure. If the sequence header is not inserted in the |
| 151 |
previous steps, the |vpl_short_name| decoder searches for a new sequence header before |
| 152 |
starting decoding. |
| 153 |
|
| 154 |
----------------------- |
| 155 |
Broken Streams Handling |
| 156 |
----------------------- |
| 157 |
|
| 158 |
Robustness and the capability to handle a broken input stream is an important |
| 159 |
part of the decoder. |
| 160 |
|
| 161 |
First, the start code prefix (ITU-T\* H.264 3.148 and ITU-T H.265 3.142) is |
| 162 |
used to separate NAL units. Then all syntax elements in the bitstream are parsed |
| 163 |
and verified. If any of the elements violate the specification, the input |
| 164 |
bitstream is considered invalid and the decoder tries to re-sync (find the next |
| 165 |
start code). Subsequent decoder behavior is dependent on which syntax element is |
| 166 |
broken: |
| 167 |
|
| 168 |
* SPS header is broken: return |
| 169 |
:cpp:enumerator:`mfxStatus::MFX_ERR_INCOMPATIBLE_VIDEO_PARAM` (HEVC decoder |
| 170 |
only, AVC decoder uses last valid). |
| 171 |
* PPS header is broken: re-sync, use last valid PPS for decoding. |
| 172 |
* Slice header is broken: skip this slice, re-sync. |
| 173 |
* Slice data is broken: corruption flags are set on output surface. |
| 174 |
|
| 175 |
Many streams have IDR frames with ``frame_num != 0`` while the specification |
| 176 |
says that “If the current picture is an IDR picture, frame_num shall be equal to |
| 177 |
0” (ITU-T H.265 7.4.3). |
| 178 |
|
| 179 |
VUI is also validated, but errors do not invalidate the whole SPS. The decoder |
| 180 |
either does not use the corrupted VUI (AVC) or resets incorrect values to |
| 181 |
default (HEVC). |
| 182 |
|
| 183 |
.. note:: Some requirements are relaxed because there are many streams which |
| 184 |
violate the strict standard but can be decoded without errors. |
| 185 |
|
| 186 |
Corruption at the reference frame is spread over all inter-coded pictures that |
| 187 |
use the reference frame for prediction. To cope with this problem you must |
| 188 |
either periodically insert I-frames (intra-coded) or use the intra-refresh |
| 189 |
technique. The intra-refresh technique allows recovery from corruptions within a |
| 190 |
predefined time interval. The main point of intra-refresh is to insert a cyclic |
| 191 |
intra-coded pattern (usually a row) of macroblocks into the inter-coded |
| 192 |
pictures, restricting motion vectors accordingly. Intra-refresh is often used in |
| 193 |
combination with recovery point SEI, where the ``recovery_frame_cnt`` is derived |
| 194 |
from the intra-refresh interval. The recovery point SEI message is well |
| 195 |
described at ITU-T H.264 D.2.7 and ITU-T H.265 D.2.8. If decoding starts from AU |
| 196 |
associated with this SEI message, then the message can be used by the decoder to |
| 197 |
determine from which picture all subsequent pictures have no errors. In |
| 198 |
comparison to IDR, the recovery point message does not mark reference pictures |
| 199 |
as "unused for reference". |
| 200 |
|
| 201 |
Besides validation of syntax elements and their constraints, the decoder also |
| 202 |
uses various hints to handle broken streams: |
| 203 |
|
| 204 |
* If there are no valid slices for the current frame, then the whole frame is |
| 205 |
skipped. |
| 206 |
* The slices which violate slice segment header semantics (ITU-T H.265 7.4.7.1) |
| 207 |
are skipped. Only the ``slice_temporal_mvp_enabled_flag`` is checked for now. |
| 208 |
* Since LTR (Long Term Reference) stays at DPB until it is explicitly |
| 209 |
cleared by IDR or MMCO, the incorrect LTR could cause long standing visual |
| 210 |
artifacts. AVC decoder uses the following approaches to handle this: |
| 211 |
|
| 212 |
* When there is a DPB overflow in the case of an incorrect MMCO command that |
| 213 |
marks the reference picture as LT, the operation is rolled back. |
| 214 |
* An IDR frame with ``frame_num != 0`` can’t be LTR. |
| 215 |
|
| 216 |
* If the decoder detects frame gapping, it inserts "fake"’" (marked as |
| 217 |
non-existing) frames, updates FrameNumWrap (ITU-T H.264 8.2.4.1) for reference |
| 218 |
frames, and applies the Sliding Window (ITU-T H.264 8.2.5.3) marking process. |
| 219 |
Fake frames are marked as reference, but since they are marked as |
| 220 |
non-existing, they are not used for inter-prediction. |
| 221 |
|
| 222 |
-------------------- |
| 223 |
VP8 Specific Details |
| 224 |
-------------------- |
| 225 |
|
| 226 |
Unlike other |vpl_short_name| supported decoders, VP8 can accept only a complete frame as |
| 227 |
input. The application should provide the complete frame accompanied by the |
| 228 |
:cpp:enumerator:`MFX_BITSTREAM_COMPLETE_FRAME` flag. This is the single specific |
| 229 |
difference. |
| 230 |
|
| 231 |
---- |
| 232 |
JPEG |
| 233 |
---- |
| 234 |
|
| 235 |
The application can use the same decoding procedures for JPEG/motion JPEG |
| 236 |
decoding, as shown in the following pseudo code: |
| 237 |
|
| 238 |
.. code-block:: c++ |
| 239 |
|
| 240 |
// optional; retrieve initialization parameters |
| 241 |
MFXVideoDECODE_DecodeHeader(...); |
| 242 |
// decoder initialization |
| 243 |
MFXVideoDECODE_Init(...); |
| 244 |
// single frame/picture decoding |
| 245 |
MFXVideoDECODE_DecodeFrameAsync(...); |
| 246 |
MFXVideoCORE_SyncOperation(...); |
| 247 |
// optional; retrieve meta-data |
| 248 |
MFXVideoDECODE_GetUserData(...); |
| 249 |
// close |
| 250 |
MFXVideoDECODE_Close(...); |
| 251 |
|
| 252 |
|
| 253 |
The :cpp:func:`MFXVideoDECODE_Query` function will return |
| 254 |
:cpp:enumerator:`mfxStatus::MFX_ERR_UNSUPPORTED` if the input bitstream contains |
| 255 |
unsupported features. |
| 256 |
|
| 257 |
For still picture JPEG decoding, the input can be any JPEG bitstreams that |
| 258 |
conform to the ITU-T Recommendation T.81 with an EXIF or JFIF header. For |
| 259 |
motion JPEG decoding, the input can be any JPEG bitstreams that conform to the |
| 260 |
ITU-T Recommendation T.81. |
| 261 |
|
| 262 |
Unlike other |vpl_short_name| decoders, JPEG decoding supports three different output |
| 263 |
color formats: :term:`NV12`, :term:`YUY2`, and :term:`RGB32`. This support |
| 264 |
sometimes requires internal color conversion and more complicated |
| 265 |
initialization. The color format of the input bitstream is described by the |
| 266 |
:cpp:member:`mfxInfoMFX::JPEGChromaFormat` and |
| 267 |
:cpp:member:`mfxInfoMFX::JPEGColorFormat` fields. The |
| 268 |
:cpp:func:`MFXVideoDECODE_DecodeHeader` function usually fills them in. If the |
| 269 |
JPEG bitstream does not contains color format information, the application |
| 270 |
should provide it. Output color format is described by general |vpl_short_name| |
| 271 |
parameters: the :cpp:member:`mfxFrameInfo::FourCC` and |
| 272 |
:cpp:member:`mfxFrameInfo::ChromaFormat` fields. |
| 273 |
|
| 274 |
Motion JPEG supports interlaced content by compressing each field (a half-height |
| 275 |
frame) individually. This behavior is incompatible with the rest of the |vpl_short_name| |
| 276 |
transcoding pipeline, where |vpl_short_name| requires fields to be in odd and even lines |
| 277 |
of the same frame surface. The decoding procedure is modified as follows: |
| 278 |
|
| 279 |
- The application calls the :cpp:func:`MFXVideoDECODE_DecodeHeader` function |
| 280 |
with the first field JPEG bitstream to retrieve initialization parameters. |
| 281 |
- The application initializes the |vpl_short_name| JPEG decoder with the following |
| 282 |
settings: |
| 283 |
|
| 284 |
- The ``PicStruct`` field of the :cpp:struct:`mfxVideoParam` structure set to |
| 285 |
the correct interlaced type, :cpp:enumerator:`MFX_PICSTRUCT_FIELD_TFF` or |
| 286 |
:cpp:enumerator:`MFX_PICSTRUCT_FIELD_BFF`, from the motion JPEG header. |
| 287 |
- Double the ``Height`` field in the :cpp:struct:`mfxVideoParam` structure as |
| 288 |
the value returned by the :cpp:func:`MFXVideoDECODE_DecodeHeader` function |
| 289 |
describes only the first field. The actual frame surface should contain both |
| 290 |
fields. |
| 291 |
|
| 292 |
- During decoding, the application sends both fields for decoding in the |
| 293 |
same :cpp:struct:`mfxBitstream`. The application should also set |
| 294 |
:cpp:member:`mfxBitstream::DataFlag` to |
| 295 |
:cpp:enumerator:`MFX_BITSTREAM_COMPLETE_FRAME`. |vpl_short_name| decodes both fields |
| 296 |
and combines them into odd and even lines according to |vpl_short_name| convention. |
| 297 |
|
| 298 |
By default, the :cpp:func:`MFXVideoDECODE_DecodeHeader` function returns the |
| 299 |
``Rotation`` parameter so that after rotation, the pixel at the |
| 300 |
first row and first column is at the top left. The application can overwrite the |
| 301 |
default rotation before calling :cpp:func:`MFXVideoDECODE_Init`. |
| 302 |
|
| 303 |
The application may specify Huffman and quantization tables during decoder |
| 304 |
initialization by attaching :cpp:struct:`mfxExtJPEGQuantTables` and |
| 305 |
:cpp:struct:`mfxExtJPEGHuffmanTables` buffers to the :cpp:struct:`mfxVideoParam` |
| 306 |
structure. In this case, the decoder ignores tables from bitstream and uses |
| 307 |
the tables specified by the application. The application can also retrieve these |
| 308 |
tables by attaching the same buffers to :cpp:struct:`mfxVideoParam` and calling |
| 309 |
:cpp:func:`MFXVideoDECODE_GetVideoParam` or |
| 310 |
:cpp:func:`MFXVideoDECODE_DecodeHeader` functions. |
| 311 |
|
| 312 |
------------------------- |
| 313 |
Multi-view Video Decoding |
| 314 |
------------------------- |
| 315 |
|
| 316 |
The |vpl_short_name| MVC decoder operates on complete MVC streams that contain all |
| 317 |
view and temporal configurations. The application can configure the |vpl_short_name| |
| 318 |
decoder to generate a subset at the decoding output. To do this, the application |
| 319 |
must understand the stream structure and use the stream information to configure |
| 320 |
the decoder for target views. |
| 321 |
|
| 322 |
The decoder initialization procedure is as follows: |
| 323 |
|
| 324 |
#. The application calls the :cpp:func:`MFXVideoDECODE_DecodeHeader` function to |
| 325 |
obtain the stream structural information. This is done in two steps: |
| 326 |
|
| 327 |
#. The application calls the :cpp:func:`MFXVideoDECODE_DecodeHeader` function |
| 328 |
with the :cpp:struct:`mfxExtMVCSeqDesc` structure attached to the |
| 329 |
:cpp:struct:`mfxVideoParam` structure. At this point, do not allocate |
| 330 |
memory for the arrays in the :cpp:struct:`mfxExtMVCSeqDesc` structure. Set |
| 331 |
the ``View``, ``ViewId``, and ``OP`` pointers to NULL and set |
| 332 |
``NumViewAlloc``, ``NumViewIdAlloc``, and ``NumOPAlloc`` to zero. The |
| 333 |
function parses the bitstream and returns |
| 334 |
:cpp:enumerator:`mfxStatus::MFX_ERR_NOT_ENOUGH_BUFFER` with the correct |
| 335 |
values for ``NumView``, ``NumViewId``, and ``NumOP``. This step can be |
| 336 |
skipped if the application is able to obtain the ``NumView``, |
| 337 |
``NumViewId``, and ``NumOP`` values from other sources. |
| 338 |
#. The application allocates memory for the ``View``, ``ViewId``, and ``OP`` |
| 339 |
arrays and calls the :cpp:func:`MFXVideoDECODE_DecodeHeader` function |
| 340 |
again. The function returns the MVC structural information in the |
| 341 |
allocated arrays. |
| 342 |
|
| 343 |
#. The application fills the :cpp:struct:`mfxExtMVCTargetViews` structure to |
| 344 |
choose the target views, based on information described in the |
| 345 |
:cpp:struct:`mfxExtMVCSeqDesc` structure. |
| 346 |
#. The application initializes the |vpl_short_name| decoder using the |
| 347 |
:cpp:func:`MFXVideoDECODE_Init` function. The application must attach both |
| 348 |
the :cpp:struct:`mfxExtMVCSeqDesc` structure and the |
| 349 |
:cpp:struct:`mfxExtMVCTargetViews` structure to the |
| 350 |
:cpp:struct:`mfxVideoParam` structure. |
| 351 |
|
| 352 |
In the above steps, do not modify the values of the |
| 353 |
:cpp:struct:`mfxExtMVCSeqDesc` structure after the |
| 354 |
:cpp:func:`MFXVideoDECODE_DecodeHeader` function, as the |vpl_short_name| decoder uses the |
| 355 |
values in the structure for internal memory allocation. Once the application |
| 356 |
configures the |vpl_short_name| decoder, the rest of the decoding procedure remains |
| 357 |
unchanged. As shown in the pseudo code below, the application calls the |
| 358 |
:cpp:func:`MFXVideoDECODE_DecodeFrameAsync` function multiple times to obtain |
| 359 |
all target views of the current frame picture, one target view at a time. The |
| 360 |
target view is identified by the ``FrameID`` field of the |
| 361 |
:cpp:struct:`mfxFrameInfo` structure. |
| 362 |
|
| 363 |
.. literalinclude:: ../snippets/prg_decoding.c |
| 364 |
:language: c++ |
| 365 |
:start-after: /*beg5*/ |
| 366 |
:end-before: /*end5*/ |
| 367 |
:lineno-start: 1 |
| 368 |
|
| 369 |
--------------------------------------------------- |
| 370 |
Combined Decode with Multi-channel Video Processing |
| 371 |
--------------------------------------------------- |
| 372 |
|
| 373 |
The |vpl_short_name| exposes interface for making decode and video processing operations |
| 374 |
in one call. Users can specify a number of output processing channels and |
| 375 |
multiple video filters per each channel. This interface supports only internal |
| 376 |
memory allocation model and returns array of processed frames through |
| 377 |
:cpp:struct:`mfxSurfaceArray` reference object as shown by the example: |
| 378 |
|
| 379 |
.. literalinclude:: ../snippets/prg_decoding_vpp.c |
| 380 |
:language: c++ |
| 381 |
:start-after: /*beg1*/ |
| 382 |
:end-before: /*end1*/ |
| 383 |
:lineno-start: 1 |
| 384 |
|
| 385 |
It's possible that different video processing channels may have different |
| 386 |
latency: |
| 387 |
|
| 388 |
.. literalinclude:: ../snippets/prg_decoding_vpp.c |
| 389 |
:language: c++ |
| 390 |
:start-after: /*beg2*/ |
| 391 |
:end-before: /*end2*/ |
| 392 |
:lineno-start: 1 |
| 393 |
|
| 394 |
Application can match decoded frame w/ specific VPP channels using |
| 395 |
:cpp:member:`mfxFrameData::TimeStamp`, :cpp:member:mfxFrameData::FrameOrder` and |
| 396 |
:cpp:member:`mfxFrameInfo::ChannelId`. |
| 397 |
|
| 398 |
Application can skip some or all channels including decoding output with help of |
| 399 |
`skip_channels` and `num_skip_channels` parameters as follows: application fills |
| 400 |
`skip_channels` array with `ChannelId`s to disable output of correspondent |
| 401 |
channels. In that case :cpp:member:`surf_array_out` would contain only surfaces |
| 402 |
for the remaining channels. If the decoder's channel and/or impacted VPP |
| 403 |
channels don't have output frame(s) for the current call (for instance, input |
| 404 |
bitstream doesn't contain complete frame or deinterlacing/FRC filter have delay) |
| 405 |
`skip_channels` parameter is ignored for this channel. |
| 406 |
|
| 407 |
If application disables all channels the SDK returns NULL as |
| 408 |
:cpp:struct:`mfxSurfaceArray`. |
| 409 |
|
| 410 |
If application doesn't need to disable any channels it sets `num_skip_channels` |
| 411 |
to zero, `skip_channels` is ignored when `num_skip_channels` is zero. |
| 412 |
|
| 413 |
If application doesn't need to make scaling or cropping operations it has to set the following fields |
| 414 |
:cpp:member:`mfxFrameInfo::Width`, :cpp:member:`mfxFrameInfo::Height`, |
| 415 |
:cpp:member:`mfxFrameInfo::CropX`, :cpp:member:`mfxFrameInfo::CropY` |
| 416 |
:cpp:member:`mfxFrameInfo::CropW`, :cpp:member:`mfxFrameInfo::CropH` of the VPP channel to zero. |
| 417 |
In that case output surfaces have the original decoded resolution and cropping. The operation supports |
| 418 |
bitstreams with resolution change without need of :cpp:func:`MFXVideoDECODE_VPP_Reset` call. |
| 419 |
|
| 420 |
.. note:: Even if more than one input compressed frame is consumed, the |
| 421 |
:cpp:func:`MFXVideoDECODE_VPP_DecodeFrameAsync` produces only one decoded |
| 422 |
frame and correspondent frames from VPP channels. |
| 423 |
|
| 424 |
--------------------------------------------------- |
| 425 |
AsyncDepth Specific Details |
| 426 |
--------------------------------------------------- |
| 427 |
|
| 428 |
`AsyncDepth` means how many asynchronous operations an application performs before the application explicitly |
| 429 |
synchronizes the result, application will set this parameter and call :cpp:func:`MFXVideoDECODE_DecodeFrameAsync`, |
| 430 |
the number of submitted frames is specified by `AsyncDepth`, then call :cpp:func:`MFXVideoCORE_SyncOperation` to synchronize |
| 431 |
the result of all frames. |
| 432 |
|
| 433 |
The following pseudo code shows the simplified asynchronous and synchronous procedure: |
| 434 |
|
| 435 |
.. literalinclude:: ../snippets/prg_decoding.c |
| 436 |
:language: c++ |
| 437 |
:start-after: /*beg7*/ |
| 438 |
:end-before: /*end7*/ |
| 439 |
:lineno-start: 1 |
| 440 |
|
| 441 |
According to memory utilization and performance, two modes are recommend for decoder pipeline: |
| 442 |
|
| 443 |
- For smallest memory footprint: the application should set AsyncDepth to 1 and then call :cpp:func:`MFXVideoCORE_SyncOperation` |
| 444 |
after each frame. This will have poor performance, since the GPU will not be kept fully occupied. |
| 445 |
- For best performance: the application should set AsyncDepth to 4 and plan to :cpp:func:`MFXVideoCORE_SyncOperation` after every |
| 446 |
4th frame, which generally will keep the GPU close to fully occupied. |