| 1 |
.. SPDX-FileCopyrightText: 2019-2020 Intel Corporation |
| 2 |
.. |
| 3 |
.. SPDX-License-Identifier: CC-BY-4.0 |
| 4 |
|
| 5 |
====================== |
| 6 |
Transcoding Procedures |
| 7 |
====================== |
| 8 |
|
| 9 |
The application can use |vpl_short_name| encoding, decoding, and video processing |
| 10 |
functions together for transcoding operations. This section describes the key |
| 11 |
aspects of connecting two or more |vpl_short_name| functions together. |
| 12 |
|
| 13 |
--------------------- |
| 14 |
Asynchronous Pipeline |
| 15 |
--------------------- |
| 16 |
|
| 17 |
The application passes the output of an upstream |vpl_short_name| function to the input of |
| 18 |
the downstream |vpl_short_name| function to construct an asynchronous pipeline. Pipeline |
| 19 |
construction is done at runtime and can be dynamically changed, as shown in the |
| 20 |
following example: |
| 21 |
|
| 22 |
.. literalinclude:: ../snippets/prg_transcoding.c |
| 23 |
:language: c++ |
| 24 |
:start-after: /*beg1*/ |
| 25 |
:end-before: /*end1*/ |
| 26 |
:lineno-start: 1 |
| 27 |
|
| 28 |
|vpl_short_name| simplifies the requirements for asynchronous pipeline synchronization. |
| 29 |
The application only needs to synchronize after the last |vpl_short_name| function. Explicit |
| 30 |
synchronization of intermediate results is not required and may slow performance. |
| 31 |
|
| 32 |
|vpl_short_name| tracks dynamic pipeline construction and verifies dependency on input |
| 33 |
and output parameters to ensure the execution order of the pipeline function. |
| 34 |
In the previous example, |vpl_short_name| will ensure :cpp:func:`MFXVideoENCODE_EncodeFrameAsync` |
| 35 |
does not begin its operation until :cpp:func:`MFXVideoDECODE_DecodeFrameAsync` or |
| 36 |
:cpp:func:`MFXVideoVPP_RunFrameVPPAsync` has finished. |
| 37 |
|
| 38 |
During the execution of an asynchronous pipeline, the application must consider |
| 39 |
the input data as "in use" and must not change it until the execution has completed. |
| 40 |
The application must also consider output data unavailable until the execution |
| 41 |
has finished. In addition, for encoders, the application must consider extended |
| 42 |
and payload buffers as "in use" while the input surface is locked. |
| 43 |
|
| 44 |
|vpl_short_name| checks dependencies by comparing the input and output parameters of each |
| 45 |
|vpl_short_name| function in the pipeline. Do not modify the contents of input and output |
| 46 |
parameters before the previous asynchronous operation finishes. Doing so will |
| 47 |
break the dependency check and can result in undefined behavior. An exception |
| 48 |
occurs when the input and output parameters are structures, in which case |
| 49 |
overwriting fields in the structures is allowed. |
| 50 |
|
| 51 |
.. note:: The dependency check works on the pointers to the structures only. |
| 52 |
|
| 53 |
There are two exceptions with respect to intermediate synchronization: |
| 54 |
|
| 55 |
- If the input is from any asynchronous operation, the application must |
| 56 |
synchronize any input before calling the |vpl_short_name| :cpp:func:`MFXVideoDECODE_DecodeFrameAsync` |
| 57 |
function. |
| 58 |
- When the application calls an asynchronous function to generate an output |
| 59 |
surface in video memory and passes that surface to a non-|vpl_short_name| component, it must |
| 60 |
explicitly synchronize the operation before passing the surface to the non-|vpl_short_name| |
| 61 |
component. |
| 62 |
|
| 63 |
.. _surface_pool_alloc: |
| 64 |
|
| 65 |
----------------------- |
| 66 |
Surface Pool Allocation |
| 67 |
----------------------- |
| 68 |
|
| 69 |
When connecting API function **A** to API function **B**, the application must |
| 70 |
take into account the requirements of both functions to calculate the number of |
| 71 |
frame surfaces in the surface pool. Typically, the application can use the formula |
| 72 |
**Na+Nb**, where **Na** is the frame surface requirements for |vpl_short_name| function **A** |
| 73 |
output, and **Nb** is the frame surface requirements for |vpl_short_name| function **B** input. |
| 74 |
|
| 75 |
For performance considerations, the application must submit multiple operations |
| 76 |
and delay synchronization as much as possible, which gives |vpl_short_name| flexibility |
| 77 |
to organize internal pipelining. For example, compare the following two operation |
| 78 |
sequences, where the first sequence is the recommended order: |
| 79 |
|
| 80 |
.. graphviz:: |
| 81 |
:caption: Recommended operation sequence |
| 82 |
|
| 83 |
digraph { |
| 84 |
rankdir=LR; |
| 85 |
labelloc="t"; |
| 86 |
label="Operation sequence 1"; |
| 87 |
f1 [shape=record label="ENCODE(F1)" ]; |
| 88 |
f2 [shape=record label="ENCODE(F2)" ]; |
| 89 |
f3 [shape=record label="SYNC(F1)" ]; |
| 90 |
f4 [shape=record label="SYNC(F2)" ]; |
| 91 |
f1->f2->f3->f4; |
| 92 |
} |
| 93 |
|
| 94 |
|
| 95 |
.. graphviz:: |
| 96 |
:caption: Operation sequence - not recommended |
| 97 |
|
| 98 |
digraph { |
| 99 |
rankdir=LR; |
| 100 |
labelloc="t"; |
| 101 |
label="Operation sequence 2"; |
| 102 |
f1 [shape=record label="ENCODE(F1)" ]; |
| 103 |
f2 [shape=record label="ENCODE(F2)" ]; |
| 104 |
f3 [shape=record label="SYNC(F1)" ]; |
| 105 |
f4 [shape=record label="SYNC(F2)" ]; |
| 106 |
f1->f3->f2->f4; |
| 107 |
} |
| 108 |
|
| 109 |
|
| 110 |
|
| 111 |
In this example, the surface pool needs additional surfaces to take into account |
| 112 |
multiple asynchronous operations before synchronization. The application can use |
| 113 |
the :cpp:member:`mfxVideoParam::AsyncDepth` field to inform a |vpl_short_name| function of |
| 114 |
the number of asynchronous operations the application plans to perform before |
| 115 |
synchronization. The corresponding |vpl_short_name| **QueryIOSurf** function will reflect |
| 116 |
this number in the :cpp:member:`mfxFrameAllocRequest::NumFrameSuggested` |
| 117 |
value. The following example shows a way of calculating the surface needs based |
| 118 |
on :cpp:member:`mfxFrameAllocRequest::NumFrameSuggested` values: |
| 119 |
|
| 120 |
.. literalinclude:: ../snippets/prg_transcoding.c |
| 121 |
:language: c++ |
| 122 |
:start-after: /*beg2*/ |
| 123 |
:end-before: /*end2*/ |
| 124 |
:lineno-start: 1 |
| 125 |
|
| 126 |
------------------------ |
| 127 |
Pipeline Error Reporting |
| 128 |
------------------------ |
| 129 |
|
| 130 |
During asynchronous pipeline construction, each pipeline stage function will |
| 131 |
return a synchronization point (sync point). These synchronization points are |
| 132 |
useful in tracking errors during the asynchronous pipeline operation. |
| 133 |
|
| 134 |
For example, assume the following pipeline: |
| 135 |
|
| 136 |
.. graphviz:: |
| 137 |
|
| 138 |
digraph { |
| 139 |
rankdir=LR; |
| 140 |
A->B->C; |
| 141 |
} |
| 142 |
|
| 143 |
The application synchronizes on sync point **C**. If the error occurs in |
| 144 |
function **C**, then the synchronization returns the exact error code. If the |
| 145 |
error occurs before function **C**, then the synchronization returns |
| 146 |
:cpp:enumerator:`mfxStatus::MFX_ERR_ABORTED`. The application can then try to |
| 147 |
synchronize on sync point **B**. Similarly, if the error occurs in function **B**, |
| 148 |
the synchronization returns the exact error code, or else |
| 149 |
:cpp:enumerator:`mfxStatus:: MFX_ERR_ABORTED`. The same logic applies if the |
| 150 |
error occurs in function **A**. |