Debian Codegraph @

doc/spec/source/programming_guide/VPL_prg_transcoding.rst

raw
1 .. SPDX-FileCopyrightText: 2019-2020 Intel Corporation
2 ..
3 .. SPDX-License-Identifier: CC-BY-4.0
4
5 ======================
6 Transcoding Procedures
7 ======================
8
9 The application can use |vpl_short_name| encoding, decoding, and video processing
10 functions together for transcoding operations. This section describes the key
11 aspects of connecting two or more |vpl_short_name| functions together.
12
13 ---------------------
14 Asynchronous Pipeline
15 ---------------------
16
17 The application passes the output of an upstream |vpl_short_name| function to the input of
18 the downstream |vpl_short_name| function to construct an asynchronous pipeline. Pipeline
19 construction is done at runtime and can be dynamically changed, as shown in the
20 following example:
21
22 .. literalinclude:: ../snippets/prg_transcoding.c
23 :language: c++
24 :start-after: /*beg1*/
25 :end-before: /*end1*/
26 :lineno-start: 1
27
28 |vpl_short_name| simplifies the requirements for asynchronous pipeline synchronization.
29 The application only needs to synchronize after the last |vpl_short_name| function. Explicit
30 synchronization of intermediate results is not required and may slow performance.
31
32 |vpl_short_name| tracks dynamic pipeline construction and verifies dependency on input
33 and output parameters to ensure the execution order of the pipeline function.
34 In the previous example, |vpl_short_name| will ensure :cpp:func:`MFXVideoENCODE_EncodeFrameAsync`
35 does not begin its operation until :cpp:func:`MFXVideoDECODE_DecodeFrameAsync` or
36 :cpp:func:`MFXVideoVPP_RunFrameVPPAsync` has finished.
37
38 During the execution of an asynchronous pipeline, the application must consider
39 the input data as "in use" and must not change it until the execution has completed.
40 The application must also consider output data unavailable until the execution
41 has finished. In addition, for encoders, the application must consider extended
42 and payload buffers as "in use" while the input surface is locked.
43
44 |vpl_short_name| checks dependencies by comparing the input and output parameters of each
45 |vpl_short_name| function in the pipeline. Do not modify the contents of input and output
46 parameters before the previous asynchronous operation finishes. Doing so will
47 break the dependency check and can result in undefined behavior. An exception
48 occurs when the input and output parameters are structures, in which case
49 overwriting fields in the structures is allowed.
50
51 .. note:: The dependency check works on the pointers to the structures only.
52
53 There are two exceptions with respect to intermediate synchronization:
54
55 - If the input is from any asynchronous operation, the application must
56 synchronize any input before calling the |vpl_short_name| :cpp:func:`MFXVideoDECODE_DecodeFrameAsync`
57 function.
58 - When the application calls an asynchronous function to generate an output
59 surface in video memory and passes that surface to a non-|vpl_short_name| component, it must
60 explicitly synchronize the operation before passing the surface to the non-|vpl_short_name|
61 component.
62
63 .. _surface_pool_alloc:
64
65 -----------------------
66 Surface Pool Allocation
67 -----------------------
68
69 When connecting API function **A** to API function **B**, the application must
70 take into account the requirements of both functions to calculate the number of
71 frame surfaces in the surface pool. Typically, the application can use the formula
72 **Na+Nb**, where **Na** is the frame surface requirements for |vpl_short_name| function **A**
73 output, and **Nb** is the frame surface requirements for |vpl_short_name| function **B** input.
74
75 For performance considerations, the application must submit multiple operations
76 and delay synchronization as much as possible, which gives |vpl_short_name| flexibility
77 to organize internal pipelining. For example, compare the following two operation
78 sequences, where the first sequence is the recommended order:
79
80 .. graphviz::
81 :caption: Recommended operation sequence
82
83 digraph {
84 rankdir=LR;
85 labelloc="t";
86 label="Operation sequence 1";
87 f1 [shape=record label="ENCODE(F1)" ];
88 f2 [shape=record label="ENCODE(F2)" ];
89 f3 [shape=record label="SYNC(F1)" ];
90 f4 [shape=record label="SYNC(F2)" ];
91 f1->f2->f3->f4;
92 }
93
94
95 .. graphviz::
96 :caption: Operation sequence - not recommended
97
98 digraph {
99 rankdir=LR;
100 labelloc="t";
101 label="Operation sequence 2";
102 f1 [shape=record label="ENCODE(F1)" ];
103 f2 [shape=record label="ENCODE(F2)" ];
104 f3 [shape=record label="SYNC(F1)" ];
105 f4 [shape=record label="SYNC(F2)" ];
106 f1->f3->f2->f4;
107 }
108
109
110
111 In this example, the surface pool needs additional surfaces to take into account
112 multiple asynchronous operations before synchronization. The application can use
113 the :cpp:member:`mfxVideoParam::AsyncDepth` field to inform a |vpl_short_name| function of
114 the number of asynchronous operations the application plans to perform before
115 synchronization. The corresponding |vpl_short_name| **QueryIOSurf** function will reflect
116 this number in the :cpp:member:`mfxFrameAllocRequest::NumFrameSuggested`
117 value. The following example shows a way of calculating the surface needs based
118 on :cpp:member:`mfxFrameAllocRequest::NumFrameSuggested` values:
119
120 .. literalinclude:: ../snippets/prg_transcoding.c
121 :language: c++
122 :start-after: /*beg2*/
123 :end-before: /*end2*/
124 :lineno-start: 1
125
126 ------------------------
127 Pipeline Error Reporting
128 ------------------------
129
130 During asynchronous pipeline construction, each pipeline stage function will
131 return a synchronization point (sync point). These synchronization points are
132 useful in tracking errors during the asynchronous pipeline operation.
133
134 For example, assume the following pipeline:
135
136 .. graphviz::
137
138 digraph {
139 rankdir=LR;
140 A->B->C;
141 }
142
143 The application synchronizes on sync point **C**. If the error occurs in
144 function **C**, then the synchronization returns the exact error code. If the
145 error occurs before function **C**, then the synchronization returns
146 :cpp:enumerator:`mfxStatus::MFX_ERR_ABORTED`. The application can then try to
147 synchronize on sync point **B**. Similarly, if the error occurs in function **B**,
148 the synchronization returns the exact error code, or else
149 :cpp:enumerator:`mfxStatus:: MFX_ERR_ABORTED`. The same logic applies if the
150 error occurs in function **A**.