Debian Codegraph @

rfc6190.html

raw
1 <pre>Internet Engineering Task Force (IETF) S. Wenger
2 Request for Comments: 6190 Independent
3 Category: Standards Track Y.-K. Wang
4 ISSN: 2070-1721 Huawei Technologies
5 T. Schierl
6 Fraunhofer HHI
7 A. Eleftheriadis
8 Vidyo
9 May 2011
10
11
12 <span class="h1">RTP Payload Format for Scalable Video Coding</span>
13
14 Abstract
15
16 This memo describes an RTP payload format for Scalable Video Coding
17 (SVC) as defined in Annex G of ITU-T Recommendation H.264, which is
18 technically identical to Amendment 3 of ISO/IEC International
19 Standard 14496-10. The RTP payload format allows for packetization
20 of one or more Network Abstraction Layer (NAL) units in each RTP
21 packet payload, as well as fragmentation of a NAL unit in multiple
22 RTP packets. Furthermore, it supports transmission of an SVC stream
23 over a single as well as multiple RTP sessions. The payload format
24 defines a new media subtype name "H264-SVC", but is still backward
25 compatible to <a href="./rfc6184">RFC 6184</a> since the base layer, when encapsulated in its
26 own RTP stream, must use the H.264 media subtype name ("H264") and
27 the packetization method specified in <a href="./rfc6184">RFC 6184</a>. The payload format
28 has wide applicability in videoconferencing, Internet video
29 streaming, and high-bitrate entertainment-quality video, among
30 others.
31
32 Status of This Memo
33
34 This is an Internet Standards Track document.
35
36 This document is a product of the Internet Engineering Task Force
37 (IETF). It represents the consensus of the IETF community. It has
38 received public review and has been approved for publication by the
39 Internet Engineering Steering Group (IESG). Further information on
40 Internet Standards is available in <a href="./rfc5741#section-2">Section&nbsp;2 of RFC 5741</a>.
41
42 Information about the current status of this document, any errata,
43 and how to provide feedback on it may be obtained at
44 <a href="http://www.rfc-editor.org/info/rfc6190">http://www.rfc-editor.org/info/rfc6190</a>.
45
46
47
48
49
50
51
52 <span class="grey">Wenger, et al. Standards Track [Page 1]</span></pre>
53 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-2" ></span>
54 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
55
56
57 Copyright Notice
58
59 Copyright (c) 2011 IETF Trust and the persons identified as the
60 document authors. All rights reserved.
61
62 This document is subject to <a href="https://www.rfc-editor.org/bcp/bcp78">BCP 78</a> and the IETF Trust's Legal
63 Provisions Relating to IETF Documents
64 (<a href="http://trustee.ietf.org/license-info">http://trustee.ietf.org/license-info</a>) in effect on the date of
65 publication of this document. Please review these documents
66 carefully, as they describe your rights and restrictions with respect
67 to this document. Code Components extracted from this document must
68 include Simplified BSD License text as described in Section 4.e of
69 the Trust Legal Provisions and are provided without warranty as
70 described in the Simplified BSD License.
71
72 This document may contain material from IETF Documents or IETF
73 Contributions published or made publicly available before November
74 10, 2008. The person(s) controlling the copyright in some of this
75 material may not have granted the IETF Trust the right to allow
76 modifications of such material outside the IETF Standards Process.
77 Without obtaining an adequate license from the person(s) controlling
78 the copyright in such materials, this document may not be modified
79 outside the IETF Standards Process, and derivative works of it may
80 not be created outside the IETF Standards Process, except to format
81 it for publication as an RFC or to translate it into languages other
82 than English.
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108 <span class="grey">Wenger, et al. Standards Track [Page 2]</span></pre>
109 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-3" ></span>
110 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
111
112
113 Table of Contents
114
115 <a href="#section-1">1</a>. Introduction ....................................................<a href="#page-5">5</a>
116 <a href="#section-1.1">1.1</a>. The SVC Codec ..............................................<a href="#page-6">6</a>
117 <a href="#section-1.1.1">1.1.1</a>. Overview ............................................<a href="#page-6">6</a>
118 <a href="#section-1.1.2">1.1.2</a>. Parameter Sets ......................................<a href="#page-8">8</a>
119 <a href="#section-1.1.3">1.1.3</a>. NAL Unit Header .....................................<a href="#page-9">9</a>
120 <a href="#section-1.2">1.2</a>. Overview of the Payload Format ............................<a href="#page-12">12</a>
121 <a href="#section-1.2.1">1.2.1</a>. Design Principles ..................................<a href="#page-12">12</a>
122 <a href="#section-1.2.2">1.2.2</a>. Transmission Modes and Packetization Modes .........<a href="#page-13">13</a>
123 <a href="#section-1.2.3">1.2.3</a>. New Payload Structures .............................<a href="#page-15">15</a>
124 <a href="#section-2">2</a>. Conventions ....................................................<a href="#page-16">16</a>
125 <a href="#section-3">3</a>. Definitions and Abbreviations ..................................<a href="#page-16">16</a>
126 <a href="#section-3.1">3.1</a>. Definitions ...............................................<a href="#page-16">16</a>
127 <a href="#section-3.1.1">3.1.1</a>. Definitions from the SVC Specification .............<a href="#page-16">16</a>
128 <a href="#section-3.1.2">3.1.2</a>. Definitions Specific to This Memo ..................<a href="#page-18">18</a>
129 <a href="#section-3.2">3.2</a>. Abbreviations .............................................<a href="#page-22">22</a>
130 <a href="#section-4">4</a>. RTP Payload Format .............................................<a href="#page-23">23</a>
131 <a href="#section-4.1">4.1</a>. RTP Header Usage ..........................................<a href="#page-23">23</a>
132 <a href="#section-4.2">4.2</a>. NAL Unit Extension and Header Usage .......................<a href="#page-23">23</a>
133 <a href="#section-4.2.1">4.2.1</a>. NAL Unit Extension .................................<a href="#page-23">23</a>
134 <a href="#section-4.2.2">4.2.2</a>. NAL Unit Header Usage ..............................<a href="#page-24">24</a>
135 <a href="#section-4.3">4.3</a>. Payload Structures ........................................<a href="#page-25">25</a>
136 <a href="#section-4.4">4.4</a>. Transmission Modes ........................................<a href="#page-28">28</a>
137 <a href="#section-4.5">4.5</a>. Packetization Modes .......................................<a href="#page-28">28</a>
138 4.5.1. Packetization Modes for Single-Session
139 Transmission .......................................<a href="#page-28">28</a>
140 4.5.2. Packetization Modes for Multi-Session
141 Transmission .......................................<a href="#page-29">29</a>
142 <a href="#section-4.6">4.6</a>. Single NAL Unit Packets ...................................<a href="#page-32">32</a>
143 <a href="#section-4.7">4.7</a>. Aggregation Packets .......................................<a href="#page-33">33</a>
144 4.7.1. Non-Interleaved Multi-Time Aggregation
145 Packets (NI-MTAPs) .................................<a href="#page-33">33</a>
146 <a href="#section-4.8">4.8</a>. Fragmentation Units (FUs) .................................<a href="#page-35">35</a>
147 <a href="#section-4.9">4.9</a>. Payload Content Scalability Information (PACSI) NAL Unit ..35
148 <a href="#section-4.10">4.10</a>. Empty NAL unit ...........................................<a href="#page-43">43</a>
149 <a href="#section-4.11">4.11</a>. Decoding Order Number (DON) ..............................<a href="#page-43">43</a>
150 4.11.1. Cross-Session DON (CS-DON) for
151 Multi-Session Transmission ........................<a href="#page-43">43</a>
152 <a href="#section-5">5</a>. Packetization Rules ............................................<a href="#page-45">45</a>
153 <a href="#section-5.1">5.1</a>. Packetization Rules for Single-Session Transmission .......<a href="#page-45">45</a>
154 <a href="#section-5.2">5.2</a>. Packetization Rules for Multi-Session Transmission ........<a href="#page-46">46</a>
155 <a href="#section-5.2.1">5.2.1</a>. NI-T/NI-TC Packetization Rules .....................<a href="#page-47">47</a>
156 <a href="#section-5.2.2">5.2.2</a>. NI-C/NI-TC Packetization Rules .....................<a href="#page-49">49</a>
157 <a href="#section-5.2.3">5.2.3</a>. I-C Packetization Rules ............................<a href="#page-50">50</a>
158 <a href="#section-5.2.4">5.2.4</a>. Packetization Rules for Non-VCL NAL Units ..........<a href="#page-50">50</a>
159 <a href="#section-5.2.5">5.2.5</a>. Packetization Rules for Prefix NAL Units ...........<a href="#page-51">51</a>
160
161
162
163
164 <span class="grey">Wenger, et al. Standards Track [Page 3]</span></pre>
165 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-4" ></span>
166 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
167
168
169 <a href="#section-6">6</a>. De-Packetization Process .......................................<a href="#page-51">51</a>
170 <a href="#section-6.1">6.1</a>. De-Packetization Process for Single-Session Transmission ..51
171 <a href="#section-6.2">6.2</a>. De-Packetization Process for Multi-Session Transmission ...<a href="#page-51">51</a>
172 6.2.1. Decoding Order Recovery for the NI-T and
173 NI-TC Modes ........................................<a href="#page-52">52</a>
174 6.2.1.1. Informative Algorithm for NI-T
175 Decoding Order Recovery within
176 an Access Unit ............................<a href="#page-55">55</a>
177 6.2.2. Decoding Order Recovery for the NI-C,
178 NI-TC, and I-C Modes ...............................<a href="#page-57">57</a>
179 <a href="#section-7">7</a>. Payload Format Parameters ......................................<a href="#page-59">59</a>
180 <a href="#section-7.1">7.1</a>. Media Type Registration ...................................<a href="#page-60">60</a>
181 <a href="#section-7.2">7.2</a>. SDP Parameters ............................................<a href="#page-75">75</a>
182 <a href="#section-7.2.1">7.2.1</a>. Mapping of Payload Type Parameters to SDP ..........<a href="#page-75">75</a>
183 <a href="#section-7.2.2">7.2.2</a>. Usage with the SDP Offer/Answer Model ..............<a href="#page-76">76</a>
184 7.2.3. Dependency Signaling in Multi-Session
185 Transmission .......................................<a href="#page-84">84</a>
186 <a href="#section-7.2.4">7.2.4</a>. Usage in Declarative Session Descriptions ..........<a href="#page-85">85</a>
187 <a href="#section-7.3">7.3</a>. Examples ..................................................<a href="#page-86">86</a>
188 <a href="#section-7.3.1">7.3.1</a>. Example for Offering a Single SVC Session ..........<a href="#page-86">86</a>
189 7.3.2. Example for Offering a Single SVC Session Using
190 scalable-layer-id ..................................<a href="#page-87">87</a>
191 <a href="#section-7.3.3">7.3.3</a>. Example for Offering Multiple Sessions in MST ......<a href="#page-87">87</a>
192 7.3.4. Example for Offering Multiple Sessions in
193 MST Including Operation with Answerer Using
194 scalable-layer-id ..................................<a href="#page-89">89</a>
195 7.3.5. Example for Negotiating an SVC Stream with
196 a Constrained Base Layer in SST ....................<a href="#page-90">90</a>
197 <a href="#section-7.4">7.4</a>. Parameter Set Considerations ..............................<a href="#page-91">91</a>
198 <a href="#section-8">8</a>. Security Considerations ........................................<a href="#page-91">91</a>
199 <a href="#section-9">9</a>. Congestion Control .............................................<a href="#page-92">92</a>
200 <a href="#section-10">10</a>. IANA Considerations ...........................................<a href="#page-93">93</a>
201 <a href="#section-11">11</a>. Informative Appendix: Application Examples ....................<a href="#page-93">93</a>
202 <a href="#section-11.1">11.1</a>. Introduction .............................................<a href="#page-93">93</a>
203 <a href="#section-11.2">11.2</a>. Layered Multicast ........................................<a href="#page-93">93</a>
204 <a href="#section-11.3">11.3</a>. Streaming ................................................<a href="#page-94">94</a>
205 11.4. Videoconferencing (Unicast to MANE, Unicast to
206 Endpoints) ...............................................<a href="#page-95">95</a>
207 <a href="#section-11.5">11.5</a>. Mobile TV (Multicast to MANE, Unicast to Endpoint) .......<a href="#page-96">96</a>
208 <a href="#section-12">12</a>. Acknowledgements ..............................................<a href="#page-97">97</a>
209 <a href="#section-13">13</a>. References ....................................................<a href="#page-97">97</a>
210 <a href="#section-13.1">13.1</a>. Normative References .....................................<a href="#page-97">97</a>
211 <a href="#section-13.2">13.2</a>. Informative References ...................................<a href="#page-98">98</a>
212
213
214
215
216
217
218
219
220 <span class="grey">Wenger, et al. Standards Track [Page 4]</span></pre>
221 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-5" ></span>
222 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
223
224
225 <span class="h2"><a class="selflink" id="section-1" href="#section-1">1</a>. Introduction</span>
226
227 This memo specifies an RTP [<a href="./rfc3550" title="&quot;RTP: A Transport Protocol for Real-Time Applications&quot;">RFC3550</a>] payload format for the Scalable
228 Video Coding (SVC) extension of the H.264/AVC video coding standard.
229 SVC is specified in Amendment 3 to ISO/IEC 14496 Part 10
230 [ISO/IEC14496-10] and equivalently in Annex G of ITU-T Rec. H.264
231 [<a href="#ref-H.264" title="&quot;Advanced video coding for generic audiovisual services&quot;">H.264</a>]. In this memo, unless explicitly stated otherwise,
232 "H.264/AVC" refers to the specification of [<a href="#ref-H.264" title="&quot;Advanced video coding for generic audiovisual services&quot;">H.264</a>] excluding Annex G.
233
234 SVC covers the entire application range of H.264/AVC, from low-
235 bitrate mobile applications, to High-Definition Television (HDTV)
236 broadcasting, and even Digital Cinema that requires nearly lossless
237 coding and hundreds of megabits per second. The scalability features
238 that SVC adds to H.264/AVC enable several system-level
239 functionalities related to the ability of a system to adapt the
240 signal to different system conditions with no or minimal processing.
241 The adaptation relates both to the capabilities of potentially
242 heterogeneous receivers (differing in screen resolution, processing
243 speed, etc.), and to differing or time-varying network conditions.
244 The adaptation can be performed at the source, the destination, or in
245 intermediate media-aware network elements (MANEs). The payload
246 format specified in this memo exposes these system-level
247 functionalities so that system designers can take direct advantage of
248 these features.
249
250 Informative note: Since SVC streams contain, by design, a sub-
251 stream that is compliant with H.264/AVC, it is trivial for a MANE
252 to filter the stream so that all SVC-specific information is
253 removed. This memo, in fact, defines a media type parameter
254 (sprop-avc-ready, <a href="#section-7.2">Section 7.2</a>) that indicates whether or not the
255 stream can be converted to one compliant with [<a href="./rfc6184" title="&quot;RTP Payload Format for H.264 Video&quot;">RFC6184</a>] by
256 eliminating RTP packets, and rewriting RTP Control Protocol (RTCP)
257 to match the changes to the RTP packet stream as specified in
258 <a href="./rfc3550#section-7">Section&nbsp;7 of [RFC3550]</a>.
259
260 This memo defines two basic modes for transmission of SVC data,
261 single-session transmission (SST) and multi-session transmission
262 (MST). In SST, a single RTP session is used for the transmission of
263 all scalability layers comprising an SVC bitstream; in MST, the
264 scalability layers are transported on different RTP sessions. In
265 SST, packetization is a straightforward extension of [<a href="./rfc6184" title="&quot;RTP Payload Format for H.264 Video&quot;">RFC6184</a>]. For
266 MST, four different modes are defined in this memo. They differ on
267 whether or not they allow interleaving, i.e., transmitting Network
268 Abstraction Layer (NAL) units in an order different than the decoding
269 order, and by the technique used to effect inter-session NAL unit
270 decoding order recovery. Decoding order recovery is performed using
271 either inter-session timestamp alignment [<a href="./rfc3550" title="&quot;RTP: A Transport Protocol for Real-Time Applications&quot;">RFC3550</a>] or cross-session
272 decoding order numbers (CS-DONs). One of the MST modes supports both
273
274
275
276 <span class="grey">Wenger, et al. Standards Track [Page 5]</span></pre>
277 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-6" ></span>
278 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
279
280
281 decoding order recovery techniques, so that receivers can select
282 their preferred technique. More details can be found in <a href="#section-1.2.2">Section</a>
283 <a href="#section-1.2.2">1.2.2</a>.
284
285 This memo further defines three new NAL unit types. The first type
286 is the payload content scalability information (PACSI) NAL unit,
287 which is used to provide an informative summary of the scalability
288 information of the data contained in an RTP packet, as well as
289 ancillary data (e.g., CS-DON values). The second and third new NAL
290 unit types are the empty NAL unit and the non-interleaved multi-time
291 aggregation packet (NI-MTAP) NAL unit. The empty NAL unit is used to
292 ensure inter-session timestamp alignment required for decoding order
293 recovery in MST. The NI-MTAP is used as a new payload structure
294 allowing the grouping of NAL units of different time instances in
295 decoding order. More details about the new packet structures can be
296 found in <a href="#section-1.2.3">Section 1.2.3</a>.
297
298 This memo also defines the signaling support for SVC transport over
299 RTP, including a new media subtype name (H264-SVC).
300
301 A non-normative overview of the SVC codec and the payload is given in
302 the remainder of this section.
303
304 <span class="h3"><a class="selflink" id="section-1.1" href="#section-1.1">1.1</a>. The SVC Codec</span>
305
306 <span class="h4"><a class="selflink" id="section-1.1.1" href="#section-1.1.1">1.1.1</a>. Overview</span>
307
308 SVC defines a coded video representation in which a given bitstream
309 offers representations of the source material at different levels of
310 fidelity (hence the term "scalable"). Scalable video coding
311 bitstreams, or scalable bitstreams, are constructed in a pyramidal
312 fashion: the coding process creates bitstream components that improve
313 the fidelity of hierarchically lower components.
314
315 The fidelity dimensions offered by SVC are spatial (picture size),
316 quality (or Signal-to-Noise Ratio (SNR)), and temporal (pictures per
317 second). Bitstream components associated with a given level of
318 spatial, quality, and temporal fidelity are identified using
319 corresponding parameters in the bitstream: dependency_id, quality_id,
320 and temporal_id (see also <a href="#section-1.1.3">Section 1.1.3</a>). The fidelity identifiers
321 have integer values, where higher values designate components that
322 are higher in the hierarchy. It is noted that SVC offers significant
323 flexibility in terms of how an encoder may choose to structure the
324 dependencies between the various components. Decoding of a
325 particular component requires the availability of all the components
326 it depends upon, either directly, or indirectly. An operation point
327
328
329
330
331
332 <span class="grey">Wenger, et al. Standards Track [Page 6]</span></pre>
333 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-7" ></span>
334 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
335
336
337 of an SVC bitstream consists of the bitstream components required to
338 be able to decode a particular dependency_id, quality_id, and
339 temporal_id combination.
340
341 The term "layer" is used in various contexts in this memo. For
342 example, in the terms "Video Coding Layer" and "Network Abstraction
343 Layer" it refers to conceptual organization levels. When referring
344 to bitstream syntax elements such as block layer or macroblock layer,
345 it refers to hierarchical bitstream structure levels. When used in
346 the context of bitstream scalability, e.g., "AVC base layer", it
347 refers to a level of representation fidelity of the source signal
348 with a specific set of NAL units included. The correct
349 interpretation is supported by providing the appropriate context.
350
351 SVC maintains the bitstream organization introduced in H.264/AVC.
352 Specifically, all bitstream components are encapsulated in Network
353 Abstraction Layer (NAL) units, which are organized as Access Units
354 (AUs). An AU is associated with a single sampling instance in time.
355 A subset of the NAL unit types correspond to the Video Coding Layer
356 (VCL), and contain the coded picture data associated with the source
357 content. Non-VCL NAL units carry ancillary data that may be
358 necessary for decoding (e.g., parameter sets as explained below) or
359 that facilitate certain system operations but are not needed by the
360 decoding process itself. Coded picture data at the various fidelity
361 dimensions are organized in slices. Within one AU, a coded picture
362 of an operation point consists of all the coded slices required for
363 decoding up to the particular combination of dependency_id and
364 quality_id values at the time instance corresponding to the AU.
365
366 It is noted that the concept of temporal scalability is already
367 present in H.264/AVC, as profiles defined in Annex A of [<a href="#ref-H.264" title="&quot;Advanced video coding for generic audiovisual services&quot;">H.264</a>]
368 already support it. Specifically, in H.264/AVC, the concept of sub-
369 sequences has been introduced to allow optional use of temporal
370 layers through Supplemental Enhancement Information (SEI) messages.
371 SVC extends this approach by exposing the temporal scalability
372 information using the temporal_id parameter, alongside (and unified
373 with) the dependency_id and quality_id values that are used for
374 spatial and quality scalability, respectively. For coded picture
375 data defined in Annex G of [<a href="#ref-H.264" title="&quot;Advanced video coding for generic audiovisual services&quot;">H.264</a>], this is accomplished by using a
376 new type of NAL unit, namely, coded slice in scalable extension NAL
377 unit (type 20), where the fidelity parameters are part of its header.
378 For coded picture data that follow H.264/AVC, and to ensure
379 compatibility with existing H.264/AVC decoders, another new type of
380 NAL unit, namely, prefix NAL unit (type 14), has been defined to
381 carry this header information. SVC additionally specifies a third
382 new type of NAL unit, namely, subset sequence parameter set NAL unit
383 (type 15), to contain sequence parameter set information for quality
384 and spatial enhancement layers. All these three newly specified NAL
385
386
387
388 <span class="grey">Wenger, et al. Standards Track [Page 7]</span></pre>
389 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-8" ></span>
390 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
391
392
393 unit types (14, 15, and 20) are among those reserved in H.264/AVC and
394 are to be ignored by decoders conforming to one or more of the
395 profiles specified in Annex A of [<a href="#ref-H.264" title="&quot;Advanced video coding for generic audiovisual services&quot;">H.264</a>].
396
397 Within an AU, the VCL NAL units associated with a given dependency_id
398 and quality_id are referred to as a "layer representation". The
399 layer representation corresponding to the lowest values of
400 dependency_id and quality_id (i.e., zero for both) is compliant by
401 design to H.264/AVC. The set of VCL and associated non-VCL NAL units
402 across all AUs in a bitstream associated with a particular
403 combination of values of dependency_id and quality_id, and regardless
404 of the value of temporal_id, is conceptually a scalable layer. For
405 backward compatibility with H.264/AVC, it is important to
406 differentiate, however, whether or not SVC-specific NAL units are
407 present in a given bitstream. This is particularly important for the
408 lowest fidelity values in terms of dependency_id and quality_id (zero
409 for both), as the corresponding VCL data are compliant with
410 H.264/AVC, and may or may not be accompanied by associated prefix NAL
411 units. This memo therefore uses the term "AVC base layer" to
412 designate the layer that does not contain SVC-specific NAL units, and
413 "SVC base layer" to designate the same layer but with the addition of
414 the associated SVC prefix NAL units. Note that the SVC specification
415 uses the term "base layer" for what in this memo will be referred to
416 as "AVC base layer". Similarly, it is also important to be able to
417 differentiate, within a layer, the temporal fidelity components it
418 contains. This memo uses the term "T0" to indicate, within a
419 particular layer, the subset that contains the NAL units associated
420 with temporal_id equal to 0.
421
422 SNR scalability in SVC is offered in two different ways. In what is
423 called coarse-grain scalability (CGS), scalability is provided by
424 including or excluding a complete layer when decoding a particular
425 bitstream. In contrast, in medium-grain scalability (MGS),
426 scalability is provided by selectively omitting the decoding of
427 specific NAL units belonging to MGS layers. The selection of the NAL
428 units to omit can be based on fixed-length fields present in the NAL
429 unit header (see also Sections <a href="#section-1.1.3">1.1.3</a> and <a href="#section-4.2">4.2</a>).
430
431 <span class="h4"><a class="selflink" id="section-1.1.2" href="#section-1.1.2">1.1.2</a>. Parameter Sets</span>
432
433 SVC maintains the parameter sets concept in H.264/AVC and introduces
434 a new type of sequence parameter set, referred to as the subset
435 sequence parameter set [<a href="#ref-H.264" title="&quot;Advanced video coding for generic audiovisual services&quot;">H.264</a>]. Subset sequence parameter sets have
436 NAL unit type equal to 15, which is different from the NAL unit type
437 value (7) of sequence parameter sets. VCL NAL units of NAL unit type
438 1 to 5 must only (indirectly) refer to sequence parameter sets, while
439 VCL NAL units of NAL unit type 20 must only (indirectly) refer to
440 subset sequence parameter sets. The references are indirect because
441
442
443
444 <span class="grey">Wenger, et al. Standards Track [Page 8]</span></pre>
445 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-9" ></span>
446 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
447
448
449 VCL NAL units refer to picture parameter sets (in their slice
450 header), which in turn refer to regular or subset sequence parameter
451 sets. Subset sequence parameter sets use a separate identifier value
452 space than sequence parameter sets.
453
454 In SVC, coded picture data from different layers may use the same or
455 different sequence and picture parameter sets. Let the variable DQId
456 be equal to dependency_id * 16 + quality_id. At any time instant
457 during the decoding process there is one active sequence parameter
458 set for the layer representation with the highest value of DQId and
459 one or more active layer SVC sequence parameter set(s) for layer
460 representations with lower values of DQId. The active sequence
461 parameter set or an active layer SVC sequence parameter set remains
462 unchanged throughout a coded video sequence in the scalable layer in
463 which the active sequence parameter set or active layer SVC sequence
464 parameter set is referred to. This means that the referred sequence
465 parameter set or subset sequence parameter set can only change at
466 instantaneous decoding refresh (IDR) access units for any layer. At
467 any time instant during the decoding process there may be one active
468 picture parameter set (for the layer representation with the highest
469 value of DQId) and one or more active layer picture parameter set(s)
470 (for layer representations with lower values of DQId). The active
471 picture parameter set or an active layer picture parameter set
472 remains unchanged throughout a layer representation in which the
473 active picture parameter set or active layer picture parameter set is
474 referred to, but may change from one AU to the next.
475
476 <span class="h4"><a class="selflink" id="section-1.1.3" href="#section-1.1.3">1.1.3</a>. NAL Unit Header</span>
477
478 SVC extends the one-byte H.264/AVC NAL unit header by three
479 additional octets for NAL units of types 14 and 20. The header
480 indicates the type of the NAL unit, the (potential) presence of bit
481 errors or syntax violations in the NAL unit payload, information
482 regarding the relative importance of the NAL unit for the decoding
483 process, the layer identification information, and other fields as
484 discussed below.
485
486 The syntax and semantics of the NAL unit header are specified in
487 [<a href="#ref-H.264" title="&quot;Advanced video coding for generic audiovisual services&quot;">H.264</a>], but the essential properties of the NAL unit header are
488 summarized below for convenience.
489
490 The first byte of the NAL unit header has the following format (the
491 bit fields are the same as defined for the one-byte H.264/AVC NAL
492 unit header, while the semantics of some fields have changed
493 slightly, in a backward-compatible way):
494
495
496
497
498
499
500 <span class="grey">Wenger, et al. Standards Track [Page 9]</span></pre>
501 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-10" ></span>
502 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
503
504
505 +---------------+
506 |0|1|2|3|4|5|6|7|
507 +-+-+-+-+-+-+-+-+
508 |F|NRI| Type |
509 +---------------+
510
511 The semantics of the components of the NAL unit type octet, as
512 specified in [<a href="#ref-H.264" title="&quot;Advanced video coding for generic audiovisual services&quot;">H.264</a>], are described briefly below. In addition to
513 the name and size of each field, the corresponding syntax element
514 name in [<a href="#ref-H.264" title="&quot;Advanced video coding for generic audiovisual services&quot;">H.264</a>] is also provided.
515
516 F: 1 bit
517 forbidden_zero_bit. H.264/AVC declares a value of 1 as a
518 syntax violation.
519
520 NRI: 2 bits
521 nal_ref_idc. A value of "00" (in binary form) indicates that
522 the content of the NAL unit is not used to reconstruct
523 reference pictures for future prediction. Such NAL units can
524 be discarded without risking the integrity of the reference
525 pictures in the same layer. A value greater than "00"
526 indicates that the decoding of the NAL unit is required to
527 maintain the integrity of reference pictures in the same layer
528 or that the NAL unit contains parameter sets.
529
530 Type: 5 bits
531 nal_unit_type. This component specifies the NAL unit type as
532 defined in Table 7-1 of [<a href="#ref-H.264" title="&quot;Advanced video coding for generic audiovisual services&quot;">H.264</a>], and later within this memo.
533 For a reference of all currently defined NAL unit types and
534 their semantics, please refer to Section 7.4.1 in [<a href="#ref-H.264" title="&quot;Advanced video coding for generic audiovisual services&quot;">H.264</a>].
535
536 In H.264/AVC, NAL unit types 14, 15, and 20 are reserved for
537 future extensions. SVC uses these three NAL unit types as
538 follows: NAL unit type 14 is used for prefix NAL unit, NAL unit
539 type 15 is used for subset sequence parameter set, and NAL unit
540 type 20 is used for coded slice in scalable extension (see
541 Section 7.4.1 in [<a href="#ref-H.264" title="&quot;Advanced video coding for generic audiovisual services&quot;">H.264</a>]). NAL unit types 14 and 20 indicate
542 the presence of three additional octets in the NAL unit header,
543 as shown below.
544
545 +---------------+---------------+---------------+
546 |0|1|2|3|4|5|6|7|0|1|2|3|4|5|6|7|0|1|2|3|4|5|6|7|
547 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
548 |R|I| PRID |N| DID | QID | TID |U|D|O| RR|
549 +---------------+---------------+---------------+
550
551
552
553
554
555
556 <span class="grey">Wenger, et al. Standards Track [Page 10]</span></pre>
557 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-11" ></span>
558 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
559
560
561 R: 1 bit
562 reserved_one_bit. Reserved bit for future extension. R must
563 be equal to 1. The value of R must be ignored by decoders.
564
565 I: 1 bit
566 idr_flag. This component specifies whether the layer
567 representation is an instantaneous decoding refresh (IDR) layer
568 representation (when equal to 1) or not (when equal to 0).
569
570 PRID: 6 bits
571 priority_id. This flag specifies a priority identifier for the
572 NAL unit. A lower value of PRID indicates a higher priority.
573
574 N: 1 bit
575 no_inter_layer_pred_flag. This flag specifies, when present in
576 a coded slice NAL unit, whether inter-layer prediction may be
577 used for decoding the coded slice (when equal to 1) or not
578 (when equal to 0).
579
580 DID: 3 bits
581 dependency_id. This component indicates the inter-layer coding
582 dependency level of a layer representation. At any access
583 unit, a layer representation with a given dependency_id may be
584 used for inter-layer prediction for coding of a layer
585 representation with a higher dependency_id, while a layer
586 representation with a given dependency_id shall not be used for
587 inter-layer prediction for coding of a layer representation
588 with a lower dependency_id.
589
590 QID: 4 bits
591 quality_id. This component indicates the quality level of an
592 MGS layer representation. At any access unit and for identical
593 dependency_id values, a layer representation with quality_id
594 equal to ql uses a layer representation with quality_id equal
595 to ql-1 for inter-layer prediction.
596
597 TID: 3 bits
598 temporal_id. This component indicates the temporal level of a
599 layer representation. The temporal_id is associated with the
600 frame rate, with lower values of _temporal_id corresponding to
601 lower frame rates. A layer representation at a given
602 temporal_id typically depends on layer representations with
603 lower temporal_id values, but it never depends on layer
604 representations with higher temporal_id values.
605
606
607
608
609
610
611
612 <span class="grey">Wenger, et al. Standards Track [Page 11]</span></pre>
613 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-12" ></span>
614 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
615
616
617 U: 1 bit
618 use_ref_base_pic_flag. A value of 1 indicates that only
619 reference base pictures are used during the inter prediction
620 process. A value of 0 indicates that the reference base
621 pictures are not used during the inter prediction process.
622
623 D: 1 bit
624 discardable_flag. A value of 1 indicates that the current NAL
625 unit is not used for decoding NAL units with values of
626 dependency_id higher than the one of the current NAL unit, in
627 the current and all subsequent access units. Such NAL units
628 can be discarded without risking the integrity of layers with
629 higher dependency_id values. discardable_flag equal to 0
630 indicates that the decoding of the NAL unit is required to
631 maintain the integrity of layers with higher dependency_id.
632
633 O: 1 bit
634 output_flag: Affects the decoded picture output process as
635 defined in Annex C of [<a href="#ref-H.264" title="&quot;Advanced video coding for generic audiovisual services&quot;">H.264</a>].
636
637 RR: 2 bits
638 reserved_three_2bits. Reserved bits for future extension. RR
639 MUST be equal to "11" (in binary form). The value of RR must
640 be ignored by decoders.
641
642 This memo extends the semantics of F, NRI, I, PRID, DID, QID, TID, U,
643 and D per Annex G of [<a href="#ref-H.264" title="&quot;Advanced video coding for generic audiovisual services&quot;">H.264</a>] as described in <a href="#section-4.2">Section 4.2</a>.
644
645 <span class="h3"><a class="selflink" id="section-1.2" href="#section-1.2">1.2</a>. Overview of the Payload Format</span>
646
647 Similar to [<a href="./rfc6184" title="&quot;RTP Payload Format for H.264 Video&quot;">RFC6184</a>], this payload format can only be used to carry
648 the raw NAL unit stream over RTP and not the bytestream format
649 specified in Annex B of [<a href="#ref-H.264" title="&quot;Advanced video coding for generic audiovisual services&quot;">H.264</a>].
650
651 The design principles, transmission modes, and packetization modes as
652 well as new payload structures are summarized in this section. It is
653 assumed that the reader is familiar with the terminology and concepts
654 defined in [<a href="./rfc6184" title="&quot;RTP Payload Format for H.264 Video&quot;">RFC6184</a>].
655
656 <span class="h4"><a class="selflink" id="section-1.2.1" href="#section-1.2.1">1.2.1</a>. Design Principles</span>
657
658 The following design principles have been observed for this payload
659 format:
660
661 o Backward compatibility with [<a href="./rfc6184" title="&quot;RTP Payload Format for H.264 Video&quot;">RFC6184</a>] wherever possible.
662
663
664
665
666
667
668 <span class="grey">Wenger, et al. Standards Track [Page 12]</span></pre>
669 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-13" ></span>
670 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
671
672
673 o The SVC base layer or any H.264/AVC compatible subset of the SVC
674 base layer, when transmitted in its own RTP stream, must be
675 encapsulated using [<a href="./rfc6184" title="&quot;RTP Payload Format for H.264 Video&quot;">RFC6184</a>]. This ensures that such an RTP
676 stream can be understood by [<a href="./rfc6184" title="&quot;RTP Payload Format for H.264 Video&quot;">RFC6184</a>] receivers.
677
678 o Media-aware network elements (MANEs) as defined in [<a href="./rfc6184" title="&quot;RTP Payload Format for H.264 Video&quot;">RFC6184</a>] are
679 signaling-aware, rely on signaling information, and have state.
680
681 o MANEs can aggregate multiple RTP streams, possibly from multiple
682 RTP sessions.
683
684 o MANEs can perform media-aware stream thinning (selective
685 elimination of packets or portions thereof). By using the payload
686 header information identifying layers within an RTP session, MANEs
687 are able to remove packets or portions thereof from the incoming
688 RTP packet stream. This implies rewriting the RTP headers of the
689 outgoing packet stream, and rewriting of RTCP packets as specified
690 in <a href="./rfc3550#section-7">Section&nbsp;7 of [RFC3550]</a>.
691
692 <span class="h4"><a class="selflink" id="section-1.2.2" href="#section-1.2.2">1.2.2</a>. Transmission Modes and Packetization Modes</span>
693
694 This memo allows the packetization of SVC data for both single-
695 session transmission (SST) and multi-session transmission (MST). In
696 the case of SST all SVC data are carried in a single RTP session. In
697 the case of MST two or more RTP sessions are used to carry the SVC
698 data, in accordance with the MST-specific packetization modes defined
699 in this memo, which are based on the packetization modes defined in
700 [<a href="./rfc6184" title="&quot;RTP Payload Format for H.264 Video&quot;">RFC6184</a>]. In MST, each RTP session is associated with one RTP
701 stream, which may carry one or more layers.
702
703 The base layer is, by design, compatible to H.264/AVC. During
704 transmission, the associated prefix NAL units, which are introduced
705 by SVC and, when present, are ignored by H.264/AVC decoders, may be
706 encapsulated within the same RTP packet stream as the H.264/AVC VCL
707 NAL units or in a different RTP packet stream (when MST is used).
708 For convenience, the term "AVC base layer" is used to refer to the
709 base layer without prefix NAL units, while the term "SVC base layer"
710 is used to refer to the base layer with prefix NAL units.
711
712 Furthermore, the base layer may have multiple temporal components
713 (i.e., supporting different frame rates). As a result, the lowest
714 temporal component ("T0") of the AVC or SVC base layer is used as the
715 starting point of the SVC bitstream hierarchy.
716
717 This memo allows encapsulating in a given RTP stream any of the
718 following three alternatives of layer combinations:
719
720
721
722
723
724 <span class="grey">Wenger, et al. Standards Track [Page 13]</span></pre>
725 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-14" ></span>
726 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
727
728
729 1. the T0 AVC base layer or the T0 SVC base layer only;
730 2. one or more enhancement layers only; or
731 3. the T0 SVC base layer, and one or more enhancement layers.
732
733 SST should be used in point-to-point unicast applications and, in
734 general, whenever the potential benefit of using multiple RTP
735 sessions does not justify the added complexity. When SST is used,
736 the layer combination cases 1 and 3 above can be used. When an
737 H.264/AVC compatible subset of the SVC base layer is transmitted
738 using SST, the packetization of [<a href="./rfc6184" title="&quot;RTP Payload Format for H.264 Video&quot;">RFC6184</a>] must be used, thus ensuring
739 compatibility with [<a href="./rfc6184" title="&quot;RTP Payload Format for H.264 Video&quot;">RFC6184</a>] receivers. When, however, one or more
740 SVC quality or spatial enhancement layers are transmitted using SST,
741 the packetization defined in this memo must be used. In SST, any of
742 the three [<a href="./rfc6184" title="&quot;RTP Payload Format for H.264 Video&quot;">RFC6184</a>] packetization modes, namely, single NAL unit
743 mode, non-interleaved mode, and interleaved mode, can be used.
744
745 MST should be used in a multicast session when different receivers
746 may request different layers of the scalable bitstream. An operation
747 point for an SVC bitstream, as defined in this memo, corresponds to a
748 set of layers that together conform to one of the profiles defined in
749 Annex A or G of [<a href="#ref-H.264" title="&quot;Advanced video coding for generic audiovisual services&quot;">H.264</a>] and, when decoded, offer a representation of
750 the original video at a certain fidelity. The number of streams used
751 in MST should be at least equal to the number of operation points
752 that may be requested by the receivers. Depending on the
753 application, this may result in each layer being carried in its own
754 RTP session, or in having multiple layers encapsulated within one RTP
755 session.
756
757 Informative note: Layered multicast is a term commonly used to
758 describe the application where multicast is used to transmit
759 layered or scalable data that has been encapsulated into more than
760 one RTP session. This application allows different receivers in
761 the multicast session to receive different operation points of the
762 scalable bitstream. Layered multicast, among other application
763 examples, is discussed in more detail in <a href="#section-11.2">Section 11.2</a>.
764
765 When MST is used, any of the three layer combinations above can be
766 used for each of the sessions. When an H.264/AVC compatible subset
767 of the SVC base layer is transmitted in its own session in MST, the
768 packetization of [<a href="./rfc6184" title="&quot;RTP Payload Format for H.264 Video&quot;">RFC6184</a>] must be used, such that [<a href="./rfc6184" title="&quot;RTP Payload Format for H.264 Video&quot;">RFC6184</a>]
769 receivers can be part of the MST and receive only this session. For
770 MST, this memo defines four different MST-specific packetization
771 modes, namely, non-interleaved timestamp (NI-T) based mode, non-
772 interleaved CS-DON (NI-C) based mode, non-interleaved combined
773 timestamp and CS-DON mode (NI-TC), and interleaved CS-DON (I-C) based
774 mode (detailed in <a href="#section-4.5.2">Section 4.5.2</a>). The modes differ depending on
775 whether the SVC data are allowed to be interleaved, i.e., to be
776 transmitted in an order different than the intended decoding order,
777
778
779
780 <span class="grey">Wenger, et al. Standards Track [Page 14]</span></pre>
781 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-15" ></span>
782 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
783
784
785 and they also differ in the mechanisms provided in order to recover
786 the correct decoding order of the NAL units across the multiple RTP
787 sessions. These four MST modes reuse the packetization modes
788 introduced in [<a href="./rfc6184" title="&quot;RTP Payload Format for H.264 Video&quot;">RFC6184</a>] for the packetization of NAL units in each of
789 their individual RTP sessions.
790
791 As the names of the MST packetization modes imply, the NI-T, NI-C,
792 and NI-TC modes do not allow interleaved transmission, while the I-C
793 mode allows interleaved transmission. With any of the three non-
794 interleaved MST packetization modes, legacy [<a href="./rfc6184" title="&quot;RTP Payload Format for H.264 Video&quot;">RFC6184</a>] receivers with
795 implementation of the non-interleaved mode specified in [<a href="./rfc6184" title="&quot;RTP Payload Format for H.264 Video&quot;">RFC6184</a>] can
796 join a multi-session transmission of SVC, to receive the base RTP
797 session encapsulated according to [<a href="./rfc6184" title="&quot;RTP Payload Format for H.264 Video&quot;">RFC6184</a>].
798
799 <span class="h4"><a class="selflink" id="section-1.2.3" href="#section-1.2.3">1.2.3</a>. New Payload Structures</span>
800
801 [<a id="ref-RFC6184">RFC6184</a>] specifies three basic payload structures, namely, single
802 NAL unit packet, aggregation packet, and fragmentation unit.
803 Depending on the basic payload structure, an RTP packet may contain a
804 NAL unit not aggregating other NAL units, one or more NAL units
805 aggregated in another NAL unit, or a fragment of a NAL unit not
806 aggregating other NAL units. Each NAL unit of a type specified in
807 [<a href="#ref-H.264" title="&quot;Advanced video coding for generic audiovisual services&quot;">H.264</a>] (i.e., 1 to 23, inclusive) may be carried in its entirety in
808 a single NAL unit packet, may be aggregated in an aggregation packet,
809 or may be fragmented and carried in a number of fragmentation unit
810 packets. To enable aggregation or fragmentation of NAL units while
811 still ensuring that the RTP packet payload is only composed of NAL
812 units, [<a href="./rfc6184" title="&quot;RTP Payload Format for H.264 Video&quot;">RFC6184</a>] introduced six new NAL unit types (24-29) to be used
813 as payload structures, selected from the NAL unit types left
814 unspecified in [<a href="#ref-H.264" title="&quot;Advanced video coding for generic audiovisual services&quot;">H.264</a>].
815
816 This memo reuses all the payload structures used in [<a href="./rfc6184" title="&quot;RTP Payload Format for H.264 Video&quot;">RFC6184</a>].
817 Furthermore, three new types of NAL units are defined: payload
818 content scalability information (PACSI) NAL unit, empty NAL unit, and
819 non-interleaved multi-time aggregation packet (NI-MTAP) (specified in
820 Sections <a href="#section-4.9">4.9</a>, <a href="#section-4.10">4.10</a>, and <a href="#section-4.7.1">4.7.1</a>, respectively).
821
822 PACSI NAL units may be used for the following purposes:
823
824 o To enable MANEs to decide whether to forward, process, or discard
825 aggregation packets, by checking in PACSI NAL units the
826 scalability information and other characteristics of the
827 aggregated NAL units, rather than looking into the aggregated NAL
828 units themselves, which are defined by the video coding
829 specification.
830
831
832
833
834
835
836 <span class="grey">Wenger, et al. Standards Track [Page 15]</span></pre>
837 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-16" ></span>
838 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
839
840
841 o To enable correct decoding order recovery in MST using the NI-C or
842 NI-TC mode, with the help of the CS-DON information included in
843 PACSI NAL units.
844
845 o To improve resilience to packet losses, e.g., by utilizing the
846 following data or information included in PACSI NAL units:
847 repeated Supplemental Enhancement Information (SEI) messages,
848 information regarding the start and end of layer representations,
849 and the indices to layer representations of the lowest temporal
850 subset.
851
852 Empty NAL units may be used to enable correct decoding order recovery
853 in MST using the NI-T or NI-TC mode. NI-MTAP NAL units may be used
854 to aggregate NAL units from multiple access units but without
855 interleaving.
856
857 <span class="h2"><a class="selflink" id="section-2" href="#section-2">2</a>. Conventions</span>
858
859 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
860 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
861 document are to be interpreted as described in <a href="https://www.rfc-editor.org/bcp/bcp14">BCP 14</a>, <a href="./rfc2119">RFC 2119</a>
862 [<a href="./rfc2119" title="&quot;Key words for use in RFCs to Indicate Requirement Levels&quot;">RFC2119</a>].
863
864 This specification uses the notion of setting and clearing a bit when
865 bit fields are handled. Setting a bit is the same as assigning that
866 bit the value of 1 (On). Clearing a bit is the same as assigning
867 that bit the value of 0 (Off).
868
869 <span class="h2"><a class="selflink" id="section-3" href="#section-3">3</a>. Definitions and Abbreviations</span>
870
871 <span class="h3"><a class="selflink" id="section-3.1" href="#section-3.1">3.1</a>. Definitions</span>
872
873 This document uses the terms and definitions of [<a href="#ref-H.264" title="&quot;Advanced video coding for generic audiovisual services&quot;">H.264</a>]. <a href="#section-3.1.1">Section</a>
874 <a href="#section-3.1.1">3.1.1</a> lists relevant definitions copied from [<a href="#ref-H.264" title="&quot;Advanced video coding for generic audiovisual services&quot;">H.264</a>] for convenience.
875
876 When there is discrepancy, the definitions in [<a href="#ref-H.264" title="&quot;Advanced video coding for generic audiovisual services&quot;">H.264</a>] take
877 precedence. <a href="#section-3.1.2">Section 3.1.2</a> gives definitions specific to this memo.
878 Some of the definitions in <a href="#section-3.1.2">Section 3.1.2</a> are also present in
879 [<a href="./rfc6184" title="&quot;RTP Payload Format for H.264 Video&quot;">RFC6184</a>] and copied here with slight adaptations as needed.
880
881 <span class="h4"><a class="selflink" id="section-3.1.1" href="#section-3.1.1">3.1.1</a>. Definitions from the SVC Specification</span>
882
883 access unit: A set of NAL units always containing exactly one primary
884 coded picture. In addition to the primary coded picture, an access
885 unit may also contain one or more redundant coded pictures, one
886 auxiliary coded picture, or other NAL units not containing slices or
887 slice data partitions of a coded picture. The decoding of an access
888 unit always results in a decoded picture.
889
890
891
892 <span class="grey">Wenger, et al. Standards Track [Page 16]</span></pre>
893 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-17" ></span>
894 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
895
896
897 base layer: A bitstream subset that contains all the NAL units with
898 the nal_unit_type syntax element equal to 1 or 5 of the bitstream and
899 does not contain any NAL unit with the nal_unit_type syntax element
900 equal to 14, 15, or 20 and conforms to one or more of the profiles
901 specified in Annex A of [<a href="#ref-H.264" title="&quot;Advanced video coding for generic audiovisual services&quot;">H.264</a>].
902
903 base quality layer representation: The layer representation of the
904 target dependency representation of an access unit that is associated
905 with the quality_id syntax element equal to 0.
906
907 coded video sequence: A sequence of access units that consists, in
908 decoding order, of an IDR access unit followed by zero or more non-
909 IDR access units including all subsequent access units up to but not
910 including any subsequent IDR access unit.
911
912 dependency representation: A subset of Video Coding Layer (VCL) NAL
913 units within an access unit that are associated with the same value
914 of the dependency_id syntax element, which is provided as part of the
915 NAL unit header or by an associated prefix NAL unit. A dependency
916 representation consists of one or more layer representations.
917
918 IDR access unit: An access unit in which the primary coded picture is
919 an IDR picture.
920
921 IDR picture: Instantaneous decoding refresh picture. A coded picture
922 in which all slices of the target dependency representation within
923 the access unit are I or EI slices that causes the decoding process
924 to mark all reference pictures as "unused for reference" immediately
925 after decoding the IDR picture. After the decoding of an IDR picture
926 all following coded pictures in decoding order can be decoded without
927 inter prediction from any picture decoded prior to the IDR picture.
928 The first picture of each coded video sequence is an IDR picture.
929
930 layer representation: A subset of VCL NAL units within an access unit
931 that are associated with the same values of the dependency_id and
932 quality_id syntax elements, which are provided as part of the VCL NAL
933 unit header or by an associated prefix NAL unit. One or more layer
934 representations represent a dependency representation.
935
936 prefix NAL unit: A NAL unit with nal_unit_type equal to 14 that
937 immediately precedes in decoding order a NAL unit with nal_unit_type
938 equal to 1, 5, or 12. The NAL unit that immediately succeeds in
939 decoding order the prefix NAL unit is referred to as the associated
940 NAL unit. The prefix NAL unit contains data associated with the
941 associated NAL unit, which are considered to be part of the
942 associated NAL unit.
943
944
945
946
947
948 <span class="grey">Wenger, et al. Standards Track [Page 17]</span></pre>
949 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-18" ></span>
950 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
951
952
953 reference base picture: A reference picture that is obtained by
954 decoding a base quality layer representation with the nal_ref_idc
955 syntax element not equal to 0 and the store_ref_base_pic_flag syntax
956 element equal to 1 of an access unit and all layer representations of
957 the access unit that are referred to by inter-layer prediction of the
958 base quality layer representation. A reference base picture is not
959 an output of the decoding process, but the samples of a reference
960 base picture may be used for inter prediction in the decoding process
961 of subsequent pictures in decoding order. Reference base picture is
962 a collective term for a reference base field or a reference base
963 frame.
964
965 scalable bitstream: A bitstream with the property that one or more
966 bitstream subsets that are not identical to the scalable bitstream
967 form another bitstream that conforms to the SVC specification
968 [<a href="#ref-H.264" title="&quot;Advanced video coding for generic audiovisual services&quot;">H.264</a>].
969
970 target dependency representation: The dependency representation of an
971 access unit that is associated with the largest value of the
972 dependency_id syntax element for all dependency representations of
973 the access unit.
974
975 target layer representation: The layer representation of the target
976 dependency representation of an access unit that is associated with
977 the largest value of the quality_id syntax element for all layer
978 representations of the target dependency representation of the access
979 unit.
980
981 <span class="h4"><a class="selflink" id="section-3.1.2" href="#section-3.1.2">3.1.2</a>. Definitions Specific to This Memo</span>
982
983 anchor layer representation: An anchor layer representation is such a
984 layer representation that, if decoding of the operation point
985 corresponding to the layer starts from the access unit containing
986 this layer representation, all the following layer representations of
987 the layer, in output order, can be correctly decoded. The output
988 order is defined in [<a href="#ref-H.264" title="&quot;Advanced video coding for generic audiovisual services&quot;">H.264</a>] as the order in which decoded pictures
989 are output from the decoded picture buffer of the decoder. As H.264
990 does not specify the picture display process, this more general term
991 is used instead of display order. An anchor layer representation is
992 a random access point to the layer the anchor layer representation
993 belongs. However, some layer representations, succeeding an anchor
994 layer representation in decoding order but preceding the anchor layer
995 representation in output order, may refer to earlier layer
996 representations for inter prediction, and hence the decoding may be
997 incorrect if random access is performed at the anchor layer
998 representation.
999
1000
1001
1002
1003
1004 <span class="grey">Wenger, et al. Standards Track [Page 18]</span></pre>
1005 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-19" ></span>
1006 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
1007
1008
1009 AVC base layer: The subset of the SVC base layer in which all prefix
1010 NAL units (type 14) are removed. Note that this is equivalent to the
1011 term "base layer" as defined in Annex G of [<a href="#ref-H.264" title="&quot;Advanced video coding for generic audiovisual services&quot;">H.264</a>].
1012
1013 base RTP session: When multi-session transmission is used, the RTP
1014 session that carries the RTP stream containing the T0 AVC base layer
1015 or the T0 SVC base layer, and zero or more enhancement layers. This
1016 RTP session does not depend on any other RTP session as indicated by
1017 mechanisms defined in <a href="#section-7.2.3">Section 7.2.3</a>. The base RTP session may carry
1018 NAL units of NAL unit type equal to 14 and 15.
1019
1020 decoding order number (DON): A field in the payload structure or a
1021 derived variable indicating NAL unit decoding order. Values of DON
1022 are in the range of 0 to 65535, inclusive. After reaching the
1023 maximum value, the value of DON wraps around to 0. Note that this
1024 definition also exists in [<a href="./rfc6184" title="&quot;RTP Payload Format for H.264 Video&quot;">RFC6184</a>] in exactly the same form.
1025
1026 Empty NAL unit: A NAL unit with NAL unit type equal to 31 and sub-
1027 type equal to 1. An empty NAL unit consists of only the two-byte NAL
1028 unit header with an empty payload.
1029
1030 enhancement RTP session: When multi-session transmission is used, an
1031 RTP session that is not the base RTP session. An enhancement RTP
1032 session typically contains an RTP stream that depends on at least one
1033 other RTP session as indicated by mechanisms defined in <a href="#section-7.2.3">Section</a>
1034 <a href="#section-7.2.3">7.2.3</a>. A lower RTP session to an enhancement RTP session is an RTP
1035 session on which the enhancement RTP session depends. The lowest RTP
1036 session for a receiver is the RTP session that does not depend on any
1037 other RTP session received by the receiver. The highest RTP session
1038 for a receiver is the RTP session on which no other RTP session
1039 received by the receiver depends.
1040
1041 cross-session decoding order number (CS-DON): A derived variable
1042 indicating NAL unit decoding order number over all NAL units within
1043 all the session-multiplexed RTP sessions that carry the same SVC
1044 bitstream.
1045
1046 default level: The level indicated by the profile-level-id parameter.
1047 In Session Description Protocol (SDP) Offer/Answer, the level is
1048 downgradable, i.e., the answer may either use the default level or a
1049 lower level. Note that this definition also exists in [<a href="./rfc6184" title="&quot;RTP Payload Format for H.264 Video&quot;">RFC6184</a>] in a
1050 slightly different form.
1051
1052 default sub-profile: The subset of coding tools, which may be all
1053 coding tools of one profile or the common subset of coding tools of
1054 more than one profile, indicated by the profile-level-id parameter.
1055 In SDP Offer/Answer, the default sub-profile must be used in a
1056
1057
1058
1059
1060 <span class="grey">Wenger, et al. Standards Track [Page 19]</span></pre>
1061 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-20" ></span>
1062 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
1063
1064
1065 symmetric manner, i.e., the answer must either use the same sub-
1066 profile as the offer or reject the offer. Note that this definition
1067 also exists in [<a href="./rfc6184" title="&quot;RTP Payload Format for H.264 Video&quot;">RFC6184</a>] in a slightly different form.
1068
1069 enhancement layer: A layer in which at least one of the values of
1070 dependency_id or quality_id is higher than 0, or a layer in which
1071 none of the NAL units is associated with the value of temporal_id
1072 equal to 0. An operation point constructed using the maximum
1073 temporal_id, dependency_id, and quality_id values associated with an
1074 enhancement layer may or may not conform to one or more of the
1075 profiles specified in Annex A of [<a href="#ref-H.264" title="&quot;Advanced video coding for generic audiovisual services&quot;">H.264</a>].
1076
1077 H.264/AVC compatible: The property of a bitstream subset of
1078 conforming to one or more of the profiles specified in Annex A of
1079 [<a href="#ref-H.264" title="&quot;Advanced video coding for generic audiovisual services&quot;">H.264</a>].
1080
1081 intra layer representation: A layer representation that contains
1082 only slices that use intra prediction, and hence do not refer to any
1083 earlier layer representation in decoding order in the same layer.
1084 Note that in SVC intra prediction includes intra-layer intra
1085 prediction as well as inter-layer intra prediction.
1086
1087 layer: A bitstream subset in which all NAL units of type 1, 5, 12,
1088 14, or 20 have the same values of dependency_id and quality_id,
1089 either directly through their NAL unit header (for NAL units of type
1090 14 or 20) or through association to a prefix (type 14) NAL unit (for
1091 NAL unit type 1, 5, or 12). A layer may contain NAL units associated
1092 with more than one values of temporal_id.
1093
1094 media-aware network element (MANE): A network element, such as a
1095 middlebox or application layer gateway that is capable of parsing
1096 certain aspects of the RTP payload headers or the RTP payload and
1097 reacting to their contents. Note that this definition also exists in
1098 [<a href="./rfc6184" title="&quot;RTP Payload Format for H.264 Video&quot;">RFC6184</a>] in exactly the same form.
1099
1100 Informative note: The concept of a MANE goes beyond normal routers
1101 or gateways in that a MANE has to be aware of the signaling (e.g.,
1102 to learn about the payload type mappings of the media streams),
1103 and in that it has to be trusted when working with Secure Real-
1104 time Transport Protocol (SRTP). The advantage of using MANEs is
1105 that they allow packets to be dropped according to the needs of
1106 the media coding. For example, if a MANE has to drop packets due
1107 to congestion on a certain link, it can identify and remove those
1108 packets whose elimination produces the least adverse effect on the
1109 user experience. After dropping packets, MANEs must rewrite RTCP
1110 packets to match the changes to the RTP packet stream as specified
1111 in <a href="./rfc3550#section-7">Section&nbsp;7 of [RFC3550]</a>.
1112
1113
1114
1115
1116 <span class="grey">Wenger, et al. Standards Track [Page 20]</span></pre>
1117 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-21" ></span>
1118 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
1119
1120
1121 multi-session transmission: The transmission mode in which the SVC
1122 stream is transmitted over multiple RTP sessions. Dependency between
1123 RTP sessions MUST be signaled according to <a href="#section-7.2.3">Section 7.2.3</a> of this
1124 memo.
1125
1126 NAL unit decoding order: A NAL unit order that conforms to the
1127 constraints on NAL unit order given in Section G.7.4.1.2 in [<a href="#ref-H.264" title="&quot;Advanced video coding for generic audiovisual services&quot;">H.264</a>].
1128 Note that this definition also exists in [<a href="./rfc6184" title="&quot;RTP Payload Format for H.264 Video&quot;">RFC6184</a>] in a slightly
1129 different form.
1130
1131 NALU-time: The value that the RTP timestamp would have if the NAL
1132 unit would be transported in its own RTP packet. Note that this
1133 definition also exists in [<a href="./rfc6184" title="&quot;RTP Payload Format for H.264 Video&quot;">RFC6184</a>] in exactly the same form.
1134
1135 operation point: An operation point is identified by a set of values
1136 of temporal_id, dependency_id, and quality_id. A bitstream
1137 corresponding to an operation point can be constructed by removing
1138 all NAL units associated with a higher value of dependency_id, and
1139 all NAL units associated with the same value of dependency_id but
1140 higher values of quality_id or temporal_id. An operation point
1141 bitstream conforms to at least one of the profiles defined in Annex A
1142 or G of [<a href="#ref-H.264" title="&quot;Advanced video coding for generic audiovisual services&quot;">H.264</a>], and offers a representation of the original video
1143 signal at a certain fidelity.
1144
1145 Informative note: Additional NAL units may be removed (with lower
1146 dependency_id or same dependency_id but lower quality_id) if they
1147 are not required for decoding the bitstream at the particular
1148 operation point. The resulting bitstream, however, may no longer
1149 conform to any of the profiles defined in Annex A or G of [<a href="#ref-H.264" title="&quot;Advanced video coding for generic audiovisual services&quot;">H.264</a>].
1150
1151 operation point representation: The set of all NAL units of an
1152 operation point within the same access unit.
1153
1154 RTP packet stream: A sequence of RTP packets with increasing sequence
1155 numbers (except for wrap-around), identical payload type and
1156 identical SSRC (Synchronization Source), carried in one RTP session.
1157 Within the scope of this memo, one RTP packet stream is utilized to
1158 transport one or more layers.
1159
1160 single-session transmission: The transmission mode in which the SVC
1161 bitstream is transmitted over a single RTP session.
1162
1163 SVC base layer: The layer that includes all NAL units associated with
1164 dependency_id and quality_id values both equal to 0, including prefix
1165 NAL units (NAL unit type 14).
1166
1167
1168
1169
1170
1171
1172 <span class="grey">Wenger, et al. Standards Track [Page 21]</span></pre>
1173 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-22" ></span>
1174 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
1175
1176
1177 SVC enhancement layer: A layer in which at least one of the values of
1178 dependency_id or quality_id is higher than 0. An operation point
1179 constructed using the maximum dependency_id and quality_id values and
1180 any temporal_id value associated with an SVC enhancement layer does
1181 not conform to any of the profiles specified in Annex A of [<a href="#ref-H.264" title="&quot;Advanced video coding for generic audiovisual services&quot;">H.264</a>].
1182
1183 SVC NAL unit: A NAL unit of NAL unit type 14, 15, or 20 as specified
1184 in Annex G of [<a href="#ref-H.264" title="&quot;Advanced video coding for generic audiovisual services&quot;">H.264</a>].
1185
1186 SVC NAL unit header: A four-byte header resulting from the addition
1187 of a three-byte SVC-specific header extension added in NAL unit types
1188 14 and 20.
1189
1190 SVC RTP session: Either the base RTP session or an enhancement RTP
1191 session.
1192
1193 T0 AVC base layer: A subset of the AVC base layer constructed by
1194 removing all VCL NAL units associated with temporal_id values higher
1195 than 0 and non-VCL NAL units and SEI messages associated only with
1196 the VCL NAL units being removed.
1197
1198 T0 SVC base layer: A subset of the SVC base layer constructed by
1199 removing all VCL NAL units associated with temporal_id values higher
1200 than 0 as well as prefix NAL units, non-VCL NAL units, and SEI
1201 messages associated only with the VCL NAL units being removed.
1202
1203 transmission order: The order of packets in ascending RTP sequence
1204 number order (in modulo arithmetic). Within an aggregation packet,
1205 the NAL unit transmission order is the same as the order of
1206 appearance of NAL units in the packet. Note that this definition
1207 also exists in [<a href="./rfc6184" title="&quot;RTP Payload Format for H.264 Video&quot;">RFC6184</a>] in exactly the same form.
1208
1209 <span class="h3"><a class="selflink" id="section-3.2" href="#section-3.2">3.2</a>. Abbreviations</span>
1210
1211 In addition to the abbreviations defined in [<a href="./rfc6184" title="&quot;RTP Payload Format for H.264 Video&quot;">RFC6184</a>], the following
1212 abbreviations are used in this memo.
1213
1214 CGS: Coarse-Grain Scalability
1215 CS-DON: Cross-Session Decoding Order Number
1216 MGS: Medium-Grain Scalability
1217 MST: Multi-Session Transmission
1218 PACSI: Payload Content Scalability Information
1219 SST: Single-Session Transmission
1220 SNR: Signal-to-Noise Ratio
1221 SVC: Scalable Video Coding
1222
1223
1224
1225
1226
1227
1228 <span class="grey">Wenger, et al. Standards Track [Page 22]</span></pre>
1229 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-23" ></span>
1230 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
1231
1232
1233 <span class="h2"><a class="selflink" id="section-4" href="#section-4">4</a>. RTP Payload Format</span>
1234
1235 <span class="h3"><a class="selflink" id="section-4.1" href="#section-4.1">4.1</a>. RTP Header Usage</span>
1236
1237 In addition to <a href="./rfc6184#section-5.1">Section&nbsp;5.1 of [RFC6184]</a>, the following rules apply.
1238
1239 o Setting of the M bit:
1240
1241 The M bit of an RTP packet for which the packet payload is an NI-MTAP
1242 MUST be equal to 1 if the last NAL unit, in decoding order, of the
1243 access unit associated with the RTP timestamp is contained in the
1244 packet.
1245
1246 o Setting of the RTP timestamp:
1247
1248 For an RTP packet for which the packet payload is an empty NAL unit,
1249 the RTP timestamp must be set according to <a href="#section-4.10">Section 4.10</a>.
1250
1251 For an RTP packet for which the packet payload is a PACSI NAL unit,
1252 the RTP timestamp MUST be equal to the NALU-time of the next non-
1253 PACSI NAL unit in transmission order. Recall that the NALU-time of a
1254 NAL unit in an MTAP is defined in [<a href="./rfc6184" title="&quot;RTP Payload Format for H.264 Video&quot;">RFC6184</a>] as the value that the RTP
1255 timestamp would have if that NAL unit would be transported in its own
1256 RTP packet.
1257
1258 o Setting of the SSRC:
1259
1260 For both SST and MST, the SSRC values MUST be set according to
1261 [<a href="./rfc3550" title="&quot;RTP: A Transport Protocol for Real-Time Applications&quot;">RFC3550</a>].
1262
1263 <span class="h3"><a class="selflink" id="section-4.2" href="#section-4.2">4.2</a>. NAL Unit Extension and Header Usage</span>
1264
1265 <span class="h4"><a class="selflink" id="section-4.2.1" href="#section-4.2.1">4.2.1</a>. NAL Unit Extension</span>
1266
1267 This memo specifies a NAL unit extension mechanism to allow for
1268 introduction of new types of NAL units, beyond the three NAL unit
1269 types left undefined in [<a href="./rfc6184" title="&quot;RTP Payload Format for H.264 Video&quot;">RFC6184</a>] (i.e., 0, 30, and 31). The
1270 extension mechanism utilizes the NAL unit type value 31 and is
1271 specified as follows. When the NAL unit type value is equal to 31,
1272 the one-byte NAL unit header consisting of the F, NRI, and Type
1273 fields as specified in <a href="#section-1.1.3">Section 1.1.3</a> is extended by one additional
1274 octet, which consists of a 5-bit field named Subtype and three 1-bit
1275 fields named J, K, and L, respectively. The additional octet is
1276 shown in the following figure.
1277
1278
1279
1280
1281
1282
1283
1284 <span class="grey">Wenger, et al. Standards Track [Page 23]</span></pre>
1285 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-24" ></span>
1286 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
1287
1288
1289 +---------------+
1290 |0|1|2|3|4|5|6|7|
1291 +-+-+-+-+-+-+-+-+
1292 | Subtype |J|K|L|
1293 +---------------+
1294
1295 The Subtype value determines the (extended) NAL unit type of this NAL
1296 unit. The interpretation of the fields J, K, and L depends on the
1297 Subtype. The semantics of the fields are as follows.
1298
1299 When Subtype is equal to 1, the NAL unit is an empty NAL unit as
1300 specified in <a href="#section-4.10">Section 4.10</a>. When Subtype is equal to 2, the NAL unit
1301 is an NI-MTAP NAL unit as specified in <a href="#section-4.7.1">Section 4.7.1</a>. All other
1302 values of Subtype (0, 3-31) are reserved for future extensions, and
1303 receivers MUST ignore the entire NAL unit when Subtype is equal to
1304 any of these reserved values.
1305
1306 <span class="h4"><a class="selflink" id="section-4.2.2" href="#section-4.2.2">4.2.2</a>. NAL Unit Header Usage</span>
1307
1308 The structure and semantics of the NAL unit header according to the
1309 H.264 specification [<a href="#ref-H.264" title="&quot;Advanced video coding for generic audiovisual services&quot;">H.264</a>] were introduced in <a href="#section-1.1.3">Section 1.1.3</a>. This
1310 section specifies the extended semantics of the NAL unit header
1311 fields F, NRI, I, PRID, DID, QID, TID, U, and D, according to this
1312 memo. When the Type field is equal to 31, the semantics of the
1313 fields in the extension NAL unit header were specified in <a href="#section-4.2.1">Section</a>
1314 <a href="#section-4.2.1">4.2.1</a>.
1315
1316 The semantics of F specified in <a href="./rfc6184#section-5.3">Section&nbsp;5.3 of [RFC6184]</a> also apply
1317 in this memo. That is, a value of 0 for F indicates that the NAL
1318 unit type octet and payload should not contain bit errors or other
1319 syntax violations, whereas a value of 1 for F indicates that the NAL
1320 unit type octet and payload may contain bit errors or other syntax
1321 violations. MANEs SHOULD set the F bit to indicate bit errors in the
1322 NAL unit.
1323
1324 For NRI, for a bitstream conforming to one of the profiles defined in
1325 Annex A of [<a href="#ref-H.264" title="&quot;Advanced video coding for generic audiovisual services&quot;">H.264</a>] and transported using [<a href="./rfc6184" title="&quot;RTP Payload Format for H.264 Video&quot;">RFC6184</a>], the semantics
1326 specified in <a href="./rfc6184#section-5.3">Section&nbsp;5.3 of [RFC6184]</a> apply, i.e., NRI also indicates
1327 the relative importance of NAL units. For a bitstream conforming to
1328 one of the profiles defined in Annex G of [<a href="#ref-H.264" title="&quot;Advanced video coding for generic audiovisual services&quot;">H.264</a>] and transported
1329 using this memo, in addition to the semantics specified in Annex G of
1330 [<a href="#ref-H.264" title="&quot;Advanced video coding for generic audiovisual services&quot;">H.264</a>], NRI also indicates the relative importance of NAL units
1331 within a layer.
1332
1333 For I, in addition to the semantics specified in Annex G of [<a href="#ref-H.264" title="&quot;Advanced video coding for generic audiovisual services&quot;">H.264</a>],
1334 according to this memo, MANEs MAY use this information to protect NAL
1335 units with I equal to 1 better than NAL units with I equal to 0.
1336 MANEs MAY also utilize information of NAL units with I equal to 1 to
1337
1338
1339
1340 <span class="grey">Wenger, et al. Standards Track [Page 24]</span></pre>
1341 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-25" ></span>
1342 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
1343
1344
1345 decide when to forward more packets for an RTP packet stream. For
1346 example, when it is detected that spatial layer switching has
1347 happened such that the operation point has changed to a higher value
1348 of DID, MANEs MAY start to forward NAL units with the higher value of
1349 DID only after forwarding a NAL unit with I equal to 1 with the
1350 higher value of DID.
1351
1352 Note that, in the context of this section, "protecting a NAL unit"
1353 means any RTP or network transport mechanism that could improve the
1354 probability of successful delivery of the packet conveying the NAL
1355 unit, including applying a Quality of Service (QoS) enabled network,
1356 Forward Error Correction (FEC), retransmissions, and advanced
1357 scheduling behavior, whenever possible.
1358
1359 For PRID, the semantics specified in Annex G of [<a href="#ref-H.264" title="&quot;Advanced video coding for generic audiovisual services&quot;">H.264</a>] apply. Note
1360 that MANEs implementing unequal error protection MAY use this
1361 information to protect NAL units with smaller PRID values better than
1362 those with larger PRID values, for example, by including only the
1363 more important NAL units in a FEC protection mechanism. The
1364 importance for the decoding process decreases as the PRID value
1365 increases.
1366
1367 For DID, QID, or TID, in addition to the semantics specified in Annex
1368 G of [<a href="#ref-H.264" title="&quot;Advanced video coding for generic audiovisual services&quot;">H.264</a>], according to this memo, values of DID, QID, or TID
1369 indicate the relative importance in their respective dimension. A
1370 lower value of DID, QID, or TID indicates a higher importance if the
1371 other two components are identical. MANEs MAY use this information
1372 to protect more important NAL units better than less important NAL
1373 units.
1374
1375 For U, in addition to the semantics specified in Annex G of [<a href="#ref-H.264" title="&quot;Advanced video coding for generic audiovisual services&quot;">H.264</a>],
1376 according to this memo, MANEs MAY use this information to protect NAL
1377 units with U equal to 1 better than NAL units with U equal to 0.
1378
1379 For D, in addition to the semantics specified in Annex G of [<a href="#ref-H.264" title="&quot;Advanced video coding for generic audiovisual services&quot;">H.264</a>],
1380 according to this memo, MANEs MAY use this information to determine
1381 whether a given NAL unit is required for successfully decoding a
1382 certain Operation Point of the SVC bitstream, hence to decide whether
1383 to forward the NAL unit.
1384
1385 <span class="h3"><a class="selflink" id="section-4.3" href="#section-4.3">4.3</a>. Payload Structures</span>
1386
1387 The NAL unit structure is central to H.264/AVC, [<a href="./rfc6184" title="&quot;RTP Payload Format for H.264 Video&quot;">RFC6184</a>], as well as
1388 SVC and this memo. In H.264/AVC and SVC, all coded bits for
1389 representing a video signal are encapsulated in NAL units. In
1390 [<a href="./rfc6184" title="&quot;RTP Payload Format for H.264 Video&quot;">RFC6184</a>], each RTP packet payload is structured as a NAL unit, which
1391 contains one or a part of one NAL unit specified in H.264/AVC, or
1392 aggregates one or more NAL units specified in H.264/AVC.
1393
1394
1395
1396 <span class="grey">Wenger, et al. Standards Track [Page 25]</span></pre>
1397 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-26" ></span>
1398 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
1399
1400
1401 [<a id="ref-RFC6184">RFC6184</a>] specifies three basic payload structures (in <a href="./rfc6184#section-5.2">Section&nbsp;5.2 of
1402 [RFC6184]</a>): single NAL unit packet, aggregation packet, fragmentation
1403 unit, and six new types (24 to 29) of NAL units. The value of the
1404 Type field of the RTP packet payload header (i.e., the first byte of
1405 the payload) may be equal to any value from 1 to 23 for a single NAL
1406 unit packet, any value from 24 to 27 for an aggregation packet, and
1407 28 or 29 for a fragmentation unit.
1408
1409 In addition to the NAL unit types defined originally for H.264/AVC,
1410 SVC defines three new NAL unit types specifically for SVC: coded
1411 slice in scalable extension NAL units (type 20), prefix NAL units
1412 (type 14), and subset sequence parameter set NAL units (type 15), as
1413 described in <a href="#section-1.1">Section 1.1</a>.
1414
1415 This memo further introduces three new types of NAL units, PACSI NAL
1416 unit (NAL unit type 30) as specified in <a href="#section-4.9">Section 4.9</a>, empty NAL unit
1417 (type 31, subtype 1) as specified in <a href="#section-4.10">Section 4.10</a>, and NI-MTAP NAL
1418 unit (type 31, subtype 2) as specified in <a href="#section-4.7.1">Section 4.7.1</a>.
1419
1420 The RTP packet payload structure in [<a href="./rfc6184" title="&quot;RTP Payload Format for H.264 Video&quot;">RFC6184</a>] is maintained with
1421 slight extensions in this memo, as follows. Each RTP packet payload
1422 is still structured as a NAL unit, which contains one or a part of
1423 one NAL unit specified in H.264/AVC and SVC, or contains one PACSI
1424 NAL unit or one empty NAL unit, or aggregates zero or more NAL units
1425 specified in H.264/AVC and SVC, zero or one PACSI NAL unit, and zero
1426 or more empty NAL units.
1427
1428 In this memo, one of the three basic payload structures,
1429 fragmentation unit, remains the same as in [<a href="./rfc6184" title="&quot;RTP Payload Format for H.264 Video&quot;">RFC6184</a>], and the other
1430 two, single NAL unit packet and aggregation packet, are extended as
1431 follows. The value of the Type field of the payload header may be
1432 equal to any value from 1 to 23, inclusive, and 30 to 31, inclusive,
1433 for a single NAL unit packet, and any value from 24 to 27, inclusive,
1434 and 31, for an aggregation packet. When the Type field of the
1435 payload header is equal to 31 and the Subtype field of the payload
1436 header is equal to 2, the packet is an aggregation packet (containing
1437 an NI-MTAP NAL unit). When the Type field of the payload header is
1438 equal to 31 and the Subtype field of the payload header is equal to
1439 1, the packet is a single NAL unit packet (containing an empty NAL
1440 unit).
1441
1442 Note that, in this memo, the length of the payload header varies
1443 depending on the value of the Type field in the first byte of the RTP
1444 packet payload. If the value is equal to 14, 20, or 30, the first
1445 four bytes of the packet payload form the payload header; otherwise,
1446 if the value is equal to 31, the first two bytes of the payload form
1447 the payload header; otherwise, the payload header is the first byte
1448 of the packet payload.
1449
1450
1451
1452 <span class="grey">Wenger, et al. Standards Track [Page 26]</span></pre>
1453 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-27" ></span>
1454 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
1455
1456
1457 Table 1 lists the NAL unit types introduced in SVC and this memo and
1458 where they are described in this memo. Table 2 summarizes the basic
1459 payload structure types for all NAL unit types when they are directly
1460 used as RTP packet payloads according to this memo. Table 3
1461 summarizes the NAL unit types allowed to be aggregated (i.e., used as
1462 aggregation units in aggregation packets) or fragmented (i.e.,
1463 carried in fragmentation units) according to this memo.
1464
1465 Table 1. NAL unit types introduced in SVC and this memo
1466
1467 Type Subtype NAL Unit Name Section Numbers
1468 -----------------------------------------------------------
1469 14 - Prefix NAL unit 1.1
1470 15 - Subset sequence parameter set 1.1
1471 20 - Coded slice in scalable extension 1.1
1472 30 - PACSI NAL unit 4.9
1473 31 0 reserved 4.2.1
1474 31 1 Empty NAL unit 4.10
1475 31 2 NI-MTAP 4.7.1
1476 31 3-31 reserved 4.2.1
1477
1478 Table 2. Basic payload structure types for all NAL unit
1479 types when they are directly used as RTP packet payloads
1480
1481 Type Subtype Basic Payload Structure
1482 ------------------------------------------
1483 0 - reserved
1484 1-23 - Single NAL Unit Packet
1485 24-27 - Aggregation Packet
1486 28-29 - Fragmentation Unit
1487 30 - Single NAL Unit Packet
1488 31 0 reserved
1489 31 1 Single NAL Unit Packet
1490 31 2 Aggregation Packet
1491 31 3-31 reserved
1492
1493
1494
1495
1496
1497
1498
1499
1500
1501
1502
1503
1504
1505
1506
1507
1508 <span class="grey">Wenger, et al. Standards Track [Page 27]</span></pre>
1509 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-28" ></span>
1510 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
1511
1512
1513 Table 3. Summary of the NAL unit types allowed to be
1514 aggregated or fragmented (yes = allowed, no = disallowed,
1515 - = not applicable/not specified)
1516
1517 Type Subtype STAP-A STAP-B MTAP16 MTAP24 FU-A FU-B NI-MTAP
1518 -------------------------------------------------------------
1519 0 - - - - - - - -
1520 1-23 - yes yes yes yes yes yes yes
1521 24-29 - no no no no no no no
1522 30 - yes yes yes yes no no yes
1523 31 0 - - - - - - -
1524 31 1 yes no no no no no yes
1525 31 2 no no no no no no no
1526 31 3-31 - - - - - - -
1527
1528 <span class="h3"><a class="selflink" id="section-4.4" href="#section-4.4">4.4</a>. Transmission Modes</span>
1529
1530 This memo enables transmission of an SVC bitstream over one or more
1531 RTP sessions. If only one RTP session is used for transmission of
1532 the SVC bitstream, the transmission mode is referred to as single-
1533 session transmission (SST); otherwise (more than one RTP session is
1534 used for transmission of the SVC bitstream), the transmission mode is
1535 referred to as multi-session transmission (MST).
1536
1537 SST SHOULD be used for point-to-point unicast scenarios, while MST
1538 SHOULD be used for point-to-multipoint multicast scenarios where
1539 different receivers requires different operation points of the same
1540 SVC bitstream, to improve bandwidth utilizing efficiency.
1541
1542 If the OPTIONAL mst-mode media type parameter (see <a href="#section-7.1">Section 7.1</a>) is
1543 not present, SST MUST be used; otherwise (mst-mode is present), MST
1544 MUST be used.
1545
1546 <span class="h3"><a class="selflink" id="section-4.5" href="#section-4.5">4.5</a>. Packetization Modes</span>
1547
1548 <span class="h4"><a class="selflink" id="section-4.5.1" href="#section-4.5.1">4.5.1</a>. Packetization Modes for Single-Session Transmission</span>
1549
1550 When SST is in use, <a href="./rfc6184#section-5.4">Section&nbsp;5.4 of [RFC6184]</a> applies with the
1551 following extensions.
1552
1553 The packetization modes specified in <a href="./rfc6184#section-5.4">Section&nbsp;5.4 of [RFC6184]</a>,
1554 namely, single NAL unit mode, non-interleaved mode, and interleaved
1555 mode, are also referred to as session packetization modes. Table 4
1556 summarizes the allowed session packetization modes for SST.
1557
1558
1559
1560
1561
1562
1563
1564 <span class="grey">Wenger, et al. Standards Track [Page 28]</span></pre>
1565 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-29" ></span>
1566 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
1567
1568
1569 Table 4. Summary of allowed session packetization modes
1570 (denoted as "Session Mode" for simplicity) for SST (yes =
1571 allowed, no = disallowed)
1572
1573 Session Mode Allowed
1574 -------------------------------------
1575 Single NAL Unit Mode yes
1576 Non-Interleaved Mode yes
1577 Interleaved Mode yes
1578
1579 For NAL unit types in the range of 0 to 29, inclusive, the NAL unit
1580 types allowed to be directly used as packet payloads for each session
1581 packetization mode are the same as specified in <a href="./rfc6184#section-5.4">Section&nbsp;5.4 of
1582 [RFC6184]</a>. For other NAL unit types, which are newly introduced in
1583 this memo, the NAL unit types allowed to be directly used as packet
1584 payloads for each session packetization mode are summarized in Table
1585 5.
1586
1587 Table 5. New NAL unit types allowed to be directly used
1588 as packet payloads for each session packetization mode
1589 (yes = allowed, no = disallowed, - = not applicable/not specified)
1590
1591 Type Subtype Single NAL Non-Interleaved Interleaved
1592 Unit Mode Mode Mode
1593 -------------------------------------------------------------
1594 30 - yes no no
1595 31 0 - - -
1596 31 1 yes yes no
1597 31 2 no yes no
1598 31 3-31 - - -
1599
1600 <span class="h4"><a class="selflink" id="section-4.5.2" href="#section-4.5.2">4.5.2</a>. Packetization Modes for Multi-Session Transmission</span>
1601
1602 For MST, this memo specifies four MST packetization modes:
1603
1604 o Non-interleaved timestamp based mode (NI-T);
1605
1606 o Non-interleaved cross-session decoding order number (CS-DON) based
1607 mode (NI-C);
1608
1609 o Non-interleaved combined timestamp and CS-DON mode (NI-TC); and
1610
1611 o Interleaved CS-DON (I-C) mode.
1612
1613 These four modes differ in two ways. First, they differ in terms of
1614 whether NAL units are required to be transmitted within each RTP
1615 session in decoding order (i.e., non-interleaved), or they are
1616 allowed to be transmitted in a different order (i.e., interleaved).
1617
1618
1619
1620 <span class="grey">Wenger, et al. Standards Track [Page 29]</span></pre>
1621 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-30" ></span>
1622 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
1623
1624
1625 Second, they differ in the mechanisms they provide in order to
1626 recover the correct decoding order of the NAL units across all RTP
1627 sessions involved.
1628
1629 The NI-T, NI-C, and NI-TC modes do not allow interleaving, and are
1630 thus targeted for systems that require relatively low end-to-end
1631 latency, e.g., conversational systems. The I-C mode allows
1632 interleaving and is thus targeted for systems that do not require
1633 very low end-to-end latency. The benefits of interleaving are the
1634 same as that of the interleaved mode specified in [<a href="./rfc6184" title="&quot;RTP Payload Format for H.264 Video&quot;">RFC6184</a>].
1635
1636 The NI-T mode uses timestamps to recover the decoding order of NAL
1637 units, whereas the NI-C and I-C modes both use the CS-DON mechanism
1638 (explained later) to do so. The NI-TC mode provides both timestamps
1639 and the CS-DON method; receivers in this case may choose to use
1640 either method for performing decoding order recovery. The MST
1641 packetization mode in use MUST be signaled by the value of the
1642 OPTIONAL mst-mode media type parameter. The used MST packetization
1643 mode governs which session packetization modes are allowed in the
1644 associated RTP sessions, which in turn govern which NAL unit types
1645 are allowed to be directly used as RTP packet payloads.
1646
1647 Table 6 summarizes the allowed session packetization modes for NI-T,
1648 NI-C, and NI-TC. Table 7 summarizes the allowed session
1649 packetization modes for I-C.
1650
1651 Table 6. Summary of allowed session packetization modes
1652 (denoted as "Session Mode" for simplicity) for NI-T, NI-C, and
1653 NI-TC (yes = allowed, no = disallowed)
1654
1655 Session Mode Base Session Enhancement Session
1656 -----------------------------------------------------------
1657 Single NAL Unit Mode yes no
1658 Non-Interleaved Mode yes yes
1659 Interleaved Mode no no
1660
1661 Table 7. Summary of allowed session packetization modes
1662 (denoted as "Session Mode" for simplicity) for I-C
1663 (yes = allowed, no = disallowed)
1664
1665 Session Mode Base Session Enhancement Session
1666 -----------------------------------------------------------
1667 Single NAL Unit Mode no no
1668 Non-Interleaved Mode no no
1669 Interleaved Mode yes yes
1670
1671
1672
1673
1674
1675
1676 <span class="grey">Wenger, et al. Standards Track [Page 30]</span></pre>
1677 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-31" ></span>
1678 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
1679
1680
1681 For NAL unit types in the range of 0 to 29, inclusive, the NAL unit
1682 types allowed to be directly used as packet payloads for each session
1683 packetization mode are the same as specified in <a href="./rfc6184#section-5.4">Section&nbsp;5.4 of
1684 [RFC6184]</a>. For other NAL unit types, which are newly introduced in
1685 this memo, the NAL unit types allowed to be directly used as packet
1686 payloads for each allowed session packetization mode for NI-T, NI-C,
1687 NI-TC, and I-C are summarized in Tables 8, 9, 10, and 11,
1688 respectively.
1689
1690 Table 8. New NAL unit types allowed to be directly used
1691 as packet payloads for each allowed session packetization
1692 mode when NI-T is in use (yes = allowed, no = disallowed,
1693 - = not applicable/not specified)
1694
1695 Type Subtype Single NAL Non-Interleaved
1696 Unit Mode Mode
1697 ---------------------------------------------------
1698 30 - yes no
1699 31 0 - -
1700 31 1 yes yes
1701 31 2 no yes
1702 31 3-31 - -
1703
1704 Table 9. New NAL unit types allowed to be directly used
1705 as packet payloads for each allowed session packetization
1706 mode when NI-C is in use (yes = allowed, no = disallowed,
1707 - = not applicable/not specified)
1708
1709 Type Subtype Single NAL Non-Interleaved
1710 Unit Mode Mode
1711 ---------------------------------------------------
1712 30 - yes yes
1713 31 0 - -
1714 31 1 no no
1715 31 2 no yes
1716 31 3-31 - -
1717
1718
1719
1720
1721
1722
1723
1724
1725
1726
1727
1728
1729
1730
1731
1732 <span class="grey">Wenger, et al. Standards Track [Page 31]</span></pre>
1733 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-32" ></span>
1734 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
1735
1736
1737 Table 10. New NAL unit types allowed to be directly used
1738 as packet payloads for each allowed session packetization
1739 mode when NI-TC is in use (yes = allowed, no = disallowed,
1740 - = not applicable/not specified)
1741
1742 Type Subtype Single NAL Non-Interleaved
1743 Unit Mode Mode
1744 ---------------------------------------------------
1745 30 - yes yes
1746 31 0 - -
1747 31 1 yes yes
1748 31 2 no yes
1749 31 3-31 - -
1750
1751 Table 11. New NAL unit types allowed to be directly used
1752 as packet payloads for the allowed session packetization
1753 mode when I-C is in use (yes = allowed, no = disallowed,
1754 - = not applicable/not specified)
1755
1756 Type Subtype Interleaved Mode
1757 ------------------------------------
1758 30 - no
1759 31 0 -
1760 31 1 no
1761 31 2 no
1762 31 3-31 -
1763
1764 When MST is in use and the MST packetization mode in use is NI-C,
1765 empty NAL units (type 31, subtype 1) MUST NOT be used, i.e., no RTP
1766 packet is allowed to contain one or more empty NAL units.
1767
1768 When MST is in use and the MST packetization mode in use is I-C, both
1769 empty NAL units (type 31, subtype 1) and NI-MTAP NAL units (type 31,
1770 subtype 2) MUST NOT be used, i.e., no RTP packet is allowed to
1771 contain one or more empty NAL units or an NI-MTAP NAL unit.
1772
1773 <span class="h3"><a class="selflink" id="section-4.6" href="#section-4.6">4.6</a>. Single NAL Unit Packets</span>
1774
1775 <a href="./rfc6184#section-5.6">Section&nbsp;5.6 of [RFC6184]</a> applies with the following extensions.
1776
1777 The payload of a single NAL unit packet MAY be a PACSI NAL unit (Type
1778 30) or an empty NAL unit (Type 31 and Subtype 1), in addition to a
1779 NAL unit with NAL unit type equal to any value from 1 to 23,
1780 inclusive.
1781
1782
1783
1784
1785
1786
1787
1788 <span class="grey">Wenger, et al. Standards Track [Page 32]</span></pre>
1789 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-33" ></span>
1790 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
1791
1792
1793 If the Type field of the first byte of the payload is not equal to
1794 31, the payload header is the first byte of the payload. Otherwise,
1795 (the Type field of the first byte of the payload is equal to 31), the
1796 payload header is the first two bytes of the payload.
1797
1798 <span class="h3"><a class="selflink" id="section-4.7" href="#section-4.7">4.7</a>. Aggregation Packets</span>
1799
1800 In addition to <a href="./rfc6184#section-5.7">Section&nbsp;5.7 of [RFC6184]</a>, the following applies in
1801 this memo.
1802
1803 <span class="h4"><a class="selflink" id="section-4.7.1" href="#section-4.7.1">4.7.1</a>. Non-Interleaved Multi-Time Aggregation Packets (NI-MTAPs)</span>
1804
1805 One new NAL unit type introduced in this memo is the non-interleaved
1806 multi-time aggregation packet (NI-MTAP). An NI-MTAP consists of one
1807 or more non-interleaved multi-time aggregation units.
1808
1809 The NAL units contained in NI-MTAPs MUST be aggregated in decoding
1810 order.
1811
1812 A non-interleaved multi-time aggregation unit for the NI-MTAP
1813 consists of 16 bits of unsigned size information of the following NAL
1814 unit (in network byte order), and 16 bits (in network byte order) of
1815 timestamp offset (TS offset) for the NAL unit. The structure is
1816 presented in Figure 1. The starting or ending position of an
1817 aggregation unit within a packet may or may not be on a 32-bit word
1818 boundary. The NAL units in the NI-MTAP are ordered in NAL unit
1819 decoding order.
1820
1821 The Type field of the NI-MTAP MUST be set equal to "31".
1822
1823 The F bit MUST be set to 0 if all the F bits of the aggregated NAL
1824 units are zero; otherwise, it MUST be set to 1.
1825
1826 The value of NRI MUST be the maximum value of NRI across all NAL
1827 units carried in the NI-MTAP packet.
1828
1829 The field Subtype MUST be equal to 2.
1830
1831 If the field J is equal to 1, the optional DON field MUST be present
1832 for each of the non-interleaved multi-time aggregation units. For
1833 SST, the J field MUST be equal to 0. For MST, in the NI-T mode the J
1834 field MUST be equal to 0, whereas in the NI-C or NI-TC mode the J
1835 field MUST be equal to 1. When the NI-C or NI-TC mode is in use, the
1836 DON field, when present, MUST represent the CS-DON value for the
1837 particular NAL unit as defined in <a href="#section-6.2.2">Section 6.2.2</a>.
1838
1839 The fields K and L MUST be both equal to 0.
1840
1841
1842
1843
1844 <span class="grey">Wenger, et al. Standards Track [Page 33]</span></pre>
1845 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-34" ></span>
1846 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
1847
1848
1849 0 1 2 3
1850 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
1851 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
1852 : NAL unit size | TS offset |
1853 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
1854 | DON (optional) | |
1855 |-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ NAL unit |
1856 | |
1857 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
1858 | :
1859 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
1860
1861 Figure 1. Non-interleaved multi-time aggregation unit for NI-MTAP
1862
1863 Let TS be the RTP timestamp of the packet carrying the NAL unit.
1864 Recall that the NALU-time of a NAL unit in an MTAP is defined in
1865 [<a href="./rfc6184" title="&quot;RTP Payload Format for H.264 Video&quot;">RFC6184</a>] as the value that the RTP timestamp would have if that NAL
1866 unit would be transported in its own RTP packet. The timestamp
1867 offset field MUST be set to a value equal to the value of the
1868 following formula:
1869
1870 if NALU-time &gt;= TS, TS offset = NALU-time - TS
1871 else, TS offset = NALU-time + (2^32 - TS)
1872
1873 For the "earliest" multi-time aggregation unit in an NI-MTAP, the
1874 timestamp offset MUST be zero. Hence, the RTP timestamp of the NI-
1875 MTAP itself is identical to the earliest NALU-time.
1876
1877 Informative note: The "earliest" multi-time aggregation unit is
1878 the one that would have the smallest extended RTP timestamp among
1879 all the aggregation units of an NI-MTAP if the aggregation units
1880 were encapsulated in single NAL unit packets. An extended
1881 timestamp is a timestamp that has more than 32 bits and is capable
1882 of counting the wraparound of the timestamp field, thus enabling
1883 one to determine the smallest value if the timestamp wraps. Such
1884 an "earliest" aggregation unit may or may not be the first one in
1885 the order in which the aggregation units are encapsulated in an
1886 NI-MTAP. The "earliest" NAL unit need not be the same as the
1887 first NAL unit in the NAL unit decoding order either.
1888
1889 Figure 2 presents an example of an RTP packet that contains an NI-
1890 MTAP that contains two non-interleaved multi-time aggregation units,
1891 labeled as 1 and 2 in the figure.
1892
1893
1894
1895
1896
1897
1898
1899
1900 <span class="grey">Wenger, et al. Standards Track [Page 34]</span></pre>
1901 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-35" ></span>
1902 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
1903
1904
1905 0 1 2 3
1906 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
1907 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
1908 | RTP Header |
1909 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
1910 |F|NRI| Type | Subtype |J|K|L| |
1911 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |
1912 | |
1913 | Non-interleaved multi-time aggregation unit #1 |
1914 : :
1915 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
1916 | | Non-interleaved multi-time |
1917 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |
1918 | aggregation unit #2 |
1919 : :
1920 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
1921 | :...OPTIONAL RTP padding |
1922 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
1923
1924 Figure 2. An RTP packet including an NI-MTAP containing two
1925 non-interleaved multi-time aggregation units
1926
1927 <span class="h3"><a class="selflink" id="section-4.8" href="#section-4.8">4.8</a>. Fragmentation Units (FUs)</span>
1928
1929 <a href="./rfc6184#section-5.8">Section&nbsp;5.8 of [RFC6184]</a> applies.
1930
1931 Informative note: In case a NAL unit with the four-byte SVC NAL
1932 unit header is fragmented, the three-byte SVC-specific header
1933 extension is considered as part of the NAL unit payload. That is,
1934 the three-byte SVC-specific header extension is only available in
1935 the first fragment of the fragmented NAL unit.
1936
1937 <span class="h3"><a class="selflink" id="section-4.9" href="#section-4.9">4.9</a>. Payload Content Scalability Information (PACSI) NAL Unit</span>
1938
1939 Another new type of NAL unit specified in this memo is the payload
1940 content scalability information (PACSI) NAL unit. The Type field of
1941 PACSI NAL units MUST be equal to 30 (a NAL unit type value left
1942 unspecified in [<a href="#ref-H.264" title="&quot;Advanced video coding for generic audiovisual services&quot;">H.264</a>] and [<a href="./rfc6184" title="&quot;RTP Payload Format for H.264 Video&quot;">RFC6184</a>]). A PACSI NAL unit MAY be
1943 carried in a single NAL unit packet or an aggregation packet, and
1944 MUST NOT be fragmented.
1945
1946 PACSI NAL units may be used for the following purposes:
1947
1948 o To enable MANEs to decide whether to forward, process, or discard
1949 aggregation packets, by checking in PACSI NAL units the
1950 scalability information and other characteristics of the
1951
1952
1953
1954
1955
1956 <span class="grey">Wenger, et al. Standards Track [Page 35]</span></pre>
1957 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-36" ></span>
1958 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
1959
1960
1961 aggregated NAL units, rather than looking into the aggregated NAL
1962 units themselves, which are defined by the video coding
1963 specification;
1964
1965 o To enable correct decoding order recovery in MST using the NI-C or
1966 NI-TC mode, with the help of the CS-DON information included in
1967 PACSI NAL units; and
1968
1969 o To improve resilience to packet losses, e.g., by utilizing the
1970 following data or information included in PACSI NAL units:
1971 repeated Supplemental Enhancement Information (SEI) messages,
1972 information regarding the start and end of layer representations,
1973 and the indices to layer representations of the lowest temporal
1974 subset.
1975
1976 PACSI NAL units MAY be ignored in the NI-T mode without affecting the
1977 decoding order recovery process.
1978
1979 When a PACSI NAL unit is present in an aggregation packet, the
1980 following applies.
1981
1982 o The PACSI NAL unit MUST be the first aggregated NAL unit in the
1983 aggregation packet.
1984
1985 o There MUST be at least one additional aggregated NAL unit in the
1986 aggregation packet.
1987
1988 o The RTP header fields and the payload header fields of the
1989 aggregation packet are set as if the PACSI NAL unit was not
1990 included in the aggregation packet.
1991
1992 o If the aggregation packet is an MTAP16, MTAP24, or NI-MTAP with
1993 the J field equal to 1, the decoding order number (DON) for the
1994 PACSI NAL unit MUST be set to indicate that the PACSI NAL unit has
1995 an identical DON to the first NAL unit in decoding order among the
1996 remaining NAL units in the aggregation packet.
1997
1998 When a PACSI NAL unit is included in a single NAL unit packet, it is
1999 associated with the next non-PACSI NAL unit in transmission order,
2000 and the RTP header fields of the packet are set as if the next non-
2001 PACSI NAL unit in transmission order was included in a single NAL
2002 unit packet.
2003
2004 The PACSI NAL unit structure is as follows. The first four octets
2005 are exactly the same as the four-byte SVC NAL unit header discussed
2006 in <a href="#section-1.1.3">Section 1.1.3</a>. They are followed by one octet containing several
2007 flags, then five optional octets, and finally zero or more SEI NAL
2008 units. Each SEI NAL unit is preceded by a 16-bit unsigned size field
2009
2010
2011
2012 <span class="grey">Wenger, et al. Standards Track [Page 36]</span></pre>
2013 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-37" ></span>
2014 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
2015
2016
2017 (in network byte order) that indicates the size of the following NAL
2018 unit in bytes (excluding these two octets, but including the NAL unit
2019 header octet of the SEI NAL unit). Figure 3 illustrates the PACSI
2020 NAL unit structure and an example of a PACSI NAL unit containing two
2021 SEI NAL units.
2022
2023 0 1 2 3
2024 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
2025 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
2026 |F|NRI| Type |R|I| PRID |N| DID | QID | TID |U|D|O| RR|
2027 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
2028 |X|Y|T|A|P|C|S|E| TL0PICIDX (o) | IDRPICID (o) |
2029 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
2030 | DONC (o) | NAL unit size 1 |
2031 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
2032 | |
2033 | SEI NAL unit 1 |
2034 | |
2035 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
2036 | | NAL unit size 2 | |
2037 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |
2038 | |
2039 | SEI NAL unit 2 |
2040 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
2041 | |
2042 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
2043
2044 Figure 3. PACSI NAL unit structure. Fields suffixed by
2045 "(o)" are OPTIONAL.
2046
2047 The bits A, P, and C are specified only if the bit X is equal to 1.
2048 The bits S and E are specified, and the fields TL0PICIDX and IDRPICID
2049 are present, only if the bit Y is equal to 1. The field DONC is
2050 present only if the bit T is equal to 1. The field T MUST be equal
2051 to 0 if the PACSI NAL unit is contained in an STAP-B, MTAP16, MTAP24,
2052 or NI-MTAP with the J field equal to 1.
2053
2054 The values of the fields in PACSI NAL unit MUST be set as follows.
2055
2056 o The F bit MUST be set to 1 if the F bit in at least one of the
2057 remaining NAL units in the aggregation packet is equal to 1 (when
2058 the PACSI NAL unit is included in an aggregation packet) or if the
2059 next non-PACSI NAL unit in transmission order has the F bit equal
2060 to 1 (when the PACSI NAL unit is included in a single NAL unit
2061 packet). Otherwise, the F bit MUST be set to 0.
2062
2063
2064
2065
2066
2067
2068 <span class="grey">Wenger, et al. Standards Track [Page 37]</span></pre>
2069 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-38" ></span>
2070 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
2071
2072
2073 o The NRI field MUST be set to the highest value of NRI field among
2074 all the remaining NAL units in the aggregation packet (when the
2075 PACSI NAL unit is included in an aggregation packet) or the value
2076 of the NRI field of the next non-PACSI NAL unit in transmission
2077 order (when the PACSI NAL unit is included in a single NAL unit
2078 packet).
2079
2080 o The Type field MUST be set to 30.
2081
2082 o The R bit MUST be set to 1. Receivers MUST ignore the value of R.
2083
2084 o The I bit MUST be set to 1 if the I bit of at least one of the
2085 remaining NAL units in the aggregation packet is equal to 1 (when
2086 the PACSI NAL unit is included in an aggregation packet) or if the
2087 I bit of the next non-PACSI NAL unit in transmission order is
2088 equal to 1 (when the PACSI NAL unit is included in a single NAL
2089 unit packet). Otherwise, the I bit MUST be set to 0.
2090
2091 o The PRID field MUST be set to the lowest value of the PRID values
2092 of the remaining NAL units in the aggregation packet (when the
2093 PACSI NAL unit is included in an aggregation packet) or the PRID
2094 value of the next non-PACSI NAL unit in transmission order (when
2095 the PACSI NAL unit is included in a single NAL unit packet).
2096
2097 o The N bit MUST be set to 1 if the N bit of all the remaining NAL
2098 units in the aggregation packet is equal to 1 (when the PACSI NAL
2099 unit is included in an aggregation packet) or if the N bit of the
2100 next non-PACSI NAL unit in transmission order is equal to 1 (when
2101 the PACSI NAL unit is included in a single NAL unit packet).
2102 Otherwise, the N bit MUST be set to 0.
2103
2104 o The DID field MUST be set to the lowest value of the DID values of
2105 the remaining NAL units in the aggregation packet (when the PACSI
2106 NAL unit is included in an aggregation packet) or the DID value of
2107 the next non-PACSI NAL unit in transmission order (when the PACSI
2108 NAL unit is included in a single NAL unit packet).
2109
2110 o The QID field MUST be set to the lowest value of the QID values of
2111 the remaining NAL units with the lowest value of DID in the
2112 aggregation packet (when the PACSI NAL unit is included in an
2113 aggregation packet) or the QID value of the next non-PACSI NAL
2114 unit in transmission order (when the PACSI NAL unit is included in
2115 a single NAL unit packet).
2116
2117 o The TID field MUST be set to the lowest value of the TID values of
2118 the remaining NAL units with the lowest value of DID in the
2119 aggregation packet (when the PACSI NAL unit is included in an
2120
2121
2122
2123
2124 <span class="grey">Wenger, et al. Standards Track [Page 38]</span></pre>
2125 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-39" ></span>
2126 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
2127
2128
2129 aggregation packet) or the TID value of the next non-PACSI NAL
2130 unit in transmission order (when the PACSI NAL unit is included in
2131 a single NAL unit packet).
2132
2133 o The U bit MUST be set to 1 if the U bit of at least one of the
2134 remaining NAL units in the aggregation packet is equal to 1 (when
2135 the PACSI NAL unit is included in an aggregation packet) or if the
2136 U bit of the next non-PACSI NAL unit in transmission order is
2137 equal to 1 (when the PACSI NAL unit is included in a single NAL
2138 unit packet). Otherwise, the U bit MUST be set to 0.
2139
2140 o The D bit MUST be set to 1 if the D value of all the remaining NAL
2141 units in the aggregation packet is equal to 1 (when the PACSI NAL
2142 unit is included in an aggregation packet) or if the D bit of the
2143 next non-PACSI NAL unit in transmission order is equal to 1 (when
2144 the PACSI NAL unit is included in a single NAL unit packet).
2145 Otherwise, the D bit MUST be set to 0.
2146
2147 o The O bit MUST be set to 1 if the O bit of at least one of the
2148 remaining NAL units in the aggregation packet is equal to 1 (when
2149 the PACSI NAL unit is included in an aggregation packet) or if the
2150 O bit of the next non-PACSI NAL unit in transmission order is
2151 equal to 1 (when the PACSI NAL unit is included in a single NAL
2152 unit packet). Otherwise, the O bit MUST be set to 0.
2153
2154 o The RR field MUST be set to "11" (in binary form). Receivers MUST
2155 ignore the value of RR.
2156
2157 o If the X bit is equal to 1, the bits A, P, and C are specified as
2158 below. Otherwise, the bits A, P, and C are unspecified, and
2159 receivers MUST ignore the values of these bits. The X bit SHOULD
2160 be identical for all the PACSI NAL units in all the RTP sessions
2161 carrying the same SVC bitstream.
2162
2163 o If the Y bit is equal to 1, the OPTIONAL fields TL0PICIDX and
2164 IDRPICID MUST be present and specified as below, and the bits S
2165 and E are also specified as below. Otherwise, the fields
2166 TL0PICIDX and IDRPICID MUST NOT be present, while the S and E bits
2167 are unspecified and receivers MUST ignore the values of these
2168 bits. The Y bit MUST be identical for all the PACSI NAL units in
2169 all the RTP sessions carrying the same SVC bitstream. The Y bit
2170 MUST be equal to 0 when the parameter packetization-mode is equal
2171 to 2.
2172
2173 o If the T bit is equal to 1, the OPTIONAL field DONC MUST be
2174 present and specified as below. Otherwise, the field DONC MUST
2175 NOT be present. The field T MUST be equal to 0 if the PACSI NAL
2176 unit is contained in an STAP-B, MTAP16, MTAP24, or NI-MTAP.
2177
2178
2179
2180 <span class="grey">Wenger, et al. Standards Track [Page 39]</span></pre>
2181 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-40" ></span>
2182 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
2183
2184
2185 o The A bit MUST be set to 1 if at least one of the remaining NAL
2186 units in the aggregation packet belongs to an anchor layer
2187 representation (when the PACSI NAL unit is included in an
2188 aggregation packet) or if the next non-PACSI NAL unit in
2189 transmission order belongs to an anchor layer representation (when
2190 the PACSI NAL unit is included in a single NAL unit packet).
2191 Otherwise, the A bit MUST be set to 0.
2192
2193 Informative note: The A bit indicates whether CGS or spatial layer
2194 switching at a non-IDR layer representation (a layer
2195 representation with nal_unit_type not equal to 5 and idr_flag not
2196 equal to 1) can be performed. With some picture coding structures
2197 a non-IDR intra layer representation can be used for random
2198 access. Compared to using only IDR layer representations, higher
2199 coding efficiency can be achieved. The H.264/AVC or SVC solution
2200 to indicate the random accessibility of a non-IDR intra layer
2201 representation is using a recovery point SEI message. The A bit
2202 offers direct access to this information, without having to parse
2203 the recovery point SEI message, which may be buried deeply in an
2204 SEI NAL unit. Furthermore, the SEI message may or may not be
2205 present in the bitstream.
2206
2207 o The P bit MUST be set to 1 if all the remaining NAL units in the
2208 aggregation packet have redundant_pic_cnt greater than 0 (when the
2209 PACSI NAL unit is included in an aggregation packet) or the next
2210 non-PACSI NAL unit in transmission order has redundant_pic_cnt
2211 greater than 0 (when the PACSI NAL unit is included in a single
2212 NAL unit packet). Otherwise, the P bit MUST be set to 0.
2213
2214 Informative note: The P bit indicates whether a packet can be
2215 discarded because it contains only redundant slice NAL units.
2216 Without this bit, the corresponding information can be obtained
2217 from the syntax element redundant_pic_cnt, which is contained in
2218 the variable-length coded slice header.
2219
2220 o The C bit MUST be set to 1 if at least one of the remaining NAL
2221 units in the aggregation packet belongs to an intra layer
2222 representation (when the PACSI NAL unit is included in an
2223 aggregation packet) or if the next non-PACSI NAL unit in
2224 transmission order belongs to an intra layer representation (when
2225 the PACSI NAL unit is included in a single NAL unit packet).
2226 Otherwise, the C bit MUST be set to 0.
2227
2228 Informative note: The C bit indicates whether a packet contains
2229 intra slices, which may be the only packets to be forwarded, e.g.,
2230 when the network conditions are particularly adverse.
2231
2232
2233
2234
2235
2236 <span class="grey">Wenger, et al. Standards Track [Page 40]</span></pre>
2237 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-41" ></span>
2238 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
2239
2240
2241 o The S bit MUST be set to 1, if the first NAL unit following the
2242 PACSI NAL unit in an aggregation packet is the first VCL NAL unit,
2243 in decoding order, of a layer representation (when the PACSI NAL
2244 unit is included in an aggregation packet) or if the next non-
2245 PACSI NAL unit in transmission order is the first VCL NAL unit, in
2246 decoding order, of a layer representation(when the PACSI NAL unit
2247 is included in a single NAL unit packet). Otherwise, the S bit
2248 MUST be set to 0.
2249
2250 o The E bit MUST be set to 1, if the last NAL unit following the
2251 PACSI NAL unit in an aggregation packet is the last VCL NAL unit,
2252 in decoding order, of a layer representation (when the PACSI NAL
2253 unit is included in an aggregation packet) or if the next non-
2254 PACSI NAL unit in transmission order is the last VCL NAL unit, in
2255 decoding order, of a layer representation (when the PACSI NAL unit
2256 is included in a single NAL unit packet). Otherwise, the E bit
2257 MUST be set to 0.
2258
2259 Informative note: In an aggregation packet it is always possible
2260 to detect the beginning or end of a layer representation by
2261 detecting changes in the values of dependency_id, quality_id, and
2262 temporal_id in NAL unit headers, except from the first and last
2263 NAL units of a packet. The S or E bits are used to provide this
2264 information, for both single NAL unit and aggregation packets, so
2265 that previous or following packets do not have to be examined.
2266 This enables MANEs to detect slice loss and take proper action
2267 such as requesting a retransmission as soon as possible, as well
2268 as to allow efficient playout buffer handling similarly to the M
2269 bit present in the RTP header. The M bit in the RTP header still
2270 indicates the end of an access unit, not the end of a layer
2271 representation.
2272
2273 o When present, the TL0PICIDX field MUST be set to equal to
2274 tl0_dep_rep_idx as specified in Annex G of [<a href="#ref-H.264" title="&quot;Advanced video coding for generic audiovisual services&quot;">H.264</a>] for the layer
2275 representation containing the first NAL unit following the PACSI
2276 NAL unit in the aggregation packet (when the PACSI NAL unit is
2277 included in an aggregation packet) or containing the next non-
2278 PACSI NAL unit in transmission order (when the PACSI NAL unit is
2279 included in a single NAL unit packet).
2280
2281 o When present, the IDRPICID field MUST be set to equal to
2282 effective_idr_pic_id as specified in Annex G of [<a href="#ref-H.264" title="&quot;Advanced video coding for generic audiovisual services&quot;">H.264</a>] for the
2283 layer representation containing the first NAL unit following the
2284 PACSI NAL unit in the aggregation packet (when the PACSI NAL unit
2285 is included in an aggregation packet) or containing the next non-
2286 PACSI NAL unit in transmission order (when the PACSI NAL unit is
2287 included in a single NAL unit packet).
2288
2289
2290
2291
2292 <span class="grey">Wenger, et al. Standards Track [Page 41]</span></pre>
2293 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-42" ></span>
2294 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
2295
2296
2297 Informative note: The TL0PICIDX and IDRPICID fields enable the
2298 detection of the loss of layer representations in the most
2299 important temporal layer (with temporal_id equal to 0) by
2300 receivers as well as MANEs. SVC provides a solution that uses SEI
2301 messages, which are harder to parse and may or may not be present
2302 in the bitstream. When the PACSI NAL unit is part of an NI-MTAP
2303 packet, it is possible to infer the correct values of
2304 tl0_dep_rep_idx and idr_pic_id for all layer representations
2305 contained in the NI-MTAP by following the rules that specify how
2306 these parameters are set as given in Annex G of [<a href="#ref-H.264" title="&quot;Advanced video coding for generic audiovisual services&quot;">H.264</a>] and by
2307 detecting the different layer representations contained in the NI-
2308 MTAP packet by detecting changes in the values of dependency_id_,
2309 quality_id, and temporal_id in the NAL unit headers as well as
2310 using the S and E flags. The only exception is if NAL units of an
2311 IDR picture are present in the NI-MTAP in a position other than
2312 the first NAL unit following the PACSI NAL unit, in which case the
2313 value of idr_pic_id cannot be inferred. In this case the NAL unit
2314 has to be partially parsed to obtain the idr_pic_id. Note that,
2315 due to the large size of IDR pictures, their inclusion in an NI-
2316 MTAP, and especially in a position other than the first NAL unit
2317 following the PACSI NAL unit, may be neither practical nor useful.
2318
2319 o When present, the field DONC indicates the cross-session decoding
2320 order number (CS-DON) for the first of the remaining NAL units in
2321 the aggregation packet (when the PACSI NAL unit is included in an
2322 aggregation packet) or the CS-DON of the next non-PACSI NAL unit
2323 in transmission order (when the PACSI NAL unit is included in a
2324 single NAL unit packet). CS-DON is further discussed in <a href="#section-4.11">Section</a>
2325 <a href="#section-4.11">4.11</a>.
2326
2327 The PACSI NAL unit MAY include a subset of the SEI NAL units
2328 associated with the access unit to which the first non-PACSI NAL unit
2329 in the aggregation packet belongs, and MUST NOT contain SEI NAL units
2330 associated with any other access unit.
2331
2332 Informative note: In H.264/AVC and SVC, within each access unit,
2333 SEI NAL units must appear before any VCL NAL unit in decoding
2334 order. Therefore, without using PACSI NAL units, SEI messages are
2335 typically only conveyed in the first of the packets carrying an
2336 access unit. Senders may repeat SEI NAL units in PACSI NAL units,
2337 so that they are repeated in more than one packet and thus
2338 increase robustness against packet losses. Receivers may use the
2339 repeated SEI messages in place of missing SEI messages.
2340
2341 For a PACSI NAL unit included in an aggregation packet, an SEI
2342 message SHOULD NOT be included in the PACSI NAL unit and also
2343 included in one of the remaining NAL units contained in the same
2344 aggregation packet.
2345
2346
2347
2348 <span class="grey">Wenger, et al. Standards Track [Page 42]</span></pre>
2349 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-43" ></span>
2350 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
2351
2352
2353 <span class="h3"><a class="selflink" id="section-4.10" href="#section-4.10">4.10</a>. Empty NAL unit</span>
2354
2355 An empty NAL unit MAY be included in a single NAL unit packet, an
2356 STAP-A or an NI-MTAP packet. Empty NAL units MUST have an RTP
2357 timestamp (when transported in a single NAL unit packet) or NALU-
2358 time (when transported in an aggregation packet) that is associated
2359 with an access unit for which there exists at least one NAL unit of
2360 type 1, 5, or 20. When MST is used, the type 1, 5, or 20 NAL unit
2361 may be in a different RTP session. Empty NAL units may be used in
2362 the decoding order recovery process of the NI-T mode as described in
2363 <a href="#section-5.2.1">Section 5.2.1</a>.
2364
2365 The packet structure is shown in the following figure.
2366
2367 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
2368 |F|NRI| Type | Subtype |J|K|L|
2369 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
2370
2371 Figure 4. Empty NAL unit structure.
2372
2373 The fields MUST be set as follows:
2374
2375 F MUST be equal to 0
2376 NRI MUST be equal to 3
2377 Type MUST be equal to 31
2378 Subtype MUST be equal to 1
2379 J MUST be equal to 0
2380 K MUST be equal to 0
2381 L MUST be equal to 0
2382
2383 <span class="h3"><a class="selflink" id="section-4.11" href="#section-4.11">4.11</a>. Decoding Order Number (DON)</span>
2384
2385 The DON concept is introduced in [<a href="./rfc6184" title="&quot;RTP Payload Format for H.264 Video&quot;">RFC6184</a>] and is used to recover the
2386 decoding order when interleaving is used within a single session.
2387 <a href="./rfc6184#section-5.5">Section&nbsp;5.5 of [RFC6184]</a> applies when using SST.
2388
2389 When using MST, it is necessary to recover the decoding order across
2390 the various RTP sessions regardless if interleaving is used or not.
2391 In addition to the timestamp mechanism described later, the CS-DON
2392 mechanism is an extension of the DON facility that can be used for
2393 this purpose, and is defined in the following section.
2394
2395 <span class="h4"><a class="selflink" id="section-4.11.1" href="#section-4.11.1">4.11.1</a>. Cross-Session DON (CS-DON) for Multi-Session Transmission</span>
2396
2397 The cross-session decoding order number (CS-DON) is a number that
2398 indicates the decoding order of NAL units across all RTP sessions
2399 involved in MST. It is similar to the DON concept in [<a href="./rfc6184" title="&quot;RTP Payload Format for H.264 Video&quot;">RFC6184</a>], but
2400 contrary to [<a href="./rfc6184" title="&quot;RTP Payload Format for H.264 Video&quot;">RFC6184</a>] where the DON was used only for interleaved
2401
2402
2403
2404 <span class="grey">Wenger, et al. Standards Track [Page 43]</span></pre>
2405 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-44" ></span>
2406 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
2407
2408
2409 packetization, in this memo it is used not only in the interleaved
2410 MST mode (I-C) but also in two of the non-interleaved MST modes (NI-C
2411 and NI-TC).
2412
2413 When the NI-C or NI-TC MST modes are in use, the packetization of
2414 each session MUST be as specified in <a href="#section-5.2.2">Section 5.2.2</a>. In PACSI NAL
2415 units the CS-DON value is explicitly coded in the field DONC. For
2416 non-PACSI NAL units the CS-DON value is derived as follows. Let SN
2417 indicate the RTP sequence number of a packet.
2418
2419 o For each non-PACSI NAL unit carried in a session using the single
2420 NAL unit session packetization mode, the CS-DON value of the NAL
2421 unit is equal to (DONC_prev_PACSI + SN_diff - 1) % 65536, wherein
2422 "%" is the modulo operation, DONC_prev_PACSI is the DONC value of
2423 the previous PACSI NAL unit with the same NALU-time as the current
2424 NAL unit, and SN_diff is calculated as follows:
2425
2426 if SN1 &gt; SN2, SN_diff = SN1 - SN2
2427 else SN_diff = SN2 + 65536 - SN1
2428
2429 where SN1 and SN2 are the SNs of the current NAL unit and the
2430 previous PACSI NAL unit with the same NALU-time, respectively.
2431
2432 o For non-PACSI NAL units carried in a session using the non-
2433 interleaved session packetization mode, the CS-DON value of each
2434 non-PACSI NAL unit is derived as follows.
2435
2436 For a non-PACSI NAL unit in a single NAL unit packet, the
2437 following applies.
2438
2439 If the previous PACSI NAL unit is contained in a single NAL
2440 unit packet, the CS-DON value of the NAL unit is calculated
2441 as above;
2442
2443 otherwise (the previous PACSI NAL unit is contained in an
2444 STAP-A packet), the CS-DON value of the NAL unit is
2445 calculated as above, with DONC_prev_PACSI being replaced by
2446 the CS-DON value of the previous non-PACSI NAL unit in
2447 decoding order (i.e., the CS-DON value of the last NAL unit
2448 of the STAP-A packet).
2449
2450 For a non-PACSI NAL unit in an STAP-A packet, the following
2451 applies.
2452
2453 If the non-PACSI NAL unit is the first non-PACSI NAL unit in
2454 the STAP-A packet, the CS-DON value of the NAL unit is equal
2455 to DONC of the PACSI NAL unit in the STAP-A packet;
2456
2457
2458
2459
2460 <span class="grey">Wenger, et al. Standards Track [Page 44]</span></pre>
2461 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-45" ></span>
2462 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
2463
2464
2465 otherwise (the non-PACSI NAL unit is not the first non-
2466 PACSI NAL unit in the STAP-A packet), the CS-DON value of
2467 the NAL unit is equal to: (the CS-DON value of the previous
2468 non-PACSI NAL unit in decoding order + 1) % 65536, wherein
2469 "%" is the modulo operation.
2470
2471 For a non-PACSI NAL unit in a number of FU-A packets, the CS-
2472 DON value of the NAL unit is calculated the same way as when
2473 the single NAL unit session packetization mode is in use, with
2474 SN1 being the SN value of the first FU-A packet.
2475
2476 For a non-PACSI NAL unit in an NI-MTAP packet, the CS-DON value
2477 is equal to the value of the DON field of the non-interleaved
2478 multi-time aggregation unit.
2479
2480 When the I-C MST packetization mode is in use, the DON values derived
2481 according to [<a href="./rfc6184" title="&quot;RTP Payload Format for H.264 Video&quot;">RFC6184</a>] for all the NAL units in each of the RTP
2482 sessions MUST indicate CS-DON values.
2483
2484 <span class="h2"><a class="selflink" id="section-5" href="#section-5">5</a>. Packetization Rules</span>
2485
2486 <a href="./rfc6184#section-6">Section&nbsp;6 of [RFC6184]</a> applies in this memo, with the following
2487 additions.
2488
2489 <span class="h3"><a class="selflink" id="section-5.1" href="#section-5.1">5.1</a>. Packetization Rules for Single-Session Transmission</span>
2490
2491 All receivers MUST support the single NAL unit packetization mode to
2492 provide backward compatibility to endpoints supporting only the
2493 single NAL unit mode of [<a href="./rfc6184" title="&quot;RTP Payload Format for H.264 Video&quot;">RFC6184</a>]. However, the use of single NAL
2494 unit packetization mode (packetization-mode equal to 0) SHOULD be
2495 avoided whenever possible, because encapsulating NAL units of small
2496 sizes in their own packets (e.g., small NAL units containing
2497 parameter sets, prefix NAL units, or SEI messages) is less efficient
2498 due to the packet header overhead.
2499
2500 All receivers MUST support the non-interleaved mode.
2501
2502 Informative note: The non-interleaved mode of [<a href="./rfc6184" title="&quot;RTP Payload Format for H.264 Video&quot;">RFC6184</a>] does allow
2503 an application to encapsulate a single NAL unit in a single RTP
2504 packet. Historically, the single NAL unit mode has been included
2505 in [<a href="./rfc6184" title="&quot;RTP Payload Format for H.264 Video&quot;">RFC6184</a>] only for compatibility with ITU-T Rec. H.241 Annex A
2506 [<a href="#ref-H.241" title="&quot;Extended video procedures and control signals for H.300-series terminals&quot;">H.241</a>]. There is no point in carrying this historic ballast
2507 towards a new application space such as the one provided with SVC.
2508 The implementation complexity increase for supporting the
2509 additional mechanisms of the non-interleaved mode (namely, STAP-A
2510 and FU-A) is minor, whereas the benefits are significant. As a
2511 result, the support of STAP-A and FU-A is required. Additionally,
2512
2513
2514
2515
2516 <span class="grey">Wenger, et al. Standards Track [Page 45]</span></pre>
2517 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-46" ></span>
2518 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
2519
2520
2521 support for two of the three NAL unit types defined in this memo,
2522 namely, empty NAL units and NI-MTAP is needed, as specified in
2523 <a href="#section-4.5.1">Section 4.5.1</a>.
2524
2525 A NAL unit of small size SHOULD be encapsulated in an aggregation
2526 packet together with one or more other NAL units. For example, non-
2527 VCL NAL units such as access unit delimiters, parameter sets, or SEI
2528 NAL units are typically small.
2529
2530 A prefix NAL unit and the NAL unit with which it is associated, and
2531 which follows the prefix NAL unit in decoding order, SHOULD be
2532 included in the same aggregation packet whenever an aggregation
2533 packet is used for the associated NAL unit, unless this would violate
2534 session MTU constraints or if fragmentation units are used for the
2535 associated NAL unit.
2536
2537 Informative note: Although the prefix NAL unit is ignored by an
2538 H.264/AVC decoder, it is necessary in the SVC decoding process.
2539
2540 Given the small size of the prefix NAL unit, it is best if it is
2541 transported in the same RTP packet as its associated NAL unit.
2542
2543 When only an H.264/AVC compatible subset of the SVC base layer is
2544 transmitted in an RTP session, the subset MUST be encapsulated
2545 according to [<a href="./rfc6184" title="&quot;RTP Payload Format for H.264 Video&quot;">RFC6184</a>]. This way, an [<a href="./rfc6184" title="&quot;RTP Payload Format for H.264 Video&quot;">RFC6184</a>] receiver will be able
2546 to receive the H.264/AVC compatible bitstream subset.
2547
2548 When a set of layers including one or more SVC enhancement layers is
2549 transmitted in an RTP session, the set SHOULD be carried in one RTP
2550 stream that SHOULD be encapsulated according to this memo.
2551
2552 <span class="h3"><a class="selflink" id="section-5.2" href="#section-5.2">5.2</a>. Packetization Rules for Multi-Session Transmission</span>
2553
2554 When MST is used, the packetization rules specified in <a href="#section-5.1">Section 5.1</a>
2555 still apply. In addition, the following packetization rules MUST be
2556 followed, to ensure that decoding order of NAL units carried in the
2557 sessions can be correctly recovered for each of the MST packetization
2558 modes using the de-packetization process specified in <a href="#section-6.2">Section 6.2</a>.
2559
2560 The NI-T and NI-TC modes both use timestamps to recover the decoding
2561 order. In order to be able to do so, it is necessary for the RTP
2562 packet stream to contain data for all sampling instances of a given
2563 RTP session in all enhancement RTP sessions that depend on the given
2564 RTP session. The NI-C and I-C modes do not have this limitation, and
2565 use the CS-DON values as a means to explicitly indicate decoding
2566 order, either directly coded in PACSI NAL units, or inferred from
2567
2568
2569
2570
2571
2572 <span class="grey">Wenger, et al. Standards Track [Page 46]</span></pre>
2573 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-47" ></span>
2574 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
2575
2576
2577 them using the packetization rules. It is noted that the NI-TC mode
2578 offers both alternatives and it is up to the receiver to select which
2579 one to use.
2580
2581 <span class="h4"><a class="selflink" id="section-5.2.1" href="#section-5.2.1">5.2.1</a>. NI-T/NI-TC Packetization Rules</span>
2582
2583 When using the NI-T mode and a PACSI NAL unit is present, the T bit
2584 MUST be equal to 0, i.e., the DONC field MUST NOT be present.
2585
2586 When using the NI-T mode, the optional parameters sprop-mst-remux-
2587 buf-size, sprop-remux-buf-req, remux-buf-cap, sprop-remux-init-buf-
2588 time, sprop-mst-max-don-diff MUST NOT be present.
2589
2590 When the NI-T or NI-TC MST mode is in use, the following applies.
2591
2592 If one or more NAL units of an access unit of sampling time instance
2593 t is present in RTP session A, then one or more NAL units of the same
2594 access unit MUST be present in any enhancement RTP session that
2595 depends on RTP session A.
2596
2597 Informative note: The mapping between RTP and NTP format
2598 timestamps is conveyed in RTCP SR packets. In addition, the
2599 mechanisms for faster media timestamp synchronization discussed in
2600 [<a href="./rfc6051" title="&quot;Rapid Synchronisation of RTP Flows&quot;">RFC6051</a>] may be used to speed up the acquisition of the RTP-to-
2601 wall-clock mapping.
2602
2603 Informative note: The rule above may require the insertion of NAL
2604 units, typically when temporal scalability is used, i.e., an
2605 enhancement RTP session does not contain any NAL units for an
2606 access unit with a particular NTP timestamp (media timestamp),
2607 which, however, is present in a lower enhancement RTP session or
2608 the base RTP session. There are two ways to insert additional NAL
2609 units in order to satisfy this rule:
2610
2611 - One option for adding additional NAL units is to use empty NAL
2612 units (defined in <a href="#section-4.10">Section 4.10</a>), which can be used by the
2613 process described in <a href="#section-6.2.1">Section 6.2.1</a> for the access unit
2614 reordering process.
2615
2616 - Additional NAL units may also be added by the encoder itself,
2617 for example, by transmitting coded data that simply instruct the
2618 decoder to repeat the previous picture. This option, however,
2619 may be difficult to use with pre-encoded content.
2620
2621 If a packet must be inserted in order to satisfy the above rule,
2622 e.g., in case of a MANE generating multiple RTP streams out of a
2623 single RTP stream, the inserted packet must have an RTP timestamp
2624 that maps to the same wall-clock time (in NTP format) as the one of
2625
2626
2627
2628 <span class="grey">Wenger, et al. Standards Track [Page 47]</span></pre>
2629 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-48" ></span>
2630 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
2631
2632
2633 the RTP timestamp of any packet of the access unit present in any
2634 lower enhancement RTP session or the base RTP session. This is easy
2635 to accomplish if the NAL unit or the packet can be inserted at the
2636 time of the RTP stream generation, since the media timestamp (NTP
2637 timestamp) must be the same for the inserted packet and the packet of
2638 the corresponding access unit. If there is no knowledge of the media
2639 time at RTP stream generation or if the RTP streams are not generated
2640 at the same instance, this can be also applied later in the
2641 transmission process. In this case the NTP timestamp of the inserted
2642 packet can be calculated as follows.
2643
2644 Assume that a packet A2 of an access unit with RTP timestamp TS_A2 is
2645 present in base RTP session A, and that no packet of that access unit
2646 is present in enhancement RTP session B, as shown in Figure 5. Thus,
2647 a packet B2 must be inserted into session B following the rule above.
2648 The most recent RTCP sender report in session A carries NTP timestamp
2649 NTP_A and the RTP timestamp TS_A. The sender report in session B
2650 with a lower NTP timestamp than NTP_A is NTP_B, and carries the RTP
2651 timestamp TS_B.
2652
2653 RTP session B:..B0........B1........(B2)......................
2654
2655 RTCP session B:.....SR(NTP_B,TS_B).............................
2656
2657 RTP session A:..A0........A1........A2........................
2658
2659 RTCP session A:..................SR(NTP_A,TS_A)................
2660
2661 -----------------|--x------|-----x---|------------------------&gt;
2662 NTP time
2663 --------------------+&lt;----------&gt;+&lt;-&gt;+------------------------&gt;
2664 t1 t2 RTP TS(B) time
2665
2666 Figure 5. Example calculation of RTP timestamp for packet
2667 insertion in an enhancement layer RTP session
2668
2669 The vertical bars ("|")in the NTP time line in the figure above
2670 indicate that access unit data is present in at least one of the
2671 sessions. The "x" marks indicate the times of the sender reports.
2672 The RTP timestamp time line for session B, shown right below the NTP
2673 time line, indicates two time segments, t1 and t2. t1 is the time
2674 difference between the sender reports between the two sessions,
2675 expressed in RTP timestamp clock ticks, and t2 is the time difference
2676 from the session A sender report to the A2 packet, again expressed in
2677 RTP timestamp clock ticks. The sum of these differences is added to
2678
2679
2680
2681
2682
2683
2684 <span class="grey">Wenger, et al. Standards Track [Page 48]</span></pre>
2685 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-49" ></span>
2686 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
2687
2688
2689 the RTP timestamp of the session report from session B in order to
2690 derive the correct RTP timestamp for the inserted packet B2. In
2691 other words:
2692
2693 TS_B2 = TS_B + t1 + t2
2694
2695 Let toRTP() be a function that calculates the RTP time difference (in
2696 clock ticks of the used clock) given an NTP timestamp difference, and
2697 effRTPdiff() be a function that calculates the effective difference
2698 between two timestamps, including wraparounds:
2699
2700 effRTPdiff( ts1, ts2 ):
2701
2702 if( ts1 &lt;= ts2 ) then
2703 effRTPdiff := ts1-ts2
2704 else
2705 effRTPDiff := (4294967296 + ts2) - ts1
2706 We have:
2707
2708 t1 = toRTP(NTP_A - NTP_B) and t2 = effRTPdiff(TS_A2, TS_A)
2709
2710 Hence in order to generate the RTP timestamp TS_B2 for the inserted
2711 packet B2, the RTP timestamp for packet B2 TS_B2 can be calculated as
2712 follows.
2713
2714 TS_B2 = TS_B + toRTP(NTP_A - NTP_B) + effRTPdiff(TS_A2, TS_A)
2715
2716 <span class="h4"><a class="selflink" id="section-5.2.2" href="#section-5.2.2">5.2.2</a>. NI-C/NI-TC Packetization Rules</span>
2717
2718 When the NI-C or NI-TC MST mode is in use, the following applies for
2719 each of the RTP sessions.
2720
2721 o For each single NAL unit packet containing a non-PACSI NAL unit,
2722 the previous packet, if present, MUST have the same RTP timestamp
2723 as the single NAL unit packet, and the following applies.
2724
2725 o If the NALU-time of the non-PACSI NAL unit is not equal to the
2726 NALU-time of the previous non-PACSI NAL unit in decoding order,
2727 the previous packet MUST contain a PACSI NAL unit containing
2728 the DONC field.
2729
2730 o In an STAP-A packet the first NAL unit in the STAP-A packet MUST
2731 be a PACSI NAL unit containing the DONC field.
2732
2733 o For an FU-A packet the previous packet MUST have the same RTP
2734 timestamp as the FU-A packet, and the following applies.
2735
2736
2737
2738
2739
2740 <span class="grey">Wenger, et al. Standards Track [Page 49]</span></pre>
2741 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-50" ></span>
2742 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
2743
2744
2745 o If the FU-A packet is the start of the fragmented NAL unit, the
2746 following applies.
2747
2748 o If the NALU-time of the fragmented NAL unit is not equal to
2749 the NALU-time of the previous non-PACSI NAL unit in decoding
2750 order, the previous packet MUST contain a PACSI NAL unit
2751 containing the DONC field;
2752
2753 o Otherwise, (the NALU-time of the fragmented NAL unit is
2754 equal to the NALU-time of the previous non-PACSI NAL unit in
2755 decoding order), the previous packet MAY contain a PACSI NAL
2756 unit containing the DONC field.
2757
2758 o Otherwise, if the FU-A packet is the end of the fragmented NAL
2759 unit, the following applies.
2760
2761 o If the next non-PACSI NAL unit in decoding order has NALU-
2762 time equal to the NALU-time of the fragmented NAL unit, and
2763 is carried in a number of FU-A packets or a single NAL unit
2764 packet, the next packet MUST be a single NAL unit packet
2765 containing a PACSI NAL unit containing the DONC field.
2766
2767 o Otherwise (the FU-A packet is neither the start nor the end
2768 of the fragmented NAL unit), the previous packet MUST be a
2769 FU-A packet.
2770
2771 o For each single NAL unit packet containing a PACSI NAL unit, if
2772 present, the PACSI NAL unit MUST contain the DONC field.
2773
2774 o When the optional media type parameter sprop-mst-csdon-always-
2775 present is equal to 1, the session packetization mode in use MUST
2776 be the non-interleaved mode, and only STAP-A and NI-MTAP packets
2777 can be used.
2778
2779 <span class="h4"><a class="selflink" id="section-5.2.3" href="#section-5.2.3">5.2.3</a>. I-C Packetization Rules</span>
2780
2781 When the I-C MST packetization mode is in use, the following applies.
2782
2783 o When a PACSI NAL unit is present, the T bit MUST be equal to 0,
2784 i.e., the DONC field is not present, and the Y bit MUST be equal
2785 to 0, i.e., the TL0PICIDX and IDRPICID are not present.
2786
2787 <span class="h4"><a class="selflink" id="section-5.2.4" href="#section-5.2.4">5.2.4</a>. Packetization Rules for Non-VCL NAL Units</span>
2788
2789 NAL units that do not directly encode video slices are known in H.264
2790 as non-VCL NAL units. Non-VCL units that are only used by, or only
2791 relevant to, enhancement RTP sessions SHOULD be sent in the lowest
2792 session to which they are relevant.
2793
2794
2795
2796 <span class="grey">Wenger, et al. Standards Track [Page 50]</span></pre>
2797 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-51" ></span>
2798 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
2799
2800
2801 Some senders, however, such as those sending pre-encoded data, may be
2802 unable to easily determine which non-VCL units are relevant to which
2803 session. Thus, non-VCL NAL units MAY, instead, be sent in a session
2804 on which the session using these non-VCL NAL units depends (e.g., the
2805 base RTP session).
2806
2807 If a non-VCL unit is relevant to more than one RTP session, neither
2808 of which depends on the other(s), the NAL unit MAY be sent in another
2809 session on which all these sessions depend.
2810
2811 <span class="h4"><a class="selflink" id="section-5.2.5" href="#section-5.2.5">5.2.5</a>. Packetization Rules for Prefix NAL Units</span>
2812
2813 <a href="#section-5.1">Section 5.1</a> of this memo applies, with the following addition. If
2814 the base layer is sent in a base RTP session using [<a href="./rfc6184" title="&quot;RTP Payload Format for H.264 Video&quot;">RFC6184</a>], prefix
2815 NAL units MAY be sent in the lowest enhancement RTP session rather
2816 than in the base RTP session.
2817
2818 <span class="h2"><a class="selflink" id="section-6" href="#section-6">6</a>. De-Packetization Process</span>
2819
2820 <span class="h3"><a class="selflink" id="section-6.1" href="#section-6.1">6.1</a>. De-Packetization Process for Single-Session Transmission</span>
2821
2822 For single-session transmission, where a single RTP session is used,
2823 the de-packetization process specified in <a href="./rfc6184#section-7">Section&nbsp;7 of [RFC6184]</a>
2824 applies.
2825
2826 <span class="h3"><a class="selflink" id="section-6.2" href="#section-6.2">6.2</a>. De-Packetization Process for Multi-Session Transmission</span>
2827
2828 For multi-session transmission, where more than one RTP session is
2829 used to receive data from the same SVC bitstream, the de-
2830 packetization process is specified as follows.
2831
2832 As for a single RTP session, the general concept behind the de-
2833 packetization process is to reorder NAL units from transmission order
2834 to the NAL unit decoding order.
2835
2836 The sessions to be received MUST be identified by mechanisms
2837 specified in <a href="#section-7.2.3">Section 7.2.3</a>. An enhancement RTP session typically
2838 contains an RTP stream that depends on at least one other RTP
2839 session, as indicated by mechanisms defined in <a href="#section-7.2.3">Section 7.2.3</a>. A
2840 lower RTP session to an enhancement RTP session is an RTP session on
2841 which the enhancement RTP session depends. The lowest RTP session
2842 for a receiver is the base RTP session, which does not depend on any
2843 other RTP session received by the receiver. The highest RTP session
2844 for a receiver is the RTP session on which no other RTP session
2845 received by the receiver depends.
2846
2847
2848
2849
2850
2851
2852 <span class="grey">Wenger, et al. Standards Track [Page 51]</span></pre>
2853 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-52" ></span>
2854 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
2855
2856
2857 For each of the RTP sessions, the RTP reception process as specified
2858 in <a href="./rfc3550">RFC 3550</a> is applied. Then the received packets are passed into
2859 the payload de-packetization process as defined in this memo.
2860
2861 The decoding order of the NAL units carried in all the associated RTP
2862 sessions is then recovered by applying one of the following
2863 subsections, depending on which of the MST packetization modes is in
2864 use.
2865
2866 <span class="h4"><a class="selflink" id="section-6.2.1" href="#section-6.2.1">6.2.1</a>. Decoding Order Recovery for the NI-T and NI-TC Modes</span>
2867
2868 The following process MUST be applied when the NI-T packetization
2869 mode is in use. The following process MAY be applied when the NI-TC
2870 packetization mode is in use.
2871
2872 The process is based on RTP session dependency signaling, RTP
2873 sequence numbers, and timestamps.
2874
2875 The decoding order of NAL units within an RTP packet stream in RTP
2876 session is given by the ordering of sequence numbers SN of the RTP
2877 packets that contain the NAL units, and the order of appearance of
2878 NAL units within a packet.
2879
2880 Timing information according to the media timestamp TS, i.e., the NTP
2881 timestamp as derived from the RTP timestamp of an RTP packet, is
2882 associated with all NAL units contained in the same RTP packet
2883 received in an RTP session.
2884
2885 For NI-MTAP packets the NALU-time is derived for each contained NAL
2886 unit by using the "TS offset" value in the NI-MTAP packet as defined
2887 in <a href="#section-4.10">Section 4.10</a>, and is used instead of the RTP packet timestamp to
2888 derive the media timestamp, e.g., using the NTP wall clock as
2889 provided via RTCP sender reports. NAL units contained in
2890 fragmentation packets are handled as defragmented, entire NAL units
2891 with their own media timestamps. All NAL units associated with the
2892 same value of media timestamp TS are part of the same access unit
2893 AU(TS). Any empty NAL units SHOULD be kept as, effectively, access
2894 unit indicators in the reordering process. Empty NAL units and PACSI
2895 NAL units SHOULD be removed before passing access unit data to the
2896 decoder.
2897
2898 Informative note: These empty NAL units are used to associate NAL
2899 units present in other RTP sessions with RTP sessions not
2900 containing any data for an access unit of a particular time
2901 instance. They act as access unit indicators in sessions that
2902 would otherwise contain no data for the particular access unit.
2903 The presence of these NAL units is ensured by the packetization
2904 rules in <a href="#section-5.2.1">Section 5.2.1</a>.
2905
2906
2907
2908 <span class="grey">Wenger, et al. Standards Track [Page 52]</span></pre>
2909 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-53" ></span>
2910 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
2911
2912
2913 It is assumed that the receiver has established an operation point
2914 (DID, QID, and TID values), and has identified the highest
2915 enhancement RTP session for this operation point. The decoding order
2916 of NAL units from multiple RTP streams in multiple RTP sessions MUST
2917 be recovered into a single sequence of NAL units, grouped into access
2918 units, by performing any process equivalent to the following steps.
2919 The general process is described in <a href="./rfc6051#section-4.2">Section&nbsp;4.2 of [RFC6051]</a>. For
2920 convenience the instructions of [<a href="./rfc6051" title="&quot;Rapid Synchronisation of RTP Flows&quot;">RFC6051</a>] are repeated and applied to
2921 NAL units rather than to full RTP packets. Additionally, SVC-
2922 specific extensions to the procedure in <a href="./rfc6051#section-4.2">Section&nbsp;4.2. of [RFC6051]</a>
2923 are presented in the following list:
2924
2925 o The process should be started with the NAL units received in
2926 the highest RTP session with the first media timestamp TS (in
2927 NTP format) available in the session's (de-jittering) buffer.
2928 It is assumed that packets in the de-jittering buffer are
2929 already stored in RTP sequence number order.
2930
2931 o Collect all NAL units associated with the same value of media
2932 timestamp TS, starting from the highest RTP session, from all
2933 the (de-jittering) buffers of the received RTP sessions. The
2934 collected NAL units will be those associated with the access
2935 unit AU(TS).
2936
2937 o Place the collected NAL units in the order of session
2938 dependency as derived by the dependency indication as specified
2939 in <a href="#section-7.2.3">Section 7.2.3</a>, starting from the lowest RTP session.
2940
2941 o Place the session ordered NAL units in decoding order within
2942 the particular access unit by satisfying the NAL unit ordering
2943 rules for SVC access units, as described in the informative
2944 algorithm provided in <a href="#section-6.2.1.1">Section 6.2.1.1</a>.
2945
2946 o Remove NI-MTAP and any PACSI NAL units from the access unit
2947 AU(TS).
2948
2949 o The access units can then be transferred to the decoder.
2950 Access units AU(TS) are transferred to the decoder in the order
2951 of appearance (given by the order of RTP sequence numbers) of
2952 media timestamp values TS in the highest RTP session associated
2953 with access unit AU(TS).
2954
2955 Informative note: Due to packet loss it is possible that not
2956 all sessions may have NAL units present for the media
2957 timestamp value TS present in the highest RTP session. In
2958 such a case, an algorithm may: a) proceed to the next
2959 complete access unit with NAL units present in all the
2960 received RTP sessions; or b) consider a new highest RTP
2961
2962
2963
2964 <span class="grey">Wenger, et al. Standards Track [Page 53]</span></pre>
2965 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-54" ></span>
2966 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
2967
2968
2969 session, the highest RTP session for which the access unit
2970 is complete, and apply the process above. The algorithm may
2971 return to the original highest RTP session when a complete
2972 and error-free access unit that contains NAL units in all
2973 the sessions is received.
2974
2975 The following gives an informative example.
2976
2977 The example shown in Figure 6 refers to three RTP sessions A, B, and
2978 C containing an SVC bitstream transmitted as 3 sources. In the
2979 example, the dependency signaling (described in <a href="#section-7.2.3">Section 7.2.3</a>)
2980 indicates that session A is the base RTP session, B is the first
2981 enhancement RTP session and depends on A, and C is the second
2982 enhancement RTP session and depends on A and B. A hierarchical
2983 picture coding prediction structure is used, in which session A has
2984 the lowest frame rate and sessions B and C have the same but higher
2985 frame rate.
2986
2987 The figure shows NAL units contained in RTP packets that are stored
2988 in the de-jittering buffer at the receiver for session de-
2989 packetization. The NAL units are already reordered according to
2990 their RTP sequence number order and, if within an aggregation packet,
2991 according to the order of their appearance within the aggregation
2992 packet. The figure indicates for the received NAL units the decoding
2993 order within the sessions, as well as the associated media (NTP)
2994 timestamps ("TS[..]"). NAL units of the same access unit within a
2995 session are grouped by "(.,.)" and share the same media timestamp TS,
2996 which is shown at the bottom of the figure. Note that the timestamps
2997 are not in increasing order since, in this example, the decoding
2998 order is different from the output/display order.
2999
3000 The process first proceeds to the NAL units associated with the first
3001 media timestamp TS[1] present in the highest session C and
3002 removes/ignores all preceding (in decoding order) NAL units to NAL
3003 units with TS[1] in each of the de-jittering buffers of RTP sessions
3004 A, B, and C. Then, starting from session C, the first media
3005 timestamp available in decoding order (TS[1]) is selected and NAL
3006 units starting from RTP session A, and sessions B and C are placed in
3007 order of the RTP session dependency as required by <a href="#section-7.2.3">Section 7.2.3</a> of
3008 this memo (in the example for TS[1]: first session B and then session
3009 C) into the access unit AU(TS[1]) associated with media timestamp
3010 TS[1]. Then the next media timestamp TS[3] in order of appearance in
3011 the highest RTP session C is processed and the process described
3012 above is repeated. Note that there may be access units with no NAL
3013 units present, e.g., in the lowest RTP session A (see, e.g., TS[1]).
3014 With TS[8], the first access unit with NAL units present in all the
3015 RTP sessions appears in the buffers.
3016
3017
3018
3019
3020 <span class="grey">Wenger, et al. Standards Track [Page 54]</span></pre>
3021 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-55" ></span>
3022 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
3023
3024
3025 C: ------------(1,2)-(3,4)--(5)---(6)---(7,8)(9,10)-(11)--(12)----
3026 | | | | | | | | | |
3027 B: -(1,2)-(3,4)-(5)---(6)--(7,8)-(9,10)-(11)-(12)--(13,14)(15,15)-
3028 | | | | | |
3029 A: -------(1)---------------(2)---(3)---------------(4)----(5)----
3030 ---------------------------------------------------decoding order--&gt;
3031
3032 TS: [4] [2] [1] [3] [8] [6] [5] [7] [12] [10]
3033
3034 Key:
3035 A, B, C - RTP sessions
3036 Integer values in "()" - NAL unit decoding order within RTP session
3037 "( )" - groups the NAL units of an access unit
3038 in an RTP session
3039 "|" - indicates corresponding NAL units of the
3040 same access unit AU(TS[..]) in the RTP
3041 sessions
3042 Integer values in "[]" - media timestamp TS, sampling time
3043 as derived, e.g., from NTP timestamp
3044 associated with the access unit AU(TS[..]),
3045 consisting of NAL units in the sessions
3046 above each TS value.
3047
3048 Figure 6. Example of decoding order recovery in multi-source
3049 transmission.
3050
3051 <span class="h5"><a class="selflink" id="section-6.2.1.1" href="#section-6.2.1.1">6.2.1.1</a>. Informative Algorithm for NI-T Decoding Order Recovery within</span>
3052 <span class="h5"> an Access Unit</span>
3053
3054 Within an access unit, the [<a href="#ref-H.264" title="&quot;Advanced video coding for generic audiovisual services&quot;">H.264</a>] specification (Sections <a href="#section-7.4.1.2.3">7.4.1.2.3</a>
3055 and G.7.4.1.2.3) constrains the valid decoding order of NAL units.
3056
3057 These constraints make it possible to reconstruct a valid decoding
3058 order for the NAL units of an access unit based only on the order of
3059 NAL units in each session, the NAL unit headers, and Supplemental
3060 Enhancement Information message headers.
3061
3062 This section specifies an informative algorithm to reconstruct a
3063 valid decoding order for NAL units within an access unit. Other NAL
3064 unit orderings may also be valid; however, any compliant NAL unit
3065 ordering will describe the same video stream and ancillary data as
3066 the one produced by this algorithm.
3067
3068 An actual implementation, of course, needs only to behave "as if"
3069 this reordering is done. In particular, NAL units that are discarded
3070 by an implementation's decoding process do not need to be reordered.
3071
3072
3073
3074
3075
3076 <span class="grey">Wenger, et al. Standards Track [Page 55]</span></pre>
3077 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-56" ></span>
3078 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
3079
3080
3081 In this algorithm, NAL units within an access unit are first ordered
3082 by NAL unit type, in the order specified in Table 12 below, except
3083 from NAL unit type 14, which is handled specially as described in the
3084 table. NAL units of the same type are then ordered as specified for
3085 the type, if necessary.
3086
3087 For the purposes of this algorithm, "session order" is the order of
3088 NAL units implied by their transmission order within an RTP session.
3089 For the non-interleaved and single NAL unit modes, this is the RTP
3090 sequence number order coupled with the order of NAL units within an
3091 aggregation unit.
3092
3093 Table 12. Ordering of NAL unit types within an Access Unit
3094
3095 Type Description / Comments
3096 -----------------------------------------------------------
3097 9 Access unit delimiter
3098
3099 7 Sequence parameter set
3100
3101 13 Sequence parameter set extension
3102
3103 15 Subset sequence parameter set
3104
3105 8 Picture parameter set
3106
3107 16-18 Reserved
3108
3109 6 Supplemental enhancement information (SEI)
3110 If an SEI message with a first payload of 0 (Buffering
3111 Period) is present, it must be the first SEI message.
3112
3113 If SEI messages with a Scalable Nesting (30) payload and
3114 a nested payload of 0 (Buffering Period) are present,
3115 these then follow the first SEI message. Such an SEI
3116 message with the all_layer_representations_in_au_flag
3117 equal to 1 is placed first, followed by any others,
3118 sorted in increasing order of DQId.
3119
3120 All other SEI messages follow in any order.
3121
3122 14 Prefix NAL unit in scalable extension
3123 1 Coded slice of a non-IDR picture
3124 5 Coded slice of an IDR picture
3125
3126
3127
3128
3129
3130
3131
3132 <span class="grey">Wenger, et al. Standards Track [Page 56]</span></pre>
3133 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-57" ></span>
3134 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
3135
3136
3137 NAL units of type 1 or 5 will be sent within only a
3138 single session for any given access unit. They are
3139 placed in session order. (Note: Any given access unit
3140 will contain only NAL units of type 1 or type 5, not
3141 both.)
3142
3143 If NAL units of type 14 are present, every NAL unit of
3144 type 1 or 5 is prefixed by a NAL unit of type 14. (Note:
3145 Within an access unit, every NAL unit of type 14 is
3146 identical, so correlation of type 14 NAL units with the
3147 other NAL units is not necessary.)
3148
3149 12 Filler data
3150
3151 The only restriction of filler data NAL units within an
3152 access unit is that they shall not precede the first VCL
3153 NAL unit with the same access unit.
3154
3155 19 Coded slice of an auxiliary coded picture without
3156 partitioning
3157
3158 These NAL units will be sent within only a single
3159 session for any given access unit, and are placed in
3160 session order.
3161
3162 20 Coded slice in scalable extension
3163 21-23 Reserved
3164
3165 Type 20 NAL units are placed in increasing order of DQId.
3166 Within each DQId value, they are placed in session order.
3167
3168 (Note: SVC slices with a given DQId value will be sent
3169 within only a single session for any given access unit.)
3170
3171 Type 21-23 NAL units are placed immediately following
3172 the non-reserved-type VCL NAL unit they follow in
3173 session order.
3174
3175 10 End of sequence
3176
3177 11 End of stream
3178
3179 <span class="h4"><a class="selflink" id="section-6.2.2" href="#section-6.2.2">6.2.2</a>. Decoding Order Recovery for the NI-C, NI-TC, and I-C Modes</span>
3180
3181 The following process MUST be used when either the NI-C or I-C MST
3182 packetization mode is in use. The following process MAY be applied
3183 when the NI-TC MST packetization mode is in use.
3184
3185
3186
3187
3188 <span class="grey">Wenger, et al. Standards Track [Page 57]</span></pre>
3189 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-58" ></span>
3190 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
3191
3192
3193 The RTP packets output from the RTP-level reception processing for
3194 each session are placed into a re-multiplexing buffer.
3195
3196 It is RECOMMENDED to set the size of the re-multiplexing buffer (in
3197 bytes) equal to or greater than the value of the sprop-remux-buf-req
3198 media type parameter of the highest RTP session the receiver
3199 receives.
3200
3201 The CS-DON value is calculated and stored for each NAL unit.
3202
3203 Informative note: The CS-DON value of a NAL unit may rely on
3204 information carried in another packet than the packet containing
3205 the NAL unit. This happens, e.g., when the CS-DON values need to
3206 be derived for non-PACSI NAL units contained in single NAL unit
3207 packets, as the single NAL unit packets themselves do not contain
3208 CS-DON information. In this case, when no packet containing
3209 required CS-DON information is received for a NAL unit, this NAL
3210 unit has to be discarded by the receiver as it cannot be fed to
3211 the decoder in the correct order. When the optional media type
3212 parameter sprop-mst-csdon-always-present is equal to 1, no such
3213 dependency exists, i.e., the CS-DON value of any particular NAL
3214 unit can be derived solely according to information in the packet
3215 containing the NAL unit, and therefore, the receiver does not need
3216 to discard any received NAL units.
3217
3218 The receiver operation is described below with the help of the
3219 following functions and constants:
3220
3221 o Function AbsDON is specified in <a href="./rfc6184#section-8.1">Section&nbsp;8.1 of [RFC6184]</a>.
3222
3223 o Function don_diff is specified in <a href="./rfc6184#section-5.5">Section&nbsp;5.5 of [RFC6184]</a>.
3224
3225 o Constant N is the value of the OPTIONAL sprop-mst-remux-buf-size
3226 media type parameter of the highest RTP session incremented by 1.
3227
3228 Initial buffering lasts until one of the following conditions is
3229 fulfilled:
3230
3231 o There are N or more VCL NAL units in the re-multiplexing buffer.
3232
3233 o If sprop-mst-max-don-diff of the highest RTP session is present,
3234 don_diff(m,n) is greater than the value of sprop-mst-max-don-diff
3235 of the highest RTP session, where n corresponds to the NAL unit
3236 having the greatest value of AbsDON among the received NAL units
3237 and m corresponds to the NAL unit having the smallest value of
3238 AbsDON among the received NAL units.
3239
3240
3241
3242
3243
3244 <span class="grey">Wenger, et al. Standards Track [Page 58]</span></pre>
3245 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-59" ></span>
3246 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
3247
3248
3249 o Initial buffering has lasted for the duration equal to or greater
3250 than the value of the OPTIONAL sprop-remux-init-buf-time media
3251 type parameter of the highest RTP session.
3252
3253 The NAL units to be removed from the re-multiplexing buffer are
3254 determined as follows:
3255
3256 o If the re-multiplexing buffer contains at least N VCL NAL units,
3257 NAL units are removed from the re-multiplexing buffer and passed
3258 to the decoder in the order specified below until the buffer
3259 contains N-1 VCL NAL units.
3260
3261 o If sprop-mst-max-don-diff of the highest RTP session is present,
3262 all NAL units m for which don_diff(m,n) is greater than sprop-
3263 max-don-diff of the highest RTP session are removed from the re-
3264 multiplexing buffer and passed to the decoder in the order
3265 specified below. Herein, n corresponds to the NAL unit having the
3266 greatest value of AbsDON among the NAL units in the re-
3267 multiplexing buffer.
3268
3269 The order in which NAL units are passed to the decoder is specified
3270 as follows:
3271
3272 o Let PDON be a variable that is initialized to 0 at the beginning
3273 of the RTP sessions.
3274
3275 o For each NAL unit associated with a value of CS-DON, a CS-DON
3276 distance is calculated as follows. If the value of CS-DON of the
3277 NAL unit is larger than the value of PDON, the CS-DON distance is
3278 equal to CS-DON - PDON. Otherwise, the CS-DON distance is equal
3279 to 65535 - PDON + CS-DON + 1.
3280
3281 o NAL units are delivered to the decoder in increasing order of CS-
3282 DON distance. If several NAL units share the same value of CS-
3283 DON distance, they can be passed to the decoder in any order.
3284
3285 o When a desired number of NAL units have been passed to the
3286 decoder, the value of PDON is set to the value of CS-DON for the
3287 last NAL unit passed to the decoder.
3288
3289 <span class="h2"><a class="selflink" id="section-7" href="#section-7">7</a>. Payload Format Parameters</span>
3290
3291 This section specifies the parameters that MAY be used to select
3292 optional features of the payload format and certain features of the
3293 bitstream. The parameters are specified here as part of the media
3294 type registration for the SVC codec. A mapping of the parameters
3295 into the Session Description Protocol (SDP) [<a href="./rfc4566" title="&quot;SDP: Session Description Protocol&quot;">RFC4566</a>] is also
3296
3297
3298
3299
3300 <span class="grey">Wenger, et al. Standards Track [Page 59]</span></pre>
3301 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-60" ></span>
3302 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
3303
3304
3305 provided for applications that use SDP. Equivalent parameters could
3306 be defined elsewhere for use with control protocols that do not use
3307 SDP.
3308
3309 Some parameters provide a receiver with the properties of the stream
3310 that will be sent. The names of all these parameters start with
3311 "sprop" for stream properties. Some of these "sprop" parameters are
3312 limited by other payload or codec configuration parameters. For
3313 example, the sprop-parameter-sets parameter is constrained by the
3314 profile-level-id parameter. The media sender selects all "sprop"
3315 parameters rather than the receiver. This uncommon characteristic of
3316 the "sprop" parameters may be incompatible with some signaling
3317 protocol concepts, in which case the use of these parameters SHOULD
3318 be avoided.
3319
3320 <span class="h3"><a class="selflink" id="section-7.1" href="#section-7.1">7.1</a>. Media Type Registration</span>
3321
3322 The media subtype for the SVC codec has been allocated from the IETF
3323 tree.
3324
3325 The receiver MUST ignore any unspecified parameter.
3326
3327 Informative note: Requiring that the receiver ignore unspecified
3328 parameters allows for backward compatibility of future extensions.
3329 For example, if a future specification that is backward compatible
3330 to this specification specifies some new parameters, then a
3331 receiver according to this specification is capable of receiving
3332 data per the new payload but ignoring those parameters newly
3333 specified in the new payload specification. This provision is
3334 also present in [<a href="./rfc6184" title="&quot;RTP Payload Format for H.264 Video&quot;">RFC6184</a>].
3335
3336 Media Type name: video
3337
3338 Media subtype name: H264-SVC
3339
3340 Required parameters: none
3341
3342 OPTIONAL parameters:
3343
3344 In the following definitions of parameters, "the stream" or "the
3345 NAL unit stream" refers to all NAL units conveyed in the current
3346 RTP session in SST, and all NAL units conveyed in the current RTP
3347 session and all NAL units conveyed in other RTP sessions that the
3348 current RTP session depends on in MST.
3349
3350
3351
3352
3353
3354
3355
3356 <span class="grey">Wenger, et al. Standards Track [Page 60]</span></pre>
3357 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-61" ></span>
3358 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
3359
3360
3361 profile-level-id:
3362 A base16 [<a href="./rfc4648" title="&quot;The Base16, Base32, and Base64 Data Encodings&quot;">RFC4648</a>] (hexadecimal) representation of the
3363 following three bytes in the sequence parameter set or subset
3364 sequence parameter set NAL unit specified in [<a href="#ref-H.264" title="&quot;Advanced video coding for generic audiovisual services&quot;">H.264</a>]: 1)
3365 profile_idc; 2) a byte herein referred to as profile-iop,
3366 composed of the values of constraint_set0_flag,
3367 constraint_set1_flag, constraint_set2_flag,
3368 constraint_set3_flag, constraint_set4_flag,
3369 constraint_set5_flag, and reserved_zero_2bits, in bit-
3370 significance order, starting from the most-significant bit, and
3371 3) level_idc. Note that reserved_zero_2bits is required to be
3372 equal to 0 in [<a href="#ref-H.264" title="&quot;Advanced video coding for generic audiovisual services&quot;">H.264</a>], but other values for it may be specified
3373 in the future by ITU-T or ISO/IEC.
3374
3375 The profile-level-id parameter indicates the default sub-
3376 profile, i.e., the subset of coding tools that may have been
3377 used to generate the stream or that the receiver supports, and
3378 the default level of the stream or the one that the receiver
3379 supports.
3380
3381 The default sub-profile is indicated collectively by the
3382 profile_idc byte and some fields in the profile-iop byte.
3383 Depending on the values of the fields in the profile-iop byte,
3384 the default sub-profile may be the same set of coding tools
3385 supported by one profile, or a common subset of coding tools of
3386 multiple profiles, as specified in Subsection G.7.4.2.1.1 of
3387 [<a href="#ref-H.264" title="&quot;Advanced video coding for generic audiovisual services&quot;">H.264</a>]. The default level is indicated by the level_idc byte,
3388 and, when profile_idc is equal to 66, 77, or 88 (the Baseline,
3389 Main, or Extended profile) and level_idc is equal to 11,
3390 additionally by bit 4 (constraint_set3_flag) of the profile-iop
3391 byte. When profile_idc is equal to 66, 77, or 88 (the
3392 Baseline, Main, or Extended profile) and level_idc is equal to
3393 11, and bit 4 (constraint_set3_flag) of the profile-iop byte is
3394 equal to 1, the default level is Level 1b.
3395
3396 Table 13 lists all profiles defined in Annexes A and G of
3397 [<a href="#ref-H.264" title="&quot;Advanced video coding for generic audiovisual services&quot;">H.264</a>] and, for each of the profiles, the possible
3398 combinations of profile_idc and profile-iop that represent the
3399 same sub-profile.
3400
3401 Table 13. Combinations of profile_idc and profile-iop
3402 representing the same sub-profile corresponding to the full set
3403 of coding tools supported by one profile. In the following, x
3404 may be either 0 or 1, while the profile names are indicated as
3405 follows. CB: Constrained Baseline profile, B: Baseline
3406 profile, M: Main profile, E: Extended profile, H: High profile,
3407 H10: High 10 profile, H42: High 4:2:2 profile, H44: High 4:4:4
3408 Predictive profile, H10I: High 10 Intra profile, H42I: High
3409
3410
3411
3412 <span class="grey">Wenger, et al. Standards Track [Page 61]</span></pre>
3413 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-62" ></span>
3414 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
3415
3416
3417 4:2:2 Intra profile, H44I: High 4:4:4 Intra profile, C44I:
3418 CAVLC 4:4:4 Intra profile, SB: Scalable Baseline profile, SH:
3419 Scalable High profile, and SHI: Scalable High Intra profile.
3420
3421 Profile profile_idc profile-iop
3422 (hexadecimal) (binary)
3423
3424 CB 42 (B) x1xx0000
3425 same as: 4D (M) 1xxx0000
3426 same as: 58 (E) 11xx0000
3427 B 42 (B) x0xx0000
3428 same as: 58 (E) 10xx0000
3429 M 4D (M) 0x0x0000
3430 E 58 00xx0000
3431 H 64 00000000
3432 H10 6E 00000000
3433 H42 7A 00000000
3434 H44 F4 00000000
3435 H10I 6E 00010000
3436 H42I 7A 00010000
3437 H44I F4 00010000
3438 C44I 2C 00010000
3439 SB 53 x0000000
3440 SH 56 0x000000
3441 SHI 56 0x010000
3442
3443 For example, in the table above, profile_idc equal to 58
3444 (Extended) with profile-iop equal to 11xx0000 indicates the
3445 same sub-profile corresponding to profile_idc equal to 42
3446 (Baseline) with profile-iop equal to x1xx0000. Note that other
3447 combinations of profile_idc and profile-iop (not listed in
3448 Table 13) may represent a sub-profile equivalent to the common
3449 subset of coding tools for more than one profile. Note also
3450 that a decoder conforming to a certain profile may be able to
3451 decode bitstreams conforming to other profiles.
3452
3453 If profile-level-id is used to indicate stream properties, it
3454 indicates that, to decode the stream, the minimum subset of
3455 coding tools a decoder has to support is the default sub-
3456 profile, and the lowest level the decoder has to support is the
3457 default level.
3458
3459 If the profile-level-id parameter is used for capability
3460 exchange or session setup, it indicates the subset of coding
3461 tools, which is equal to the default sub-profile, that the
3462 codec supports for both receiving and sending. If max-recv-
3463 level is not present, the default level from profile-level-id
3464 indicates the highest level the codec wishes to support. If
3465
3466
3467
3468 <span class="grey">Wenger, et al. Standards Track [Page 62]</span></pre>
3469 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-63" ></span>
3470 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
3471
3472
3473 max-recv-level is present, it indicates the highest level the
3474 codec supports for receiving. For either receiving or sending,
3475 all levels that are lower than the highest level supported MUST
3476 also be supported.
3477
3478 Informative note: Capability exchange and session setup
3479 procedures should provide means to list the capabilities for
3480 each supported sub-profile separately. For example, the
3481 one-of-N codec selection procedure of the SDP Offer/Answer
3482 model can be used (<a href="./rfc3264#section-10.2">Section&nbsp;10.2 of [RFC3264]</a>). The one-of-N
3483 codec selection procedure may also be used to provide
3484 different combinations of profile_idc and profile-iop that
3485 represent the same sub-profile. When there are many
3486 different combinations of profile_idc and profile-iop that
3487 represent the same sub-profile, using the one-of-N codec
3488 selection procedure may result in a fairly large SDP
3489 message. Therefore, a receiver should understand the
3490 different equivalent combinations of profile_idc and
3491 profile-iop that represent the same sub-profile, and be
3492 ready to accept an offer using any of the equivalent
3493 combinations.
3494
3495 If no profile-level-id is present, the Baseline Profile without
3496 additional constraints at Level 1 MUST be implied.
3497
3498 max-recv-level:
3499 This parameter MAY be used to indicate the highest level a
3500 receiver supports when the highest level is higher than the
3501 default level (the level indicated by profile-level-id). The
3502 value of max-recv-level is a base16 (hexadecimal)
3503 representation of the two bytes after the syntax element
3504 profile_idc in the sequence parameter set NAL unit specified in
3505 [<a href="#ref-H.264" title="&quot;Advanced video coding for generic audiovisual services&quot;">H.264</a>]: profile-iop (as defined above) and level_idc. If (the
3506 level_idc byte of max-recv-level is equal to 11 and bit 4 of
3507 the profile-iop byte of max-recv-level is equal to 1) or (the
3508 level_idc byte of max-recv-level is equal to 9 and bit 4 of the
3509 profile-iop byte of max-recv-level is equal to 0), the highest
3510 level the receiver supports is Level 1b. Otherwise, the
3511 highest level the receiver supports is equal to the level_idc
3512 byte of max-recv-level divided by 10.
3513
3514 max-recv-level MUST NOT be present if the highest level the
3515 receiver supports is not higher than the default level.
3516
3517 max-recv-base-level:
3518 This parameter MAY be used to indicate the highest level a
3519 receiver supports for the base layer when negotiating an SVC
3520 stream. The value of max-recv-base-level is a base16
3521
3522
3523
3524 <span class="grey">Wenger, et al. Standards Track [Page 63]</span></pre>
3525 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-64" ></span>
3526 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
3527
3528
3529 (hexadecimal) representation of the two bytes after the syntax
3530 element profile_idc in the sequence parameter set NAL unit
3531 specified in [<a href="#ref-H.264" title="&quot;Advanced video coding for generic audiovisual services&quot;">H.264</a>]: profile-iop (as defined above) and
3532 level_idc. If (the level_idc byte of max-recv-level is equal
3533 to 11 and bit 4 of the profile-iop byte of max-recv-level is
3534 equal to 1) or (the level_idc byte of max-recv-level is equal
3535 to 9 and bit 4 of the profile-iop byte of max-recv-level is
3536 equal to 0), the highest level the receiver supports for the
3537 base layer is Level 1b. Otherwise, the highest level the
3538 receiver supports for the base layer is equal to the level_idc
3539 byte of max-recv-level divided by 10.
3540
3541 max-mbps, max-fs, max-cpb, max-dpb, and max-br:
3542 The common properties of these parameters are specified in
3543 [<a href="./rfc6184" title="&quot;RTP Payload Format for H.264 Video&quot;">RFC6184</a>].
3544
3545 max-mbps: This parameter is as specified in [<a href="./rfc6184" title="&quot;RTP Payload Format for H.264 Video&quot;">RFC6184</a>].
3546
3547 max-fs: This parameter is as specified in [<a href="./rfc6184" title="&quot;RTP Payload Format for H.264 Video&quot;">RFC6184</a>].
3548
3549 max-cpb: The value of max-cpb is an integer indicating the maximum
3550 coded picture buffer size in units of 1000 bits for the VCL HRD
3551 parameters and in units of 1200 bits for the NAL HRD
3552 parameters. Note that this parameter does not use units of
3553 cpbBrVclFactor and cpbBrNALFactor (see Table A-1 of [<a href="#ref-H.264" title="&quot;Advanced video coding for generic audiovisual services&quot;">H.264</a>]).
3554 The max-cpb parameter signals that the receiver has more memory
3555 than the minimum amount of coded picture buffer memory required
3556 by the signaled highest level conveyed in the value of the
3557 profile-level-id parameter or the max-recv-level parameter.
3558 When max-cpb is signaled, the receiver MUST be able to decode
3559 NAL unit streams that conform to the signaled highest level,
3560 with the exception that the MaxCPB value in Table A-1 of
3561 [<a href="#ref-H.264" title="&quot;Advanced video coding for generic audiovisual services&quot;">H.264</a>] for the signaled highest level is replaced with the
3562 value of max-cpb (after taking cpbBrVclFactor and
3563 cpbBrNALFactor into consideration when needed). The value of
3564 max-cpb (after taking cpbBrVclFactor and cpbBrNALFactor into
3565 consideration when needed) MUST be greater than or equal to the
3566 value of MaxCPB given in Table A-1 of [<a href="#ref-H.264" title="&quot;Advanced video coding for generic audiovisual services&quot;">H.264</a>] for the highest
3567 level. Senders MAY use this knowledge to construct coded video
3568 streams with greater variation of bitrate than can be achieved
3569 with the MaxCPB value in Table A-1 of [<a href="#ref-H.264" title="&quot;Advanced video coding for generic audiovisual services&quot;">H.264</a>].
3570
3571
3572
3573
3574
3575
3576
3577
3578
3579
3580 <span class="grey">Wenger, et al. Standards Track [Page 64]</span></pre>
3581 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-65" ></span>
3582 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
3583
3584
3585 Informative note: The coded picture buffer is used in the
3586 Hypothetical Reference Decoder (HRD, Annex C) of [<a href="#ref-H.264" title="&quot;Advanced video coding for generic audiovisual services&quot;">H.264</a>].
3587 The use of the HRD is recommended in SVC encoders to verify
3588 that the produced bitstream conforms to the standard and to
3589 control the output bitrate. Thus, the coded picture buffer
3590 is conceptually independent of any other potential buffers
3591 in the receiver, including de-interleaving, re-multiplexing,
3592 and de-jitter buffers. The coded picture buffer need not be
3593 implemented in decoders as specified in Annex C of [<a href="#ref-H.264" title="&quot;Advanced video coding for generic audiovisual services&quot;">H.264</a>];
3594 standard-compliant decoders can have any buffering
3595 arrangements provided that they can decode standard-
3596 compliant bitstreams. Thus, in practice, the input buffer
3597 for video decoder can be integrated with the de-
3598 interleaving, re-multiplexing, and de-jitter buffers of the
3599 receiver.
3600
3601 max-dpb: This parameter is as specified in [<a href="./rfc6184" title="&quot;RTP Payload Format for H.264 Video&quot;">RFC6184</a>].
3602
3603 max-br: The value of max-br is an integer indicating the maximum
3604 video bitrate in units of 1000 bits per second for the VCL HRD
3605 parameters and in units of 1200 bits per second for the NAL HRD
3606 parameters. Note that this parameter does not use units of
3607 cpbBrVclFactor and cpbBrNALFactor (see Table A-1 of [<a href="#ref-H.264" title="&quot;Advanced video coding for generic audiovisual services&quot;">H.264</a>]).
3608
3609 The max-br parameter signals that the video decoder of the
3610 receiver is capable of decoding video at a higher bitrate than
3611 is required by the signaled highest level conveyed in the value
3612 of the profile-level-id parameter or the max-recv-level
3613 parameter.
3614
3615 When max-br is signaled, the video codec of the receiver MUST
3616 be able to decode NAL unit streams that conform to the signaled
3617 highest level, with the following exceptions in the limits
3618 specified by the highest level:
3619
3620 o The value of max-br (after taking cpbBrVclFactor and
3621 cpbBrNALFactor into consideration when needed) replaces the
3622 MaxBR value in Table A-1 of [<a href="#ref-H.264" title="&quot;Advanced video coding for generic audiovisual services&quot;">H.264</a>] for the highest level.
3623
3624 o When the max-cpb parameter is not present, the result of the
3625 following formula replaces the value of MaxCPB in Table A-1
3626 of [<a href="#ref-H.264" title="&quot;Advanced video coding for generic audiovisual services&quot;">H.264</a>]: (MaxCPB of the signaled level) * max-br / (MaxBR
3627 of the signaled highest level).
3628
3629 For example, if a receiver signals capability for Main profile
3630 Level 1.2 with max-br equal to 1550, this indicates a maximum
3631 video bitrate of 1550 kbits/sec for VCL HRD parameters, a
3632
3633
3634
3635
3636 <span class="grey">Wenger, et al. Standards Track [Page 65]</span></pre>
3637 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-66" ></span>
3638 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
3639
3640
3641 maximum video bitrate of 1860 kbits/sec for NAL HRD parameters,
3642 and a CPB size of 4036458 bits (1550000 / 384000 * 1000 *
3643 1000).
3644
3645 The value of max-br (after taking cpbBrVclFactor and
3646 cpbBrNALFactor into consideration when needed) MUST be greater
3647 than or equal to the value MaxBR given in Table A-1 of [<a href="#ref-H.264" title="&quot;Advanced video coding for generic audiovisual services&quot;">H.264</a>]
3648 for the signaled highest level.
3649
3650 Senders MAY use this knowledge to send higher-bitrate video as
3651 allowed in the level definition of SVC, to achieve improved
3652 video quality.
3653
3654 Informative note: This parameter was added primarily to
3655 complement a similar codepoint in the ITU-T Recommendation
3656 H.245, so as to facilitate signaling gateway designs. No
3657 assumption can be made from the value of this parameter that
3658 the network is capable of handling such bitrates at any
3659 given time. In particular, no conclusion can be drawn that
3660 the signaled bitrate is possible under congestion control
3661 constraints.
3662
3663 redundant-pic-cap:
3664 This parameter is as specified in [<a href="./rfc6184" title="&quot;RTP Payload Format for H.264 Video&quot;">RFC6184</a>].
3665
3666 sprop-parameter-sets:
3667 This parameter MAY be used to convey any sequence parameter
3668 set, subset sequence parameter set, and picture parameter set
3669 NAL units (herein referred to as the initial parameter set NAL
3670 units) that can be placed in the NAL unit stream to precede any
3671 other NAL units in decoding order and that are associated with
3672 the default level of profile-level-id. The parameter MUST NOT
3673 be used to indicate codec capability in any capability exchange
3674 procedure. The value of the parameter is a comma (',')
3675 separated list of base64 [<a href="./rfc4648" title="&quot;The Base16, Base32, and Base64 Data Encodings&quot;">RFC4648</a>] representations of the
3676 parameter set NAL units as specified in Sections <a href="#section-7.3.2.1">7.3.2.1</a>,
3677 7.3.2.2, and G.7.3.2.1 of [<a href="#ref-H.264" title="&quot;Advanced video coding for generic audiovisual services&quot;">H.264</a>]. Note that the number of
3678 bytes in a parameter set NAL unit is typically less than 10,
3679 but a picture parameter set NAL unit can contain several
3680 hundreds of bytes.
3681
3682 Informative note: When several payload types are offered in
3683 the SDP Offer/Answer model, each with its own sprop-
3684 parameter-sets parameter, then the receiver cannot assume
3685 that those parameter sets do not use conflicting storage
3686 locations (i.e., identical values of parameter set
3687
3688
3689
3690
3691
3692 <span class="grey">Wenger, et al. Standards Track [Page 66]</span></pre>
3693 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-67" ></span>
3694 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
3695
3696
3697 identifiers). Therefore, a receiver should buffer all
3698 sprop-parameter-sets and make them available to the decoder
3699 instance that decodes a certain payload type.
3700
3701 sprop-level-parameter-sets:
3702 This parameter MAY be used to convey any sequence, subset
3703 sequence, and picture parameter set NAL units (herein referred
3704 to as the initial parameter set NAL units) that can be placed
3705 in the NAL unit stream to precede any other NAL units in
3706 decoding order and that are associated with one or more levels
3707 different than the default level of profile-level-id. The
3708 parameter MUST NOT be used to indicate codec capability in any
3709 capability exchange procedure.
3710
3711 The sprop-level-parameter-sets parameter contains parameter
3712 sets for one or more levels that are different than the default
3713 level. All parameter sets targeted for use when one level of
3714 the default sub-profile is accepted by a receiver are clustered
3715 and prefixed with a three-byte field that has the same syntax
3716 as profile-level-id. This enables the receiver to install the
3717 parameter sets for the accepted level and discard the rest.
3718 The three-byte field is named PLId, and all parameter sets
3719 associated with one level are named PSL, which has the same
3720 syntax as sprop-parameter-sets. Parameter sets for each level
3721 are represented in the form of PLId:PSL, i.e., PLId followed by
3722 a colon (':') and the base64 [<a href="./rfc4648" title="&quot;The Base16, Base32, and Base64 Data Encodings&quot;">RFC4648</a>] representation of the
3723 initial parameter set NAL units for the level. Each pair of
3724 PLId:PSL is also separated by a colon. Note that a PSL can
3725 contain multiple parameter sets for that level, separated with
3726 commas (',').
3727
3728 The subset of coding tools indicated by each PLId field MUST be
3729 equal to the default sub-profile, and the level indicated by
3730 each PLId field MUST be different than the default level.
3731
3732 Informative note: This parameter allows for efficient level
3733 downgrade or upgrade in SDP Offer/Answer and out-of-band
3734 transport of parameter sets, simultaneously.
3735
3736 in-band-parameter-sets:
3737 This parameter MAY be used to indicate a receiver capability.
3738 The value MAY be equal to either 0 or 1. The value 1 indicates
3739 that the receiver discards out-of-band parameter sets in sprop-
3740 parameter-sets and sprop-level-parameter-sets, therefore the
3741 sender MUST transmit all parameter sets in-band. The value 0
3742 indicates that the receiver utilizes out-of-band parameter sets
3743 included in sprop-parameter-sets and/or sprop-level-parameter-
3744 sets. However, in this case, the sender MAY still choose to
3745
3746
3747
3748 <span class="grey">Wenger, et al. Standards Track [Page 67]</span></pre>
3749 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-68" ></span>
3750 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
3751
3752
3753 send parameter sets in-band. When the parameter is not
3754 present, this receiver capability is not specified, and
3755 therefore the sender MAY send out-of-band parameter sets only,
3756 or it MAY send in-band-parameter-sets only, or it MAY send
3757 both.
3758
3759 packetization-mode:
3760 This parameter is as specified in [<a href="./rfc6184" title="&quot;RTP Payload Format for H.264 Video&quot;">RFC6184</a>]. When the mst-mode
3761 parameter is present, the value of this parameter is
3762 additionally constrained as follows. If mst-mode is equal to
3763 "NI-T", "NI-C", or "NI-TC", packetization-mode MUST NOT be
3764 equal to 2. Otherwise, (mst-mode is equal to "I-C"),
3765 packetization-mode MUST be equal to 2.
3766
3767 sprop-interleaving-depth:
3768 This parameter is as specified in [<a href="./rfc6184" title="&quot;RTP Payload Format for H.264 Video&quot;">RFC6184</a>].
3769
3770 sprop-deint-buf-req:
3771 This parameter is as specified in [<a href="./rfc6184" title="&quot;RTP Payload Format for H.264 Video&quot;">RFC6184</a>].
3772
3773 deint-buf-cap:
3774 This parameter is as specified in [<a href="./rfc6184" title="&quot;RTP Payload Format for H.264 Video&quot;">RFC6184</a>].
3775
3776 sprop-init-buf-time:
3777 This parameter is as specified in [<a href="./rfc6184" title="&quot;RTP Payload Format for H.264 Video&quot;">RFC6184</a>].
3778
3779 sprop-max-don-diff:
3780 This parameter is as specified in [<a href="./rfc6184" title="&quot;RTP Payload Format for H.264 Video&quot;">RFC6184</a>].
3781
3782 max-rcmd-nalu-size:
3783 This parameter is as specified in [<a href="./rfc6184" title="&quot;RTP Payload Format for H.264 Video&quot;">RFC6184</a>].
3784
3785 mst-mode:
3786 This parameter MAY be used to signal the properties of a NAL
3787 unit stream or the capabilities of a receiver implementation.
3788 If this parameter is present, multi-session transmission MUST
3789 be used. Otherwise (this parameter is not present), single-
3790 session transmission MUST be used. When this parameter is
3791 present, the following applies. When the value of mst-mode is
3792 equal to "NI-T", the NI-T mode MUST be used. When the value of
3793 mst-mode is equal to "NI-C", the NI-C mode MUST be used. When
3794 the value of mst-mode is equal to "NI-TC", the NI-TC mode MUST
3795 be used. When the value of mst-mode is equal to "I-C", the I-C
3796 mode MUST be used. The value of mst-mode MUST have one of the
3797 following tokens: "NI-T", "NI-C", "NI-TC", or "I-C".
3798
3799 All RTP sessions in an MST MUST have the same value of mst-
3800 mode.
3801
3802
3803
3804 <span class="grey">Wenger, et al. Standards Track [Page 68]</span></pre>
3805 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-69" ></span>
3806 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
3807
3808
3809 sprop-mst-csdon-always-present:
3810 This parameter MUST NOT be present when mst-mode is not present
3811 or the value of mst-mode is equal to "NI-T" or "I-C". This
3812 parameter signals the properties of the NAL unit stream. When
3813 sprop-mst-csdon-always-present is present and the value is
3814 equal to 1, packetization-mode MUST be equal to 1, and all the
3815 RTP packets carrying the NAL unit stream MUST be STAP-A packets
3816 containing a PACSI NAL unit that further contains the DONC
3817 field or NI-MTAP packets with the J field equal to 1. When
3818 sprop-mst-csdon-always-present is present and the value is
3819 equal to 1, the CS-DON value of any particular NAL unit can be
3820 derived solely according to information in the packet
3821 containing the NAL unit.
3822
3823 When sprop-mst-csdon-always-present is present in the current
3824 RTP session, it MUST be present also in all the RTP sessions
3825 the current RTP session depends on and the value of sprop-mst-
3826 csdon-always-present is identical for the current RTP session
3827 and all the RTP sessions on which the current RTP session
3828 depends.
3829
3830 sprop-mst-remux-buf-size:
3831 This parameter MUST NOT be present when mst-mode is not present
3832 or the value of mst-mode is equal to "NI-T". This parameter
3833 MUST be present when mst-mode is present and the value of mst-
3834 mode is equal to "NI-C", "NI-TC", or "I-C".
3835
3836 This parameter signals the properties of the NAL unit stream.
3837 It MUST be set to a value one less than the minimum re-
3838 multiplexing buffer size (in NAL units), so that it is
3839 guaranteed that receivers can reconstruct NAL unit decoding
3840 order as specified in Sub<a href="#section-6.2.2">section 6.2.2</a>.
3841
3842 The value of sprop-mst-remux-buf-size MUST be an integer in the
3843 range of 0 to 32767, inclusive.
3844
3845 sprop-remux-buf-req:
3846 This parameter MUST NOT be present when mst-mode is not present
3847 or the value of mst-mode is equal to "NI-T". It MUST be
3848 present when mst-mode is present and the value of mst-mode is
3849 equal to "NI-C", "NI-TC", or "I-C".
3850
3851 sprop-remux-buf-req signals the required size of the re-
3852 multiplexing buffer for the NAL unit stream. It is guaranteed
3853 that receivers can recover the decoding order of the received
3854 NAL units from the current RTP session and the RTP sessions the
3855
3856
3857
3858
3859
3860 <span class="grey">Wenger, et al. Standards Track [Page 69]</span></pre>
3861 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-70" ></span>
3862 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
3863
3864
3865 current RTP session depends on as specified in <a href="#section-6.2.2">Section 6.2.2</a>,
3866 when the re-multiplexing buffer size is of at least the value
3867 of sprop-remux-buf-req in units of bytes.
3868
3869 The value of sprop-remux-buf-req MUST be an integer in the
3870 range of 0 to 4294967295, inclusive.
3871
3872 remux-buf-cap:
3873 This parameter MUST NOT be present when mst-mode is not present
3874 or the value of mst-mode is equal to "NI-T". This parameter
3875 MAY be used to signal the capabilities of a receiver
3876 implementation and indicates the amount of re-multiplexing
3877 buffer space in units of bytes that the receiver has available
3878 for recovering the NAL unit decoding order as specified in
3879 <a href="#section-6.2.2">Section 6.2.2</a>. A receiver is able to handle any NAL unit
3880 stream for which the value of the sprop-remux-buf-req parameter
3881 is smaller than or equal to this parameter.
3882
3883 If the parameter is not present, then a value of 0 MUST be used
3884 for remux-buf-cap. The value of remux-buf-cap MUST be an
3885 integer in the range of 0 to 4294967295, inclusive.
3886
3887 sprop-remux-init-buf-time:
3888 This parameter MAY be used to signal the properties of the NAL
3889 unit stream. The parameter MUST NOT be present if mst-mode is
3890 not present or the value of mst-mode is equal to "NI-T".
3891
3892 The parameter signals the initial buffering time that a
3893 receiver MUST wait before starting to recover the NAL unit
3894 decoding order as specified in <a href="#section-6.2.2">Section 6.2.2</a> of this memo.
3895
3896 The parameter is coded as a non-negative base10 integer
3897 representation in clock ticks of a 90-kHz clock. If the
3898 parameter is not present, then no initial buffering time value
3899 is defined. Otherwise, the value of sprop-remux-init-buf-time
3900 MUST be an integer in the range of 0 to 4294967295, inclusive.
3901
3902 sprop-mst-max-don-diff:
3903 This parameter MAY be used to signal the properties of the NAL
3904 unit stream. It MUST NOT be used to signal transmitter or
3905 receiver or codec capabilities. The parameter MUST NOT be
3906 present if mst-mode is not present or the value of mst-mode is
3907 equal to "NI-T". sprop-mst-max-don-diff is an integer in the
3908 range of 0 to 32767, inclusive. If sprop-mst-max-don-diff is
3909 not present, the value of the parameter is unspecified. sprop-
3910 mst-max-don-diff is calculated same as sprop-max-don-diff as
3911 specified in [<a href="./rfc6184" title="&quot;RTP Payload Format for H.264 Video&quot;">RFC6184</a>], with decoding order number being
3912 replaced by cross-session decoding order number.
3913
3914
3915
3916 <span class="grey">Wenger, et al. Standards Track [Page 70]</span></pre>
3917 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-71" ></span>
3918 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
3919
3920
3921 sprop-scalability-info:
3922 This parameter MAY be used to convey the NAL unit containing
3923 the scalability information SEI message as specified in Annex G
3924 of [<a href="#ref-H.264" title="&quot;Advanced video coding for generic audiovisual services&quot;">H.264</a>]. This parameter MAY be used to signal the contained
3925 layers of an SVC bitstream. The parameter MUST NOT be used to
3926 indicate codec capability in any capability exchange procedure.
3927 The value of the parameter is the base64 [<a href="./rfc4648" title="&quot;The Base16, Base32, and Base64 Data Encodings&quot;">RFC4648</a>]
3928 representation of the NAL unit containing the scalability
3929 information SEI message. If present, the NAL unit MUST contain
3930 only one SEI message that is a scalability information SEI
3931 message.
3932
3933 This parameter MAY be used in an offering or declarative SDP
3934 message to indicate what layers (operation points) can be
3935 provided. A receiver MAY indicate its choice of one layer
3936 using the optional media type parameter scalable-layer-id.
3937
3938 scalable-layer-id:
3939 This parameter MAY be used to signal a receiver's choice of the
3940 offers or declared operation points or layers using sprop-
3941 scalability-info or sprop-operation-point-info. The value of
3942 scalable-layer-id is a base16 representation of the layer_id[ i
3943 ] syntax element in the scalability information SEI message as
3944 specified in Annex G of [<a href="#ref-H.264" title="&quot;Advanced video coding for generic audiovisual services&quot;">H.264</a>] or layer-ID contained in sprop-
3945 operation-point-info.
3946
3947 sprop-operation-point-info:
3948 This parameter MAY be used to describe the operation points of
3949 an RTP session. The value of this parameter consists of a
3950 comma-separated list of operation-point-description vectors.
3951 The values given by the operation-point-description vectors are
3952 the same as, or are derived from, the values that would be
3953 given for a scalable layer in the scalability information SEI
3954 message as specified in Annex G of [<a href="#ref-H.264" title="&quot;Advanced video coding for generic audiovisual services&quot;">H.264</a>], where the term
3955 scalable layer in the scalability information SEI message
3956 refers to all NAL units associated with the same values of
3957 temporal_id, dependency_id, and quality_id. In this memo, such
3958 a set of NAL units is called an operation point.
3959
3960 Each operation-point-description vector has ten elements,
3961 provided as a comma-separated list of values as defined below.
3962 The first value of the operation-point-description vector is
3963 preceded by a '&lt;', and the last value of the operation-point-
3964 description vector is followed by a '&gt;'. If the sprop-
3965 operation-point-info is followed by exactly one operation-
3966 point-description vector, this describes the highest operation
3967 point contained in the RTP session. If there are two or more
3968
3969
3970
3971
3972 <span class="grey">Wenger, et al. Standards Track [Page 71]</span></pre>
3973 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-72" ></span>
3974 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
3975
3976
3977 operation-point-description vectors, the first describes the
3978 lowest and the last describes the highest operation point
3979 contained in the RTP session.
3980
3981 The values given by the operation-point-description vector are
3982 as follows, in the order listed:
3983
3984 - layer-ID: This value specifies the layer identifier of the
3985 operation point, which is identical to the layer_id that
3986 would be indicated (for the same values of dependency_id,
3987 quality_id, and temporal_id) in the scalability information
3988 SEI message. This field MAY be empty, indicating that the
3989 value is unspecified. When there are multiple operation-
3990 point-description vectors with layer-ID, the values of
3991 layer-ID do not need to be consecutive.
3992
3993 - temporal-ID: This value specifies the temporal_id of the
3994 operation point. This field MUST NOT be empty.
3995
3996 - dependency-ID: This values specifies the dependency_id of
3997 the operation point. This field MUST NOT be empty.
3998
3999 - quality-ID: This values specifies the quality_id of the
4000 operation point. This field MUST NOT be empty.
4001
4002 - profile-level-ID: This value specifies the profile-level-idc
4003 of the operation point in the base16 format. The default
4004 sub-profile or default level indicated by the parameter
4005 profile-level-ID in the sprop-operation-point-info vector
4006 SHALL be equal to or lower than the default sub-profile or
4007 default level indicated by profile-level-id, which may be
4008 either present or the default value is taken. This field
4009 MAY be empty, indicating that the value is unspecified.
4010
4011 - avg-framerate: This value specifies the average frame rate
4012 of the operation point. This value is given as an integer
4013 in frames per 256 seconds. The field MAY be empty,
4014 indicating that the value is unspecified.
4015
4016 - width: This value specifies the width dimension in pixels of
4017 decoded frames for the operation point. This parameter is
4018 not directly given in the scalability information SEI
4019 message. This field MAY be empty, indicating that the value
4020 is unspecified.
4021
4022
4023
4024
4025
4026
4027
4028 <span class="grey">Wenger, et al. Standards Track [Page 72]</span></pre>
4029 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-73" ></span>
4030 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
4031
4032
4033 - height: This value gives the height dimension in pixels of
4034 decoded frames for the operation point. This parameter is
4035 not directly given in the scalability information SEI. This
4036 field MAY be empty, indicating that the value is
4037 unspecified.
4038
4039 - avg-bitrate: This value specifies the average bitrate of the
4040 operation point. This parameter is given as an integer in
4041 kbits per second over the entire stream. Note that this
4042 parameter is provided in the scalability information SEI
4043 message in bits per second and calculated over a variable
4044 time window. This field MAY be empty, indicating that the
4045 value is unspecified.
4046
4047 - max-bitrate: This value specifies the maximum bitrate of the
4048 operation point. This parameter is given as an integer in
4049 kbits per second and describes the maximum bitrate per each
4050 one-second window. Note that this parameter is provided in
4051 the scalability information SEI message in bits per second
4052 and is calculated over a variable time window. This field
4053 MAY be empty, indicating that the value is unspecified.
4054
4055 Similarly to sprop-scalability-info, this parameter MAY be
4056 used in an offering or declarative SDP message to indicate
4057 what layers (operation points) can be provided. A receiver
4058 MAY indicate its choice of the highest layer it wants to
4059 send and/or receive using the optional media type parameter
4060 scalable-layer-id.
4061
4062 sprop-no-NAL-reordering-required:
4063 This parameter MAY be used to signal the properties of the NAL
4064 unit stream. This parameter MUST NOT be present when mst-mode
4065 is not present or the value of mst-mode is not equal to "NI-T".
4066 The presence of this parameter indicates that no reordering of
4067 non-VCL or VCL NAL units is required for the decoding order
4068 recovery process.
4069
4070 sprop-avc-ready:
4071 This parameter MAY be used to indicate the properties of the
4072 NAL unit stream. The presence of this parameter indicates that
4073 the RTP session, if used in SST, or used in MST combined with
4074 other RTP sessions also with this parameter present, can be
4075 processed by a [<a href="./rfc6184" title="&quot;RTP Payload Format for H.264 Video&quot;">RFC6184</a>] receiver. This parameter MAY be used
4076 with RTP sessions with media subtype H264-SVC.
4077
4078 Encoding considerations:
4079 This media type is framed and binary; see Section 4.8 of <a href="./rfc4288">RFC</a>
4080 <a href="./rfc4288">4288</a> [<a href="./rfc4288" title="&quot;Media Type Specifications and Registration Procedures&quot;">RFC4288</a>].
4081
4082
4083
4084 <span class="grey">Wenger, et al. Standards Track [Page 73]</span></pre>
4085 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-74" ></span>
4086 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
4087
4088
4089 Security considerations:
4090 See <a href="./rfc6190#section-8">Section&nbsp;8 of RFC 6190</a>.
4091
4092 Published specification:
4093 Please refer to <a href="./rfc6190">RFC 6190</a> and its <a href="#section-13">Section 13</a>.
4094
4095 Additional information:
4096 none
4097
4098 File extensions: none
4099
4100 Macintosh file type code: none
4101
4102 Object identifier or OID: none
4103
4104 Person &amp; email address to contact for further information:
4105
4106 Ye-Kui Wang, [email protected]
4107
4108 Intended usage: COMMON
4109
4110 Restrictions on usage:
4111 This media type depends on RTP framing, and hence is only
4112 defined for transfer via RTP [<a href="./rfc3550" title="&quot;RTP: A Transport Protocol for Real-Time Applications&quot;">RFC3550</a>]. Transport within other
4113 framing protocols is not defined at this time.
4114
4115 Interoperability considerations:
4116 The media subtype name contains "SVC" to avoid potential
4117 conflict with <a href="./rfc3984">RFC 3984</a> and its potential future replacement RTP
4118 payload format for H.264 non-SVC profiles.
4119
4120 Applications that use this media type:
4121 Real-time video applications like video streaming, video
4122 telephony, and video conferencing.
4123
4124 Author:
4125
4126 Ye-Kui Wang, [email protected]
4127
4128 Change controller:
4129 IETF Audio/Video Transport working group delegated from the
4130 IESG.
4131
4132
4133
4134
4135
4136
4137
4138
4139
4140 <span class="grey">Wenger, et al. Standards Track [Page 74]</span></pre>
4141 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-75" ></span>
4142 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
4143
4144
4145 <span class="h3"><a class="selflink" id="section-7.2" href="#section-7.2">7.2</a>. SDP Parameters</span>
4146
4147 <span class="h4"><a class="selflink" id="section-7.2.1" href="#section-7.2.1">7.2.1</a>. Mapping of Payload Type Parameters to SDP</span>
4148
4149 The media type video/H264-SVC string is mapped to fields in the
4150 Session Description Protocol (SDP) as follows:
4151
4152 o The media name in the "m=" line of SDP MUST be video.
4153
4154 o The encoding name in the "a=rtpmap" line of SDP MUST be H264-SVC
4155 (the media subtype).
4156
4157 o The clock rate in the "a=rtpmap" line MUST be 90000.
4158
4159 o The OPTIONAL parameters profile-level-id, max-recv-level, max-
4160 recv-base-level, max-mbps, max-fs, max-cpb, max-dpb, max-br,
4161 redundant-pic-cap, in-band-parameter-sets, packetization-mode,
4162 sprop-interleaving-depth, deint-buf-cap, sprop-deint-buf-req,
4163 sprop-init-buf-time, sprop-max-don-diff, max-rcmd-nalu-size, mst-
4164 mode, sprop-mst-csdon-always-present, sprop-mst-remux-buf-size,
4165 sprop-remux-buf-req, remux-buf-cap, sprop-remux-init-buf-time,
4166 sprop-mst-max-don-diff, and scalable-layer-id, when present, MUST
4167 be included in the "a=fmtp" line of SDP. These parameters are
4168 expressed as a media type string, in the form of a semicolon-
4169 separated list of parameter=value pairs.
4170
4171 o The OPTIONAL parameters sprop-parameter-sets, sprop-level-
4172 parameter-sets, sprop-scalability-info, sprop-operation-point-
4173 info, sprop-no-NAL-reordering-required, and sprop-avc-ready, when
4174 present, MUST be included in the "a=fmtp" line of SDP or conveyed
4175 using the "fmtp" source attribute as specified in <a href="./rfc5576#section-6.3">Section&nbsp;6.3 of
4176 [RFC5576]</a>. For a particular media format (i.e., RTP payload
4177 type), a sprop-parameter-sets or sprop-level-parameter-sets MUST
4178 NOT be both included in the "a=fmtp" line of SDP and conveyed
4179 using the "fmtp" source attribute. When included in the "a=fmtp"
4180 line of SDP, these parameters are expressed as a media type
4181 string, in the form of a semicolon-separated list of
4182 parameter=value pairs. When conveyed using the "fmtp" source
4183 attribute, these parameters are only associated with the given
4184 source and payload type as parts of the "fmtp" source attribute.
4185
4186 Informative note: Conveyance of sprop-parameter-sets and
4187 sprop-level-parameter-sets using the "fmtp" source attribute
4188 allows for out-of-band transport of parameter sets in
4189 topologies like Topo-Video-switch-MCU [<a href="./rfc5117" title="&quot;RTP Topologies&quot;">RFC5117</a>].
4190
4191
4192
4193
4194
4195
4196 <span class="grey">Wenger, et al. Standards Track [Page 75]</span></pre>
4197 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-76" ></span>
4198 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
4199
4200
4201 <span class="h4"><a class="selflink" id="section-7.2.2" href="#section-7.2.2">7.2.2</a>. Usage with the SDP Offer/Answer Model</span>
4202
4203 When an SVC stream (with media subtype H264-SVC) is offered over RTP
4204 using SDP in an Offer/Answer model [<a href="./rfc3264" title="&quot;An Offer/Answer Model with Session Description Protocol (SDP)&quot;">RFC3264</a>] for negotiation for
4205 unicast usage, the following limitations and rules apply:
4206
4207 o The parameters identifying a media format configuration for SVC
4208 are profile-level-id, packetization-mode, and mst-mode. These
4209 media configuration parameters (except for the level part of
4210 profile-level-id) MUST be used symmetrically when the answerer
4211 does not include scalable-layer-id in the answer; i.e., the
4212 answerer MUST either maintain all configuration parameters or
4213 remove the media format (payload type) completely, if one or more
4214 of the parameter values are not supported. Note that the level
4215 part of profile-level-id includes level_idc, and, for indication
4216 of level 1b when profile_idc is equal to 66, 77, or 88, bit 4
4217 (constraint_set3_flag) of profile-iop. The level part of profile-
4218 level-id is changeable.
4219
4220 Informative note: The requirement for symmetric use does not
4221 apply for the level part of profile-level-id, and does not
4222 apply for the other stream properties and capability
4223 parameters.
4224
4225 Informative note: In [<a href="#ref-H.264" title="&quot;Advanced video coding for generic audiovisual services&quot;">H.264</a>], all the levels except for Level
4226 1b are equal to the value of level_idc divided by 10. Level 1b
4227 is a level higher than Level 1.0 but lower than Level 1.1, and
4228 is signaled in an ad hoc manner. For the Baseline, Main, and
4229 Extended profiles (with profile_idc equal to 66, 77, and 88,
4230 respectively), Level 1b is indicated by level_idc equal to 11
4231 (i.e., the same as level 1.1) and constraint_set3_flag equal to
4232 1. For other profiles, Level 1b is indicated by level_idc
4233 equal to 9 (but note that Level 1b for these profiles is still
4234 higher than Level 1, which has level_idc equal to 10, and lower
4235 than Level 1.1). In SDP Offer/Answer, an answer may indicate a
4236 level equal to or lower than the level indicated in the offer.
4237 Due to the ad hoc indication of Level 1b, offerers and
4238 answerers must check the value of bit 4 (constraint_set3_flag)
4239 of the middle octet of the parameter profile-level-id, when
4240 profile_idc is equal to 66, 77, or 88 and level_idc is equal to
4241 11.
4242
4243 To simplify handling and matching of these configurations, the
4244 same RTP payload type number used in the offer should also be used
4245 in the answer, as specified in [<a href="./rfc3264" title="&quot;An Offer/Answer Model with Session Description Protocol (SDP)&quot;">RFC3264</a>]. The same RTP payload
4246 type number used in the offer MUST also be used in the answer when
4247 the answer includes scalable-layer-id. When the answer does not
4248 include scalable-layer-id, the answer MUST NOT contain a payload
4249
4250
4251
4252 <span class="grey">Wenger, et al. Standards Track [Page 76]</span></pre>
4253 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-77" ></span>
4254 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
4255
4256
4257 type number used in the offer unless the configuration is exactly
4258 the same as in the offer or the configuration in the answer only
4259 differs from that in the offer with a level lower than the default
4260 level offered.
4261
4262 Informative note: When an offerer receives an answer that does
4263 not include scalable-layer-id it has to compare payload types
4264 not declared in the offer based on the media type (i.e.,
4265 video/H264-SVC) and the above media configuration parameters
4266 with any payload types it has already declared. This will
4267 enable it to determine whether the configuration in question is
4268 new or if it is equivalent to configuration already offered,
4269 since a different payload type number may be used in the
4270 answer.
4271
4272 Since an SVC stream may contain multiple operation points, a
4273 facility is provided so that an answerer can select a different
4274 operation point than the entire SVC stream. Specifically,
4275 different operation points MAY be described using the sprop-
4276 scalability-info or sprop-operation-point-info parameters. The
4277 first one carries the entire scalability information SEI message
4278 defined in Annex G of [<a href="#ref-H.264" title="&quot;Advanced video coding for generic audiovisual services&quot;">H.264</a>], whereas the second one may be
4279 derived, e.g., as a subset of this SEI message that only contains
4280 key information about an operation point. Operation points, in
4281 both cases, are associated with a layer identifier.
4282
4283 If such information (sprop-operation-point-info or sprop-
4284 scalability-info) is provided in an offer, an answerer MAY select
4285 from the various operation points offered in the sprop-
4286 scalability-information or sprop-operation-point-info parameters
4287 by including scalable-layer-id in the answer. By this, the
4288 answerer indicates its selection of a particular operation point
4289 in the received and/or in the sent stream. When such operation
4290 point selection takes place, i.e., the answerer includes scalable-
4291 layer-id in the answer, the media configuration parameters MUST
4292 NOT be present in the answer. Rather, the media configuration
4293 that the answerer will use for receiving and/or sending is the one
4294 used for the selected operation point as indicated in the offer.
4295
4296 Informative note: The ability to perform operation point
4297 selection enables a receiver to utilize the scalable nature of
4298 an SVC stream.
4299
4300 o The parameter max-recv-level, when present, declares the highest
4301 level supported for receiving. In case max-recv-level is not
4302 present, the highest level supported for receiving is equal to the
4303
4304
4305
4306
4307
4308 <span class="grey">Wenger, et al. Standards Track [Page 77]</span></pre>
4309 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-78" ></span>
4310 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
4311
4312
4313 default level indicated by the level part of profile-level-id.
4314 max-recv-level, when present, MUST be higher than the default
4315 level.
4316
4317 o The parameter max-recv-base-level, when present, declares the
4318 highest level of the base layer supported for receiving. When
4319 max-recv-base-level is not present, the highest level supported
4320 for the base layer is not constrained separately from the SVC
4321 stream containing the base layer. The endpoint at the other side
4322 MUST NOT send a scalable stream for which the base layer is of a
4323 level higher than max-recv-base-level. Parameters declaring
4324 receiver capabilities above the default level (max-mbps, max-
4325 smbps, max-fs, max-cpb, max-dpb, max-br, and max-recv-level) do
4326 not apply to the base layer when max-recv-base-level is present.
4327
4328 o The parameters sprop-deint-buf-req, sprop-interleaving-depth,
4329 sprop-max-don-diff, sprop-init-buf-time, sprop-mst-csdon-always-
4330 present, sprop-remux-buf-req, sprop-mst-remux-buf-size, sprop-
4331 remux-init-buf-time, sprop-mst-max-don-diff, sprop-scalability-
4332 information, sprop-operation-point-info, sprop-no-NAL-reordering-
4333 required, and sprop-avc-ready describe the properties of the NAL
4334 unit stream that the offerer or answerer is sending for the media
4335 format configuration. This differs from the normal usage of the
4336 Offer/Answer parameters: normally such parameters declare the
4337 properties of the stream that the offerer or the answerer is able
4338 to receive. When dealing with SVC, the offerer assumes that the
4339 answerer will be able to receive media encoded using the
4340 configuration being offered.
4341
4342 Informative note: The above parameters apply for any stream
4343 sent by the declaring entity with the same configuration; i.e.,
4344 they are dependent on their source. Rather than being bound to
4345 the payload type, the values may have to be applied to another
4346 payload type when being sent, as they apply for the
4347 configuration.
4348
4349 o The capability parameters max-mbps, max-fs, max-cpb, max-dpb, max-
4350 br, redundant-pic-cap, and max-rcmd-nalu-size MAY be used to
4351 declare further capabilities of the offerer or answerer for
4352 receiving. These parameters MUST NOT be present when the
4353 direction attribute is sendonly, and the parameters describe the
4354 limitations of what the offerer or answerer accepts for receiving
4355 streams.
4356
4357 o When mst-mode is not present and packetization-mode is equal to 2,
4358 the following applies.
4359
4360
4361
4362
4363
4364 <span class="grey">Wenger, et al. Standards Track [Page 78]</span></pre>
4365 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-79" ></span>
4366 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
4367
4368
4369 o An offerer has to include the size of the de-interleaving
4370 buffer, sprop-deint-buf-req, in the offer. To enable the
4371 offerer and answerer to inform each other about their
4372 capabilities for de-interleaving buffering, both parties are
4373 RECOMMENDED to include deint-buf-cap. It is also RECOMMENDED
4374 to consider offering multiple payload types with different
4375 buffering requirements when the capabilities of the receiver
4376 are unknown.
4377
4378 o When mst-mode is present and equal to "NI-C", "NI-TC", or "I-C",
4379 the following applies.
4380
4381 o An offerer has to include sprop-remux-buf-req in the offer. To
4382 enable the offerer and answerer to inform each other about
4383 their capabilities for re-multiplexing buffering, both parties
4384 are RECOMMENDED to include remux-buf-cap. It is also
4385 RECOMMENDED to consider offering multiple payload types with
4386 different buffering requirements when the capabilities of the
4387 receiver are unknown.
4388
4389 o The sprop-parameter-sets or sprop-level-parameter-sets parameter,
4390 when present (included in the "a=fmtp" line of SDP or conveyed
4391 using the "fmtp" source attribute as specified in <a href="./rfc5576#section-6.3">Section&nbsp;6.3 of
4392 [RFC5576]</a>), is used for out-of-band transport of parameter sets.
4393 However, when out-of-band transport of parameter sets is used,
4394 parameter sets MAY still be additionally transported in-band.
4395
4396 The answerer MAY use either out-of-band or in-band transport of
4397 parameter sets for the stream it is sending, regardless of whether
4398 out-of-band parameter sets transport has been used in the offerer-
4399 to-answerer direction. Parameter sets included in an answer are
4400 independent of those parameter sets included in the offer, as they
4401 are used for decoding two different video streams, one from the
4402 answerer to the offerer, and the other in the opposite direction.
4403
4404 The following rules apply to transport of parameter sets in the
4405 offerer-to-answerer direction.
4406
4407 o An offer MAY include either or both of sprop-parameter- sets
4408 and sprop-level-parameter-sets. If neither sprop-parameter-
4409 sets nor sprop-level-parameter-sets is present in the offer,
4410 then only in-band transport of parameter sets is used.
4411
4412 o If the answer includes in-band-parameter-sets equal to 1, then
4413 the offerer MUST transmit parameter sets in-band. Otherwise,
4414 the following applies.
4415
4416
4417
4418
4419
4420 <span class="grey">Wenger, et al. Standards Track [Page 79]</span></pre>
4421 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-80" ></span>
4422 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
4423
4424
4425 o If the level to use in the offerer-to-answerer direction is
4426 equal to the default level in the offer, the following
4427 applies.
4428
4429 The answerer MUST be prepared to use the parameter sets
4430 included in sprop-parameter-sets, when present, for
4431 decoding the incoming NAL unit stream, and ignore sprop-
4432 level-parameter-sets, when present.
4433
4434 When sprop-parameter-sets is not present in the offer,
4435 in-band transport of parameter sets MUST be used.
4436
4437 o Otherwise (the level to use in the offerer-to-answerer
4438 direction is not equal to the default level in the offer),
4439 the following applies.
4440
4441 The answerer MUST be prepared to use the parameter sets
4442 that are included in sprop-level-parameter-sets for the
4443 accepted level (i.e., the default level in the answer,
4444 which is also the level to use in the offerer-to-answerer
4445 direction), when present, for decoding the incoming NAL
4446 unit stream, and ignore all other parameter sets included
4447 in sprop-level-parameter-sets and sprop-parameter-sets,
4448 when present.
4449
4450 When no parameter sets for the accepted level are present
4451 in the sprop-level-parameter-sets, in-band transport of
4452 parameter sets MUST be used.
4453
4454 The following rules apply to transport of parameter sets in the
4455 answerer-to-offerer direction.
4456
4457 o An answer MAY include either sprop-parameter-sets or sprop-
4458 level-parameter-sets, but MUST NOT include both of the two. If
4459 neither sprop-parameter-sets nor sprop-level-parameter-sets is
4460 present in the answer, then only in-band transport of parameter
4461 sets is used.
4462
4463 o If the offer includes in-band-parameter-sets equal to 1, then
4464 the answerer MUST NOT include sprop-parameter-sets or sprop-
4465 level-parameter-sets in the answer and MUST transmit parameter
4466 sets in-band. Otherwise, the following applies.
4467
4468 o If the level to use in the answerer-to-offerer direction is
4469 equal to the default level in the answer, the following
4470 applies.
4471
4472
4473
4474
4475
4476 <span class="grey">Wenger, et al. Standards Track [Page 80]</span></pre>
4477 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-81" ></span>
4478 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
4479
4480
4481 The offerer MUST be prepared to use the parameter sets
4482 included in sprop-parameter-sets, when present, for
4483 decoding the incoming NAL unit stream, and ignore sprop-
4484 level-parameter-sets, when present.
4485
4486 When sprop-parameter-sets is not present in the answer,
4487 the answerer MUST transmit parameter sets in-band.
4488
4489 o Otherwise (the level to use in the answerer-to-offerer
4490 direction is not equal to the default level in the answer),
4491 the following applies.
4492
4493 The offerer MUST be prepared to use the parameter sets
4494 that are included in sprop-level-parameter-sets for the
4495 level to use in the answerer-to-offerer direction, when
4496 present in the answer, for decoding the incoming NAL unit
4497 stream, and ignore all other parameter sets included in
4498 sprop-level-parameter-sets and sprop-parameter-sets, when
4499 present in the answer.
4500
4501 When no parameter sets for the level to use in the
4502 answerer-to-offerer direction are present in sprop-level-
4503 parameter-sets in the answer, the answerer MUST transmit
4504 parameter sets in-band.
4505
4506 When sprop-parameter-sets or sprop-level-parameter-sets is
4507 conveyed using the "fmtp" source attribute as specified in <a href="./rfc5576#section-6.3">Section</a>
4508 <a href="./rfc5576#section-6.3">6.3 of [RFC5576]</a>, the receiver of the parameters MUST store the
4509 parameter sets included in the sprop-parameter-sets or sprop-
4510 level-parameter-sets for the accepted level and associate them to
4511 the source given as a part of the "fmtp" source attribute.
4512 Parameter sets associated with one source MUST only be used to
4513 decode NAL units conveyed in RTP packets from the same source.
4514 When this mechanism is in use, SSRC collision detection and
4515 resolution MUST be performed as specified in [<a href="./rfc5576" title="&quot;Source-Specific Media Attributes in the Session Description Protocol (SDP)&quot;">RFC5576</a>].
4516
4517 Informative note: Conveyance of sprop-parameter-sets and sprop-
4518 level-parameter-sets using the "fmtp" source attribute may be
4519 used in topologies like Topo-Video-switch-MCU [<a href="./rfc5117" title="&quot;RTP Topologies&quot;">RFC5117</a>] to
4520 enable out-of-band transport of parameter sets.
4521
4522 For streams being delivered over multicast, the following rules
4523 apply:
4524
4525 o The media format configuration is identified by profile-level- id,
4526 including the level part, packetization-mode, and mst-mode. These
4527 media format configuration parameters (including the level part of
4528 profile-level-id) MUST be used symmetrically; i.e., the answerer
4529
4530
4531
4532 <span class="grey">Wenger, et al. Standards Track [Page 81]</span></pre>
4533 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-82" ></span>
4534 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
4535
4536
4537 MUST either maintain all configuration parameters or remove the
4538 media format (payload type) completely. Note that this implies
4539 that the level part of profile-level-id for Offer/Answer in
4540 multicast is not changeable.
4541
4542 To simplify handling and matching of these configurations, the
4543 same RTP payload type number used in the offer should also be used
4544 in the answer, as specified in [<a href="./rfc3264" title="&quot;An Offer/Answer Model with Session Description Protocol (SDP)&quot;">RFC3264</a>]. An answer MUST NOT
4545 contain a payload type number used in the offer unless the
4546 configuration is the same as in the offer.
4547
4548 o Parameter sets received MUST be associated with the originating
4549 source, and MUST be only used in decoding the incoming NAL unit
4550 stream from the same source.
4551
4552 o The rules for other parameters are the same as above for unicast
4553 as long as the above rules are obeyed.
4554
4555 Table 14 lists the interpretation of all the parameters that MUST be
4556 used for the various combinations of offer, answer, and direction
4557 attributes. Note that the two columns wherein the scalable-layer-id
4558 parameter is used only apply to answers, whereas the other columns
4559 apply to both offers and answers.
4560
4561 Table 14. Interpretation of parameters for various combinations of
4562 offers, answers, direction attributes, with and without scalable-
4563 layer-id. Columns that do not indicate offer or answer apply to
4564 both.
4565
4566
4567
4568
4569
4570
4571
4572
4573
4574
4575
4576
4577
4578
4579
4580
4581
4582
4583
4584
4585
4586
4587
4588 <span class="grey">Wenger, et al. Standards Track [Page 82]</span></pre>
4589 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-83" ></span>
4590 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
4591
4592
4593 sendonly --+
4594 answer: recvonly,scalable-layer-id --+ |
4595 recvonly w/o scalable-layer-id --+ | |
4596 answer: sendrecv, scalable-layer-id --+ | | |
4597 sendrecv w/o scalable-layer-id --+ | | | |
4598 | | | | |
4599 profile-level-id C X C X P
4600 max-recv-level R R R R -
4601 max-recv-base-level R R R R -
4602 packetization-mode C X C X P
4603 mst-mode C X C X P
4604 sprop-avc-ready P P - - P
4605 sprop-deint-buf-req P P - - P
4606 sprop-init-buf-time P P - - P
4607 sprop-interleaving-depth P P - - P
4608 sprop-max-don-diff P P - - P
4609 sprop-mst-csdon-always-present P P - - P
4610 sprop-mst-max-don-diff P P - - P
4611 sprop-mst-remux-buf-size P P - - P
4612 sprop-no-NAL-reordering-required P P - - P
4613 sprop-operation-point-info P P - - P
4614 sprop-remux-buf-req P P - - P
4615 sprop-remux-init-buf-time P P - - P
4616 sprop-scalability-info P P - - P
4617 deint-buf-cap R R R R -
4618 max-br R R R R -
4619 max-cpb R R R R -
4620 max-dpb R R R R -
4621 max-fs R R R R -
4622 max-mbps R R R R -
4623 max-rcmd-nalu-size R R R R -
4624 redundant-pic-cap R R R R -
4625 remux-buf-cap R R R R -
4626 in-band-parameter-sets R R R R -
4627 sprop-parameter-sets S S - - S
4628 sprop-level-parameter-sets S S - - S
4629 scalable-layer-id X O X O -
4630
4631 Legend:
4632
4633 C: configuration for sending and receiving streams
4634 P: properties of the stream to be sent
4635 R: receiver capabilities
4636 S: out-of-band parameter sets
4637 O: operation point selection
4638 X: MUST NOT be present
4639 -: not usable, when present SHOULD be ignored
4640
4641
4642
4643
4644 <span class="grey">Wenger, et al. Standards Track [Page 83]</span></pre>
4645 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-84" ></span>
4646 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
4647
4648
4649 Parameters used for declaring receiver capabilities are in general
4650 downgradable; i.e., they express the upper limit for a sender's
4651 possible behavior. Thus, a sender MAY select to set its encoder
4652 using only lower/lesser or equal values of these parameters.
4653
4654 Parameters declaring a configuration point are not changeable, with
4655 the exception of the level part of the profile-level-id parameter for
4656 unicast usage. This expresses values a receiver expects to be used
4657 and must be used verbatim on the sender side. If level downgrading
4658 (for profile-level-id) is used, an answerer MUST NOT include the
4659 scalable-layer-id parameter.
4660
4661 When a sender's capabilities are declared, and non-downgradable
4662 parameters are used in this declaration, then these parameters
4663 express a configuration that is acceptable for the sender to receive
4664 streams. In order to achieve high interoperability levels, it is
4665 often advisable to offer multiple alternative configurations, e.g.,
4666 for the packetization mode. It is impossible to offer multiple
4667 configurations in a single payload type. Thus, when multiple
4668 configuration offers are made, each offer requires its own RTP
4669 payload type associated with the offer.
4670
4671 A receiver SHOULD understand all media type parameters, even if it
4672 only supports a subset of the payload format's functionality. This
4673 ensures that a receiver is capable of understanding when an offer to
4674 receive media can be downgraded to what is supported by the receiver
4675 of the offer.
4676
4677 An answerer MAY extend the offer with additional media format
4678 configurations. However, to enable their usage, in most cases a
4679 second offer is required from the offerer to provide the stream
4680 property parameters that the media sender will use. This also has
4681 the effect that the offerer has to be able to receive this media
4682 format configuration, not only to send it.
4683
4684 If an offerer wishes to have non-symmetric capabilities between
4685 sending and receiving, the offerer can allow asymmetric levels via
4686 level-asymmetry-allowed equal to 1. Alternatively, the offerer can
4687 offer different RTP sessions, i.e., different media lines declared as
4688 "recvonly" and "sendonly", respectively. This may have further
4689 implications on the system, and may require additional external
4690 semantics to associate the two media lines.
4691
4692 <span class="h4"><a class="selflink" id="section-7.2.3" href="#section-7.2.3">7.2.3</a>. Dependency Signaling in Multi-Session Transmission</span>
4693
4694 If MST is used, the rules on signaling media decoding dependency in
4695 SDP as defined in [<a href="./rfc5583" title="&quot;Signaling Media Decoding Dependency in the Session Description Protocol (SDP)&quot;">RFC5583</a>] apply. The rules on "hierarchical or
4696 layered encoding" with multicast in <a href="./rfc4566#section-5.7">Section&nbsp;5.7 of [RFC4566]</a> do not
4697
4698
4699
4700 <span class="grey">Wenger, et al. Standards Track [Page 84]</span></pre>
4701 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-85" ></span>
4702 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
4703
4704
4705 apply, i.e., the notation for Connection Data "c=" SHALL NOT be used
4706 with more than one address. Additionally, the order of dependencies
4707 of the RTP sessions indicated by the "a=depend" attribute as defined
4708 in [<a href="./rfc5583" title="&quot;Signaling Media Decoding Dependency in the Session Description Protocol (SDP)&quot;">RFC5583</a>] MUST represent the decoding order of the VC) NAL units
4709 in an access unit, i.e., the order of session dependency is given
4710 from the base or the lowest enhancement RTP session (the most
4711 important) to the highest enhancement RTP session (the least
4712 important).
4713
4714 <span class="h4"><a class="selflink" id="section-7.2.4" href="#section-7.2.4">7.2.4</a>. Usage in Declarative Session Descriptions</span>
4715
4716 When SVC over RTP is offered with SDP in a declarative style, as in
4717 Real Time Streaming Protocol (RTSP) [<a href="./rfc2326" title="&quot;Real Time Streaming Protocol (RTSP)&quot;">RFC2326</a>] or Session Announcement
4718 Protocol (SAP) [<a href="./rfc2974" title="&quot;Session Announcement Protocol&quot;">RFC2974</a>], the following considerations are necessary.
4719
4720 o All parameters capable of indicating both stream properties and
4721 receiver capabilities are used to indicate only stream properties.
4722 For example, in this case, the parameter profile-level-id declares
4723 the values used by the stream, not the capabilities for receiving
4724 streams. This results in that the following interpretation of the
4725 parameters MUST be used:
4726
4727 Declaring actual configuration or stream properties:
4728
4729 - profile-level-id
4730 - packetization-mode
4731 - mst-mode
4732 - sprop-deint-buf-req
4733 - sprop-interleaving-depth
4734 - sprop-max-don-diff
4735 - sprop-init-buf-time
4736 - sprop-mst-csdon-always-present
4737 - sprop-mst-remux-buf-size
4738 - sprop-remux-buf-req
4739 - sprop-remux-init-buf-time
4740 - sprop-mst-max-don-diff
4741 - sprop-scalability-info
4742 - sprop-operation-point-info
4743 - sprop-no-NAL-reordering-required
4744 - sprop-avc-ready
4745
4746 Out-of-band transporting of parameter sets:
4747
4748 - sprop-parameter-sets
4749 - sprop-level-parameter-sets
4750
4751
4752
4753
4754
4755
4756 <span class="grey">Wenger, et al. Standards Track [Page 85]</span></pre>
4757 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-86" ></span>
4758 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
4759
4760
4761 Not usable (when present, they SHOULD be ignored):
4762
4763 - max-mbps
4764 - max-fs
4765 - max-cpb
4766 - max-dpb
4767 - max-br
4768 - max-recv-level
4769 - max-recv-base-level
4770 - redundant-pic-cap
4771 - max-rcmd-nalu-size
4772 - deint-buf-cap
4773 - remux-buf-cap
4774 - scalable-layer-id
4775
4776 o A receiver of the SDP is required to support all parameters and
4777 values of the parameters provided; otherwise, the receiver MUST
4778 reject (RTSP) or not participate in (SAP) the session. It falls
4779 on the creator of the session to use values that are expected to
4780 be supported by the receiving application.
4781
4782 <span class="h3"><a class="selflink" id="section-7.3" href="#section-7.3">7.3</a>. Examples</span>
4783
4784 In the following examples, "{data}" is used to indicate a data string
4785 encoded as base64.
4786
4787 <span class="h4"><a class="selflink" id="section-7.3.1" href="#section-7.3.1">7.3.1</a>. Example for Offering a Single SVC Session</span>
4788
4789 Example 1: The offerer offers one video media description including
4790 two RTP payload types. The first payload type offers H264, and the
4791 second offers H264-SVC. Both payload types have different fmtp
4792 parameters as profile-level-id, packetization-mode, and sprop-
4793 parameter-sets.
4794
4795 Offerer -&gt; Answerer SDP message:
4796
4797 m=video 20000 RTP/AVP 97 96
4798 a=rtpmap:96 H264/90000
4799 a=fmtp:96 profile-level-id=4de00a; packetization-mode=0;
4800 sprop-parameter-sets={sps0},{pps0};
4801 a=rtpmap:97 H264-SVC/90000
4802 a=fmtp:97 profile-level-id=53000c; packetization-mode=1;
4803 sprop-parameter-sets={sps0},{pps0},{sps1},{pps1};
4804
4805 If the answerer does not support media subtype H264-SVC, it can issue
4806 an answer accepting only the base layer offer (payload type 96). In
4807 the following example, the receiver supports H264-SVC, so it lists
4808 payload type 97 first as the preferred option.
4809
4810
4811
4812 <span class="grey">Wenger, et al. Standards Track [Page 86]</span></pre>
4813 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-87" ></span>
4814 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
4815
4816
4817 Answerer -&gt; Offerer SDP message:
4818
4819 m=video 40000 RTP/AVP 97 96
4820 a=rtpmap:96 H264/90000
4821 a=fmtp:96 profile-level-id=4de00a; packetization-mode=0;
4822 sprop-parameter-sets={sps2},{pps2};
4823 a=rtpmap:97 H264-SVC/90000
4824 a=fmtp:97 profile-level-id=53000c; packetization-mode=1;
4825 sprop-parameter-sets={sps2},{pps2},{sps3},{pps3};
4826
4827 <span class="h4"><a class="selflink" id="section-7.3.2" href="#section-7.3.2">7.3.2</a>. Example for Offering a Single SVC Session Using</span>
4828 <span class="h4"> scalable-layer-id</span>
4829
4830 Example 2: Offerer offers the same media configurations as shown in
4831 the example above for receiving and sending the stream, but using a
4832 single RTP payload type and including sprop-operation-point-info.
4833
4834 Offerer -&gt; Answerer SDP message:
4835
4836 m=video 20000 RTP/AVP 97
4837 a=rtpmap:97 H264-SVC/90000
4838 a=fmtp:97 profile-level-id=53000c; packetization-mode=1;
4839 sprop-parameter-sets={sps0},{sps1},{pps0},{pps1};
4840 sprop-operation-point-info=&lt;1,0,0,0,4de00a,3200,176,144,128,
4841 256&gt;,&lt;2,1,1,0,53000c,6400,352,288,256,512&gt;;
4842
4843 In this example, the receiver supports H264-SVC and chooses the lower
4844 operation point offered in the RTP payload type for sending and
4845 receiving the stream.
4846
4847 Answerer -&gt; Offerer SDP message:
4848
4849 m=video 40000 RTP/AVP 97
4850 a=rtpmap:97 H264-SVC/90000
4851 a=fmtp:97 sprop-parameter-sets={sps2},{sps3},{pps2},{pps3};
4852 scalable-layer-id=1;
4853
4854 In an equivalent example showing the use of sprop-scalability-info
4855 instead using the sprop-operation-point-info, the sprop-operation-
4856 point-info would be exchanged by the sprop-scalability-info followed
4857 by the binary (base16) representation of the Scalability Information
4858 SEI message.
4859
4860 <span class="h4"><a class="selflink" id="section-7.3.3" href="#section-7.3.3">7.3.3</a>. Example for Offering Multiple Sessions in MST</span>
4861
4862 Example 3: In this example, the offerer offers a multi-session
4863 transmission with up to three sessions. The base session media
4864 description includes payload types that are backward compatible with
4865
4866
4867
4868 <span class="grey">Wenger, et al. Standards Track [Page 87]</span></pre>
4869 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-88" ></span>
4870 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
4871
4872
4873 [<a href="./rfc6184" title="&quot;RTP Payload Format for H.264 Video&quot;">RFC6184</a>], and three different payload types are offered. The other
4874 two media are using payload types with media subtype H264-SVC. In
4875 each media description, different values of profile-level-id,
4876 packetization-mode, mst-mode, and sprop-parameter-sets are offered.
4877
4878 Offerer -&gt; Answerer SDP message:
4879
4880 a=group:DDP L1 L2 L3
4881 m=video 20000 RTP/AVP 96 97 98
4882 a=rtpmap:96 H264/90000
4883 a=fmtp:96 profile-level-id=4de00a; packetization-mode=0;
4884 mst-mode=NI-T; sprop-parameter-sets={sps0},{pps0};
4885 a=rtpmap:97 H264/90000
4886 a=fmtp:97 profile-level-id=4de00a; packetization-mode=1;
4887 mst-mode=NI-TC; sprop-parameter-sets={sps0},{pps0};
4888 a=rtpmap:98 H264/90000
4889 a=fmtp:98 profile-level-id=4de00a; packetization-mode=2;
4890 mst-mode=I-C; init-buf-time=156320;
4891 sprop-parameter-sets={sps0},{pps0};
4892 a=mid:L1
4893 m=video 20002 RTP/AVP 99 100
4894 a=rtpmap:99 H264-SVC/90000
4895 a=fmtp:99 profile-level-id=53000c; packetization-mode=1;
4896 mst-mode=NI-T; sprop-parameter-sets={sps1},{pps1};
4897 a=rtpmap:100 H264-SVC/90000
4898 a=fmtp:100 profile-level-id=53000c; packetization-mode=2;
4899 mst-mode=I-C; sprop-parameter-sets={sps1},{pps1};
4900 a=mid:L2
4901 a=depend:99 lay L1:96,97; 100 lay L1:98
4902 m=video 20004 RTP/AVP 101
4903 a=rtpmap:101 H264-SVC/90000
4904 a=fmtp:101 profile-level-id=53001F; packetization-mode=1;
4905 mst-mode=NI-T; sprop-parameter-sets={sps2},{pps2};
4906 a=mid:L3
4907 a=depend:101 lay L1:96,97 L2:99
4908
4909 It is assumed that in this example the answerer only supports the NI-
4910 T mode for multi-session transmission. For this reason, it chooses
4911 the corresponding payload type (96) for the base RTP session. For
4912 the two enhancement RTP sessions, the answerer also chooses the
4913 payload types that use the NI-T mode (99 and 101).
4914
4915
4916
4917
4918
4919
4920
4921
4922
4923
4924 <span class="grey">Wenger, et al. Standards Track [Page 88]</span></pre>
4925 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-89" ></span>
4926 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
4927
4928
4929 Answerer -&gt; Offerer SDP message:
4930
4931 a=group:DDP L1 L2 L3
4932 m=video 40000 RTP/AVP 96
4933 a=rtpmap:96 H264/90000
4934 a=fmtp:96 profile-level-id=4de00a; packetization-mode=0;
4935 mst-mode=NI-T; sprop-parameter-sets={sps3},{pps3};
4936 a=mid:L1
4937 m=video 40002 RTP/AVP 99
4938 a=rtpmap:99 H264-SVC/90000
4939 a=fmtp:99 profile-level-id=53000c; packetization-mode=1;
4940 mst-mode=NI-T; sprop-parameter-sets={sps4},{pps4};
4941 a=mid:L2
4942 a=depend:99 lay L1:96
4943 m=video 40004 RTP/AVP 101
4944 a=rtpmap:101 H264-SVC/90000
4945 a=fmtp:101 profile-level-id=53001F; packetization-mode=1;
4946 mst-mode=NI-T; sprop-parameter-sets={sps5},{pps5};
4947 a=mid:L3
4948 a=depend:101 lay L1:96 L2:99
4949
4950 <span class="h4"><a class="selflink" id="section-7.3.4" href="#section-7.3.4">7.3.4</a>. Example for Offering Multiple Sessions in MST Including</span>
4951 <span class="h4"> Operation with Answerer Using scalable-layer-id</span>
4952
4953 Example 4: In this example, the offerer offers a multi-session
4954 transmission of three layers with up to two sessions. The base
4955 session media description has a payload type that is backward
4956 compatible with [<a href="./rfc6184" title="&quot;RTP Payload Format for H.264 Video&quot;">RFC6184</a>]. Note that no parameter sets are provided,
4957 in which case in-band transport must be used. The other media
4958 description contains two enhancement layers and uses the media
4959 subtype H264-SVC. It includes two operation point definitions.
4960
4961 Offerer -&gt; Answerer SDP message:
4962
4963 a=group:DDP L1 L2
4964 m=video 20000 RTP/AVP 96
4965 a=rtpmap:96 H264/90000
4966 a=fmtp:96 profile-level-id=4de00a; packetization-mode=0;
4967 mst-mode=NI-T;
4968 a=mid:L1
4969 m=video 20002 RTP/AVP 97
4970 a=rtpmap:97 H264-SVC/90000
4971 a=fmtp:97 profile-level-id=53001F; packetization-mode=1;
4972 mst-mode=NI-TC; sprop-operation-point-info=&lt;2,0,1,0,53000c,
4973 3200,352,288,384,512&gt;,&lt;3,1,2,0,53001F,6400,704,576,768,1024&gt;;
4974 a=mid:L2
4975 a=depend:97 lay L1:96
4976
4977
4978
4979
4980 <span class="grey">Wenger, et al. Standards Track [Page 89]</span></pre>
4981 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-90" ></span>
4982 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
4983
4984
4985 It is assumed that the answerer wants to send and receive the base
4986 layer (payload type 96), but it only wants to send and receive the
4987 lower enhancement layer, i.e., the one with layer id equal to 2. For
4988 this reason, the response will include the selection of the desired
4989 layer by setting scalable-layer-id equal to 2. Note that the answer
4990 only includes the scalable-layer-id information. The answer could
4991 include sprop-parameter-sets in the response.
4992
4993 Answerer -&gt; Offerer SDP message:
4994
4995 a=group:DDP L1 L2
4996 m=video 40000 RTP/AVP 96
4997 a=rtpmap:96 H264/90000
4998 a=fmtp:96 profile-level-id=4de00a; packetization-mode=0;
4999 mst-mode=NI-T;
5000 a=mid:L1
5001 m=video 40002 RTP/AVP 97
5002 a=rtpmap:97 H264-SVC/90000
5003 a=fmtp:97 scalable-layer-id=2;
5004 a=mid:L2
5005 a=depend:97 lay L1:96
5006
5007 <span class="h4"><a class="selflink" id="section-7.3.5" href="#section-7.3.5">7.3.5</a>. Example for Negotiating an SVC Stream with a Constrained Base</span>
5008 <span class="h4"> Layer in SST</span>
5009
5010 Example 5: The offerer (Alice) offers one video description including
5011 two RTP payload types with differing levels and packetization modes.
5012
5013 Offerer -&gt; Answerer SDP message:
5014
5015 m=video 20000 RTP/AVP 97 96
5016 a=rtpmap:96 H264-SVC/90000
5017 a=fmtp:96 profile-level-id=53001e; packetization-mode=0;
5018 a=rtpmap:97 H264-SVC/90000
5019 a=fmtp:97 profile-level-id=53001f; packetization-mode=1;
5020
5021 The answerer (Bridge) chooses packetization mode 1, and indicates
5022 that it would receive an SVC stream with the base layer being
5023 constrained.
5024
5025 Answerer -&gt; Offerer SDP message:
5026
5027 m=video 40000 RTP/AVP 97
5028 a=rtpmap:97 H264-SVC/90000
5029 a=fmtp:97 profile-level-id=53001f; packetization-mode=1;
5030 max-recv-base-level=000d
5031
5032
5033
5034
5035
5036 <span class="grey">Wenger, et al. Standards Track [Page 90]</span></pre>
5037 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-91" ></span>
5038 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
5039
5040
5041 The answering endpoint must send an SVC stream at Level 3.1. Since
5042 the offering endpoint did not declare max-recv-base-level, the base
5043 layer of the SVC stream the answering endpoint must send is not
5044 specifically constrained. The offering endpoint (Alice) must send an
5045 SVC stream at Level 3.1, for which the base layer must be of a level
5046 not higher than Level 1.3.
5047
5048 <span class="h3"><a class="selflink" id="section-7.4" href="#section-7.4">7.4</a>. Parameter Set Considerations</span>
5049
5050 <a href="./rfc6184#section-8.4">Section&nbsp;8.4 of [RFC6184]</a> applies in this memo, with the following
5051 applies additionally for multi-session transmission (MST).
5052
5053 In MST, regardless of out-of-band or in-band transport of parameter
5054 sets are in use, parameter sets required for decoding NAL units
5055 carried in one particular RTP session SHOULD be carried in the same
5056 session, MAY be carried in a session that the particular RTP session
5057 depends on, and MUST NOT be carried in a session that the particular
5058 RTP session does not depend on.
5059
5060 <span class="h2"><a class="selflink" id="section-8" href="#section-8">8</a>. Security Considerations</span>
5061
5062 The security considerations of the RTP Payload Format for H.264 Video
5063 specification [<a href="./rfc6184" title="&quot;RTP Payload Format for H.264 Video&quot;">RFC6184</a>] apply. Additionally, the following applies.
5064
5065 Decoders MUST exercise caution with respect to the handling of
5066 reserved NAL unit types and reserved SEI messages, particularly if
5067 they contain active elements, and MUST restrict their domain of
5068 applicability to the presentation containing the stream. The safest
5069 way is to simply discard these NAL units and SEI messages.
5070
5071 When integrity protection is applied to a stream, care MUST be taken
5072 that the stream being transported may be scalable; hence a receiver
5073 may be able to access only part of the entire stream.
5074
5075 End-to-end security with either authentication, integrity, or
5076 confidentiality protection will prevent a MANE from performing media-
5077 aware operations other than discarding complete packets. And in the
5078 case of confidentiality protection it will even be prevented from
5079 performing discarding of packets in a media-aware way. To allow any
5080 MANE to perform its operations, it will be required to be a trusted
5081 entity that is included in the security context establishment. This
5082 applies both for the media path and for the RTCP path, if RTCP
5083 packets need to be rewritten.
5084
5085
5086
5087
5088
5089
5090
5091
5092 <span class="grey">Wenger, et al. Standards Track [Page 91]</span></pre>
5093 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-92" ></span>
5094 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
5095
5096
5097 <span class="h2"><a class="selflink" id="section-9" href="#section-9">9</a>. Congestion Control</span>
5098
5099 Within any given RTP session carrying payload according to this
5100 specification, the provisions of <a href="./rfc6184#section-10">Section&nbsp;10 of [RFC6184]</a> apply.
5101 Reducing the session bitrate is possible by one or more of the
5102 following means:
5103
5104 a) Within the highest layer identified by the DID field remove any
5105 NAL units with QID higher than a certain value.
5106
5107 b) Remove all NAL units with TID higher than a certain value.
5108
5109 c) Remove all NAL units associated with a DID higher than a certain
5110 value.
5111
5112 Informative note: Removal of all coded slice NAL units
5113 associated with DIDs higher than a certain value in the entire
5114 stream is required in order to preserve conformance of the
5115 resulting SVC stream.
5116
5117 d) Utilize the PRID field to indicate the relative importance of NAL
5118 units, and remove all NAL units associated with a PRID higher than
5119 a certain value. Note that the use of the PRID is application-
5120 specific.
5121
5122 e) Remove NAL units or entire packets according to application-
5123 specific rules. The result will depend on the particular coding
5124 structure used as well as any additional application-specific
5125 functionality (e.g., concealment performed at the receiving
5126 decoder). In general, this will result in the reception of a non-
5127 conforming bitstream and hence the decoder behavior is not
5128 specified by [<a href="#ref-H.264" title="&quot;Advanced video coding for generic audiovisual services&quot;">H.264</a>]. Significant artifacts may therefore appear
5129 in the decoded output if the particular decoder implementation
5130 does not take appropriate action in response to congestion
5131 control.
5132
5133 Informative note: The discussion above is centered on NAL units
5134 rather than packets, primarily because that is the level where
5135 senders can meaningfully manipulate the scalable bitstream. The
5136 mapping of NAL units to RTP packets is fairly flexible when using
5137 aggregation packets. Depending on the nature of the congestion
5138 control algorithm, the "dimension" of congestion measurement
5139 (packet count or bitrate) and reaction to it (reducing packet
5140 count or bitrate or both) can be adjusted accordingly.
5141
5142 All aforementioned means are available to the RTP sender, regardless
5143 of whether that sender is located in the sending endpoint or in a
5144 mixer-based MANE.
5145
5146
5147
5148 <span class="grey">Wenger, et al. Standards Track [Page 92]</span></pre>
5149 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-93" ></span>
5150 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
5151
5152
5153 When a translator-based MANE is employed, then the MANE MAY
5154 manipulate the session only on the MANE's outgoing path, so that the
5155 sensed end-to-end congestion falls within the permissible envelope.
5156 As with all translators, in this case, the MANE needs to rewrite RTCP
5157 RRs to reflect the manipulations it has performed on the session.
5158
5159 Informative note: Applications MAY also implement, in addition or
5160 separately, other congestion control mechanisms, e.g., as
5161 described in [<a href="./rfc5775" title="&quot;Asynchronous Layered Coding (ALC) Protocol Instantiation&quot;">RFC5775</a>] and [<a href="#ref-Yan" title="&quot;Media- and TCP-friendly congestion control for scalable video streams&quot;">Yan</a>].
5162
5163 <span class="h2"><a class="selflink" id="section-10" href="#section-10">10</a>. IANA Considerations</span>
5164
5165 A new media type, as specified in <a href="#section-7.1">Section 7.1</a> of this memo, has been
5166 registered with IANA.
5167
5168 <span class="h2"><a class="selflink" id="section-11" href="#section-11">11</a>. Informative Appendix: Application Examples</span>
5169
5170 <span class="h3"><a class="selflink" id="section-11.1" href="#section-11.1">11.1</a>. Introduction</span>
5171
5172 Scalable video coding is a concept that has been around since at
5173 least MPEG-2 [<a href="#ref-MPEG2">MPEG2</a>], which goes back as early as 1993.
5174 Nevertheless, it has never gained wide acceptance, perhaps partly
5175 because applications didn't materialize in the form envisioned during
5176 standardization.
5177
5178 ISO/IEC MPEG and ITU-T VCEG, respectively, performed a requirement
5179 analysis for the SVC project. The MPEG and VCEG requirement
5180 documents are available in [<a href="#ref-JVT-N026" title="&quot;SVC requirements specified by MPEG (ISO/IEC JTC1 SC29 WG11)&quot;">JVT-N026</a>] and [<a href="#ref-JVT-N027" title="&quot;SVC requirements specified by VCEG (ITU-T SG16 Q.6)&quot;">JVT-N027</a>], respectively.
5181
5182 The following introduces four main application scenarios that the
5183 authors consider relevant and that are implementable with this
5184 specification.
5185
5186 <span class="h3"><a class="selflink" id="section-11.2" href="#section-11.2">11.2</a>. Layered Multicast</span>
5187
5188 This well-understood form of the use of layered coding [<a href="#ref-McCanne" title="&quot;Receiver- driven layered multicast&quot;">McCanne</a>]
5189 implies that all layers are individually conveyed in their own RTP
5190 packet streams, each carried in its own RTP session using the IP
5191 (multicast) address and port number as the single demultiplexing
5192 point. Receivers "tune" into the layers by subscribing to the IP
5193 multicast, normally by using IGMP [<a href="#ref-IGMP" title="&quot;Internet Group Management Protocol, Version 3&quot;">IGMP</a>]. Depending on the
5194 application scenario, it is also possible to convey a number of
5195 layers in one RTP session, when finer operation points within the
5196 subset of layers are not needed.
5197
5198 Layered multicast has the great advantage of simplicity and easy
5199 implementation. However, it has also the great disadvantage of
5200 utilizing many different transport addresses. While the authors
5201
5202
5203
5204 <span class="grey">Wenger, et al. Standards Track [Page 93]</span></pre>
5205 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-94" ></span>
5206 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
5207
5208
5209 consider this not to be a major problem for a professionally
5210 maintained content server, receiving client endpoints need to open
5211 many ports to IP multicast addresses in their firewalls. This is a
5212 practical problem from a firewall and network address translation
5213 (NAT) viewpoint. Furthermore, even today IP multicast is not as
5214 widely deployed as many wish.
5215
5216 The authors consider layered multicast an important application
5217 scenario for the following reasons. First, it is well understood and
5218 the implementation constraints are well known. Second, there may
5219 well be large-scale IP networks outside the immediate Internet
5220 context that may wish to employ layered multicast in the future. One
5221 possible example could be a combination of content creation and core-
5222 network distribution for the various mobile TV services, e.g., those
5223 being developed by 3GPP (MBMS) [<a href="#ref-MBMS">MBMS</a>] and DVB (DVB-H) [<a href="#ref-DVB-H" title="ETSI TR 102 377">DVB-H</a>].
5224
5225 <span class="h3"><a class="selflink" id="section-11.3" href="#section-11.3">11.3</a>. Streaming</span>
5226
5227 In this scenario, a streaming server has a repository of stored SVC
5228 coded layers for a given content. At the time of streaming, and
5229 according to the capabilities, connectivity, and congestion situation
5230 of the client(s), the streaming server generates and serves a
5231 scalable stream. Both unicast and multicast serving is possible. At
5232 the same time, the streaming server may use the same repository of
5233 stored layers to compose different streams (with a different set of
5234 layers) intended for other audiences.
5235
5236 As every endpoint receives only a single SVC RTP session, the number
5237 of firewall pinholes can be optimized to one.
5238
5239 The main difference between this scenario and straightforward
5240 simulcasting lies in the architecture and the requirements of the
5241 streaming server, and is therefore out of the scope of IETF
5242 standardization. However, compelling arguments can be made why such
5243 a streaming server design makes sense. One possible argument is
5244 related to storage space and channel bandwidth. Another is bandwidth
5245 adaptability without transcoding -- a considerable advantage in a
5246 congestion controlled network. When the streaming server learns
5247 about congestion, it can reduce the sending bitrate by choosing fewer
5248 layers when composing the layered stream; see <a href="#section-9">Section 9</a>. SVC is
5249 designed to gracefully support both bandwidth ramp-down and bandwidth
5250 ramp-up with a considerable dynamic range. This payload format is
5251 designed to allow for bandwidth flexibility in the mentioned sense.
5252 While, in theory, a transcoding step could achieve a similar dynamic
5253 range, the computational demands are impractically high and video
5254 quality is typically lowered -- therefore, few (if any) streaming
5255 servers implement full transcoding.
5256
5257
5258
5259
5260 <span class="grey">Wenger, et al. Standards Track [Page 94]</span></pre>
5261 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-95" ></span>
5262 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
5263
5264
5265 <span class="h3"><a class="selflink" id="section-11.4" href="#section-11.4">11.4</a>. Videoconferencing (Unicast to MANE, Unicast to Endpoints)</span>
5266
5267 Videoconferencing has traditionally relied on Multipoint Control
5268 Units (MCUs). These units connect endpoints in a star configuration
5269 and operate as follows. Coded video is transmitted from each
5270 endpoint to the MCU, where it is decoded, scaled, and composited to
5271 construct output frames, which are then re-encoded and transmitted to
5272 the endpoint(s). In systems supporting personalized layout (each
5273 user is allowed to select the layout of his/her screen), the
5274 compositing and encoding process is performed for each of the
5275 receiving endpoints. Even without personalized layout, rate matching
5276 still requires that the encoding process at the MCU is performed
5277 separately for each endpoint. As a result, MCUs have considerable
5278 complexity and introduce significant delay. The cascaded encodings
5279 also reduce the video quality. Particularly for multipoint
5280 connections, interactive communication is cumbersome as the end-to-
5281 end delay is very high [<a href="#ref-G.114" title="&quot;One-way transmission time&quot;">G.114</a>]. A simpler architecture is the
5282 switching MCU, in which one of the incoming video streams is
5283 redirected to the receiving endpoints. Obviously, only one user at a
5284 time can be seen and rate matching cannot be performed, thus forcing
5285 all transmitting endpoints to transmit at the lowest bit rate
5286 available in the MCU-to-endpoint connections.
5287
5288 With scalable video coding the MCU can be replaced with an
5289 application-level router (ALR): this unit simply selects which
5290 incoming packets should be transmitted to which of the receiving
5291 endpoints [<a href="#ref-Eleft" title="&quot;Multipoint Videoconferencing with Scalable Video Coding&quot;">Eleft</a>]. In such a system, each endpoint performs its own
5292 composition of the incoming video streams. Assuming, for example, a
5293 system that uses spatial scalability with two layers, personalized
5294 layout is equivalent to instructing the ALR to only send the required
5295 packets for the corresponding resolution to the particular endpoint.
5296 Similarly, rate matching at the ALR for a particular endpoint can be
5297 performed by selecting an appropriate subset of the incoming video
5298 packets to transmit to the particular endpoint. Personalized layout
5299 and rate matching thus become routing decisions, and require no
5300 signal processing. Note that scalability also allows participants to
5301 enjoy the best video quality afforded by their links, i.e., users no
5302 longer have to be forced to operate at the quality supported by the
5303 weakest endpoint. Most importantly, the ALR has an insignificant
5304 contribution to the end-to-end delay, typically an order of magnitude
5305 less than an MCU. This makes it possible to have fully interactive
5306 multipoint conferences with even a very large number of participants.
5307 There are significant advantages as well in terms of error resilience
5308 and, in fact, error tolerance can be increased by nearly an order of
5309 magnitude here as well (e.g., using unequal error protection).
5310 Finally, the very low delay of an ALR allows these systems to be
5311
5312
5313
5314
5315
5316 <span class="grey">Wenger, et al. Standards Track [Page 95]</span></pre>
5317 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-96" ></span>
5318 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
5319
5320
5321 cascaded, with significant benefits in terms of system design and
5322 deployment. Cascading of traditional MCUs is impossible due to the
5323 very high delay that even a single MCU introduces.
5324
5325 Scalable video coding enables a very significant paradigm shift in
5326 videoconferencing systems, bringing the complexity of video
5327 communication systems (particularly the servers residing within the
5328 network) in line with other types of network applications.
5329
5330 <span class="h3"><a class="selflink" id="section-11.5" href="#section-11.5">11.5</a>. Mobile TV (Multicast to MANE, Unicast to Endpoint)</span>
5331
5332 This scenario is a bit more complex, and designed to optimize the
5333 network traffic in a core network, while still requiring only a
5334 single pinhole in the endpoint's firewall. One of its key
5335 applications is the mobile TV market.
5336
5337 Consider a large private IP network, e.g., the core network of the
5338 Third Generation Partnership Project (3GPP). Streaming servers
5339 within this core network can be assumed to be professionally
5340 maintained. It is assumed that these servers can have many ports
5341 open to the network and that layered multicast is a real option.
5342 Therefore, the streaming server multicasts SVC scalable layers,
5343 instead of simulcasting different representations of the same content
5344 at different bitrates.
5345
5346 Also consider many endpoints of different classes. Some of these
5347 endpoints may lack the processing power or the display size to
5348 meaningfully decode all layers; others may have these capabilities.
5349 Users of some endpoints may wish not to pay for high quality and are
5350 happy with a base service, which may be cheaper or even free. Other
5351 users are willing to pay for high quality. Finally, some connected
5352 users may have a bandwidth problem in that they can't receive the
5353 bandwidth they would want to receive -- be it through congestion,
5354 connectivity, change of service quality, or for whatever other
5355 reasons. However, all these users have in common that they don't
5356 want to be exposed too much, and therefore the number of firewall
5357 pinholes needs to be small.
5358
5359 This situation can be handled best by introducing middleboxes close
5360 to the edge of the core network, which receive the layered multicast
5361 streams and compose the single SVC scalable bitstream according to
5362 the needs of the endpoint connected. These middleboxes are called
5363 MANEs throughout this specification. In practice, the authors
5364 envision the MANE to be part of (or at least physically and
5365 topologically close to) the base station of a mobile network, where
5366 all the signaling and media traffic necessarily are multiplexed on
5367 the same physical link.
5368
5369
5370
5371
5372 <span class="grey">Wenger, et al. Standards Track [Page 96]</span></pre>
5373 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-97" ></span>
5374 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
5375
5376
5377 MANEs necessarily need to be fairly complex devices. They certainly
5378 need to understand the signaling, so, for example, to associate the
5379 payload type octet in the RTP header with the SVC payload type.
5380
5381 A MANE may aggregate multiple RTP streams, possibly from multiple RTP
5382 sessions, thus to reduce the number of firewall pinholes required at
5383 the endpoints, or may optimize the outgoing RTP stream to the MTU
5384 size of the outgoing path by utilizing the aggregation and
5385 fragmentation mechanisms of this memo. This type of MANE is
5386 conceptually easy to implement and can offer powerful features,
5387 primarily because it necessarily can "see" the payload (including the
5388 RTP payload headers), utilize the wealth of layering information
5389 available therein, and manipulate it.
5390
5391 A MANE can also perform stream thinning, in order to adhere to
5392 congestion control principles as discussed in <a href="#section-9">Section 9</a>. While the
5393 implementation of the forward (media) channel of such a MANE appears
5394 to be comparatively simple, the need to rewrite RTCP RRs makes even
5395 such a MANE a complex device.
5396
5397 While the implementation complexity of either case of a MANE, as
5398 discussed above, is fairly high, the computational demands are
5399 comparatively low.
5400
5401 <span class="h2"><a class="selflink" id="section-12" href="#section-12">12</a>. Acknowledgements</span>
5402
5403 Miska Hannuksela contributed significantly to the designs of the
5404 PACSI NAL unit and the NI-C mode for decoding order recovery. Roni
5405 Even organized and coordinated the design team for the development of
5406 this memo, and provided valuable comments. Jonathan Lennox
5407 contributed to the NAL unit reordering algorithm for MST and provided
5408 input on several parts of this memo. Peter Amon, Sam Ganesan, Mike
5409 Nilsson, Colin Perkins, and Thomas Wiegand were members of the design
5410 team and provided valuable contributions. Magnus Westerlund has also
5411 made valuable comments. Charles Eckel and Stuart Taylor provided
5412 valuable comments after the first WGLC for this document. Xiaohui
5413 (Joanne) Wei helped improving Table 13 and the SDP examples.
5414
5415 The work of Thomas Schierl has been supported by the European
5416 Commission under contract number FP7-ICT-248036, project COAST.
5417
5418 <span class="h2"><a class="selflink" id="section-13" href="#section-13">13</a>. References</span>
5419
5420 <span class="h3"><a class="selflink" id="section-13.1" href="#section-13.1">13.1</a>. Normative References</span>
5421
5422 [<a id="ref-H.264">H.264</a>] ITU-T Recommendation H.264, "Advanced video coding for
5423 generic audiovisual services", March 2010.
5424
5425
5426
5427
5428 <span class="grey">Wenger, et al. Standards Track [Page 97]</span></pre>
5429 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-98" ></span>
5430 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
5431
5432
5433 [<a id="ref-RFC6184">RFC6184</a>] Wang, Y.-K., Even, R., Kristensen, T., and R. Jesup, "RTP
5434 Payload Format for H.264 Video", <a href="./rfc6184">RFC 6184</a>, May 2011.
5435
5436 [ISO/IEC14496-10]
5437 ISO/IEC International Standard 14496-10:2005.
5438
5439 [<a id="ref-RFC2119">RFC2119</a>] Bradner, S., "Key words for use in RFCs to Indicate
5440 Requirement Levels", <a href="https://www.rfc-editor.org/bcp/bcp14">BCP 14</a>, <a href="./rfc2119">RFC 2119</a>, March 1997.
5441
5442 [<a id="ref-RFC3264">RFC3264</a>] Rosenberg, J. and H. Schulzrinne, "An Offer/Answer Model
5443 with Session Description Protocol (SDP)", <a href="./rfc3264">RFC 3264</a>, June
5444 2002.
5445
5446 [<a id="ref-RFC3550">RFC3550</a>] Schulzrinne, H., Casner, S., Frederick, R., and V.
5447 Jacobson, "RTP: A Transport Protocol for Real-Time
5448 Applications", STD 64, <a href="./rfc3550">RFC 3550</a>, July 2003.
5449
5450 [<a id="ref-RFC4288">RFC4288</a>] Freed, N. and J. Klensin, "Media Type Specifications and
5451 Registration Procedures", <a href="https://www.rfc-editor.org/bcp/bcp13">BCP 13</a>, <a href="./rfc4288">RFC 4288</a>, December 2005.
5452
5453 [<a id="ref-RFC4566">RFC4566</a>] Handley, M., Jacobson, V., and C. Perkins, "SDP: Session
5454 Description Protocol", <a href="./rfc4566">RFC 4566</a>, July 2006.
5455
5456 [<a id="ref-RFC4648">RFC4648</a>] Josefsson, S., "The Base16, Base32, and Base64 Data
5457 Encodings", <a href="./rfc4648">RFC 4648</a>, October 2006.
5458
5459 [<a id="ref-RFC5576">RFC5576</a>] Lennox, J., Ott, J., and T. Schierl, "Source-Specific
5460 Media Attributes in the Session Description Protocol
5461 (SDP)", <a href="./rfc5576">RFC 5576</a>, June 2009.
5462
5463 [<a id="ref-RFC5583">RFC5583</a>] Schierl, T. and S. Wenger, "Signaling Media Decoding
5464 Dependency in the Session Description Protocol (SDP)", <a href="./rfc5583">RFC</a>
5465 <a href="./rfc5583">5583</a>, July 2009.
5466
5467 [<a id="ref-RFC6051">RFC6051</a>] Perkins, C. and T. Schierl, "Rapid Synchronisation of RTP
5468 Flows", <a href="./rfc6051">RFC 6051</a>, November 2010.
5469
5470 <span class="h3"><a class="selflink" id="section-13.2" href="#section-13.2">13.2</a>. Informative References</span>
5471
5472 [<a id="ref-DVB-H">DVB-H</a>] DVB - Digital Video Broadcasting (DVB); DVB-H
5473 Implementation Guidelines, ETSI TR 102 377, 2005.
5474
5475 [<a id="ref-Eleft">Eleft</a>] Eleftheriadis, A., R. Civanlar, and O. Shapiro,
5476 "Multipoint Videoconferencing with Scalable Video Coding",
5477 Journal of Zhejiang University SCIENCE A, Vol. 7, Nr. 5,
5478 April 2006, pp. 696-705. (Proceedings of the Packet Video
5479 2006 Workshop.)
5480
5481
5482
5483
5484 <span class="grey">Wenger, et al. Standards Track [Page 98]</span></pre>
5485 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-99" ></span>
5486 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
5487
5488
5489 [<a id="ref-G.114">G.114</a>] ITU-T Rec. G.114, "One-way transmission time", May 2003.
5490
5491 [<a id="ref-H.241">H.241</a>] ITU-T Rec. H.241, "Extended video procedures and control
5492 signals for H.300-series terminals", May 2006.
5493
5494 [<a id="ref-IGMP">IGMP</a>] Cain, B., Deering, S., Kouvelas, I., Fenner, B., and A.
5495 Thyagarajan, "Internet Group Management Protocol, Version
5496 3", <a href="./rfc3376">RFC 3376</a>, October 2002.
5497
5498 [<a id="ref-JVT-N026">JVT-N026</a>] Ohm J.-R., Koenen, R., and Chiariglione, L. (ed.), "SVC
5499 requirements specified by MPEG (ISO/IEC JTC1 SC29 WG11)",
5500 JVT-N026, available from <a href="http://ftp3.itu.ch/av-arch/jvt-site/2005_01_HongKong/JVT-N026.doc">http://ftp3.itu.ch/av-arch/</a>
5501 <a href="http://ftp3.itu.ch/av-arch/jvt-site/2005_01_HongKong/JVT-N026.doc">jvt-site/2005_01_HongKong/JVT-N026.doc</a>, Hong Kong, China,
5502 January 2005.
5503
5504 [<a id="ref-JVT-N027">JVT-N027</a>] Sullivan, G. and Wiegand, T. (ed.), "SVC requirements
5505 specified by VCEG (ITU-T SG16 Q.6)", JVT-N027, available
5506 from <a href="http://ftp3.itu.int/av-arch/jvt-site/2005_01_HongKong/JVT-N027.doc">http://ftp3.itu.int/av-arch/</a>
5507 <a href="http://ftp3.itu.int/av-arch/jvt-site/2005_01_HongKong/JVT-N027.doc">jvt-site/2005_01_HongKong/JVT-N027.doc</a>, Hong Kong, China,
5508 January 2005.
5509
5510 [<a id="ref-McCanne">McCanne</a>] McCanne, S., Jacobson, V., and Vetterli, M., "Receiver-
5511 driven layered multicast", in Proc. of ACM SIGCOMM'96,
5512 pages 117-130, Stanford, CA, August 1996.
5513
5514 [<a id="ref-MBMS">MBMS</a>] 3GPP - Technical Specification Group Services and System
5515 Aspects; Multimedia Broadcast/Multicast Service (MBMS);
5516 Protocols and codecs (Release 6), December 2005.
5517
5518 [<a id="ref-MPEG2">MPEG2</a>] ISO/IEC International Standard 13818-2:1993.
5519
5520 [<a id="ref-RFC2326">RFC2326</a>] Schulzrinne, H., Rao, A., and R. Lanphier, "Real Time
5521 Streaming Protocol (RTSP)", <a href="./rfc2326">RFC 2326</a>, April 1998.
5522
5523 [<a id="ref-RFC2974">RFC2974</a>] Handley, M., Perkins, C., and E. Whelan, "Session
5524 Announcement Protocol", <a href="./rfc2974">RFC 2974</a>, October 2000.
5525
5526 [<a id="ref-RFC5117">RFC5117</a>] Westerlund, M. and S. Wenger, "RTP Topologies", <a href="./rfc5117">RFC 5117</a>,
5527 January 2008.
5528
5529 [<a id="ref-RFC5775">RFC5775</a>] Luby, M., Watson, M., and L. Vicisano, "Asynchronous
5530 Layered Coding (ALC) Protocol Instantiation", <a href="./rfc5775">RFC 5775</a>,
5531 April 2010.
5532
5533 [<a id="ref-Yan">Yan</a>] Yan, J., Katrinis, K., May, M., and Plattner, R., "Media-
5534 and TCP-friendly congestion control for scalable video
5535 streams", in IEEE Trans. Multimedia, pages 196-206, April
5536 2006.
5537
5538
5539
5540 <span class="grey">Wenger, et al. Standards Track [Page 99]</span></pre>
5541 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-100" ></span>
5542 <span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span>
5543
5544
5545 Authors' Addresses
5546
5547 Stephan Wenger
5548 2400 Skyfarm Dr.
5549 Hillsborough, CA 94010
5550 USA
5551
5552 Phone: +1-415-713-5473
5553 EMail: [email protected]
5554
5555
5556 Ye-Kui Wang
5557 Huawei Technologies
5558 400 Crossing Blvd, 2nd Floor
5559 Bridgewater, NJ 08807
5560 USA
5561
5562 Phone: +1-908-541-3518
5563 EMail: [email protected]
5564
5565
5566 Thomas Schierl
5567 Fraunhofer HHI
5568 Einsteinufer 37
5569 D-10587 Berlin
5570 Germany
5571
5572 Phone: +49-30-31002-227
5573 EMail: [email protected]
5574
5575
5576 Alex Eleftheriadis
5577 Vidyo, Inc.
5578 433 Hackensack Ave.
5579 Hackensack, NJ 07601
5580 USA
5581
5582 Phone: +1-201-467-5135
5583 EMail: [email protected]
5584
5585
5586
5587
5588
5589
5590
5591
5592
5593
5594
5595
5596 Wenger, et al. Standards Track [Page 100]
5597 </pre>