| 1 |
<pre>Internet Engineering Task Force (IETF) S. Wenger |
| 2 |
Request for Comments: 6190 Independent |
| 3 |
Category: Standards Track Y.-K. Wang |
| 4 |
ISSN: 2070-1721 Huawei Technologies |
| 5 |
T. Schierl |
| 6 |
Fraunhofer HHI |
| 7 |
A. Eleftheriadis |
| 8 |
Vidyo |
| 9 |
May 2011 |
| 10 |
|
| 11 |
|
| 12 |
<span class="h1">RTP Payload Format for Scalable Video Coding</span> |
| 13 |
|
| 14 |
Abstract |
| 15 |
|
| 16 |
This memo describes an RTP payload format for Scalable Video Coding |
| 17 |
(SVC) as defined in Annex G of ITU-T Recommendation H.264, which is |
| 18 |
technically identical to Amendment 3 of ISO/IEC International |
| 19 |
Standard 14496-10. The RTP payload format allows for packetization |
| 20 |
of one or more Network Abstraction Layer (NAL) units in each RTP |
| 21 |
packet payload, as well as fragmentation of a NAL unit in multiple |
| 22 |
RTP packets. Furthermore, it supports transmission of an SVC stream |
| 23 |
over a single as well as multiple RTP sessions. The payload format |
| 24 |
defines a new media subtype name "H264-SVC", but is still backward |
| 25 |
compatible to <a href="./rfc6184">RFC 6184</a> since the base layer, when encapsulated in its |
| 26 |
own RTP stream, must use the H.264 media subtype name ("H264") and |
| 27 |
the packetization method specified in <a href="./rfc6184">RFC 6184</a>. The payload format |
| 28 |
has wide applicability in videoconferencing, Internet video |
| 29 |
streaming, and high-bitrate entertainment-quality video, among |
| 30 |
others. |
| 31 |
|
| 32 |
Status of This Memo |
| 33 |
|
| 34 |
This is an Internet Standards Track document. |
| 35 |
|
| 36 |
This document is a product of the Internet Engineering Task Force |
| 37 |
(IETF). It represents the consensus of the IETF community. It has |
| 38 |
received public review and has been approved for publication by the |
| 39 |
Internet Engineering Steering Group (IESG). Further information on |
| 40 |
Internet Standards is available in <a href="./rfc5741#section-2">Section 2 of RFC 5741</a>. |
| 41 |
|
| 42 |
Information about the current status of this document, any errata, |
| 43 |
and how to provide feedback on it may be obtained at |
| 44 |
<a href="http://www.rfc-editor.org/info/rfc6190">http://www.rfc-editor.org/info/rfc6190</a>. |
| 45 |
|
| 46 |
|
| 47 |
|
| 48 |
|
| 49 |
|
| 50 |
|
| 51 |
|
| 52 |
<span class="grey">Wenger, et al. Standards Track [Page 1]</span></pre> |
| 53 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-2" ></span> |
| 54 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 55 |
|
| 56 |
|
| 57 |
Copyright Notice |
| 58 |
|
| 59 |
Copyright (c) 2011 IETF Trust and the persons identified as the |
| 60 |
document authors. All rights reserved. |
| 61 |
|
| 62 |
This document is subject to <a href="https://www.rfc-editor.org/bcp/bcp78">BCP 78</a> and the IETF Trust's Legal |
| 63 |
Provisions Relating to IETF Documents |
| 64 |
(<a href="http://trustee.ietf.org/license-info">http://trustee.ietf.org/license-info</a>) in effect on the date of |
| 65 |
publication of this document. Please review these documents |
| 66 |
carefully, as they describe your rights and restrictions with respect |
| 67 |
to this document. Code Components extracted from this document must |
| 68 |
include Simplified BSD License text as described in Section 4.e of |
| 69 |
the Trust Legal Provisions and are provided without warranty as |
| 70 |
described in the Simplified BSD License. |
| 71 |
|
| 72 |
This document may contain material from IETF Documents or IETF |
| 73 |
Contributions published or made publicly available before November |
| 74 |
10, 2008. The person(s) controlling the copyright in some of this |
| 75 |
material may not have granted the IETF Trust the right to allow |
| 76 |
modifications of such material outside the IETF Standards Process. |
| 77 |
Without obtaining an adequate license from the person(s) controlling |
| 78 |
the copyright in such materials, this document may not be modified |
| 79 |
outside the IETF Standards Process, and derivative works of it may |
| 80 |
not be created outside the IETF Standards Process, except to format |
| 81 |
it for publication as an RFC or to translate it into languages other |
| 82 |
than English. |
| 83 |
|
| 84 |
|
| 85 |
|
| 86 |
|
| 87 |
|
| 88 |
|
| 89 |
|
| 90 |
|
| 91 |
|
| 92 |
|
| 93 |
|
| 94 |
|
| 95 |
|
| 96 |
|
| 97 |
|
| 98 |
|
| 99 |
|
| 100 |
|
| 101 |
|
| 102 |
|
| 103 |
|
| 104 |
|
| 105 |
|
| 106 |
|
| 107 |
|
| 108 |
<span class="grey">Wenger, et al. Standards Track [Page 2]</span></pre> |
| 109 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-3" ></span> |
| 110 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 111 |
|
| 112 |
|
| 113 |
Table of Contents |
| 114 |
|
| 115 |
<a href="#section-1">1</a>. Introduction ....................................................<a href="#page-5">5</a> |
| 116 |
<a href="#section-1.1">1.1</a>. The SVC Codec ..............................................<a href="#page-6">6</a> |
| 117 |
<a href="#section-1.1.1">1.1.1</a>. Overview ............................................<a href="#page-6">6</a> |
| 118 |
<a href="#section-1.1.2">1.1.2</a>. Parameter Sets ......................................<a href="#page-8">8</a> |
| 119 |
<a href="#section-1.1.3">1.1.3</a>. NAL Unit Header .....................................<a href="#page-9">9</a> |
| 120 |
<a href="#section-1.2">1.2</a>. Overview of the Payload Format ............................<a href="#page-12">12</a> |
| 121 |
<a href="#section-1.2.1">1.2.1</a>. Design Principles ..................................<a href="#page-12">12</a> |
| 122 |
<a href="#section-1.2.2">1.2.2</a>. Transmission Modes and Packetization Modes .........<a href="#page-13">13</a> |
| 123 |
<a href="#section-1.2.3">1.2.3</a>. New Payload Structures .............................<a href="#page-15">15</a> |
| 124 |
<a href="#section-2">2</a>. Conventions ....................................................<a href="#page-16">16</a> |
| 125 |
<a href="#section-3">3</a>. Definitions and Abbreviations ..................................<a href="#page-16">16</a> |
| 126 |
<a href="#section-3.1">3.1</a>. Definitions ...............................................<a href="#page-16">16</a> |
| 127 |
<a href="#section-3.1.1">3.1.1</a>. Definitions from the SVC Specification .............<a href="#page-16">16</a> |
| 128 |
<a href="#section-3.1.2">3.1.2</a>. Definitions Specific to This Memo ..................<a href="#page-18">18</a> |
| 129 |
<a href="#section-3.2">3.2</a>. Abbreviations .............................................<a href="#page-22">22</a> |
| 130 |
<a href="#section-4">4</a>. RTP Payload Format .............................................<a href="#page-23">23</a> |
| 131 |
<a href="#section-4.1">4.1</a>. RTP Header Usage ..........................................<a href="#page-23">23</a> |
| 132 |
<a href="#section-4.2">4.2</a>. NAL Unit Extension and Header Usage .......................<a href="#page-23">23</a> |
| 133 |
<a href="#section-4.2.1">4.2.1</a>. NAL Unit Extension .................................<a href="#page-23">23</a> |
| 134 |
<a href="#section-4.2.2">4.2.2</a>. NAL Unit Header Usage ..............................<a href="#page-24">24</a> |
| 135 |
<a href="#section-4.3">4.3</a>. Payload Structures ........................................<a href="#page-25">25</a> |
| 136 |
<a href="#section-4.4">4.4</a>. Transmission Modes ........................................<a href="#page-28">28</a> |
| 137 |
<a href="#section-4.5">4.5</a>. Packetization Modes .......................................<a href="#page-28">28</a> |
| 138 |
4.5.1. Packetization Modes for Single-Session |
| 139 |
Transmission .......................................<a href="#page-28">28</a> |
| 140 |
4.5.2. Packetization Modes for Multi-Session |
| 141 |
Transmission .......................................<a href="#page-29">29</a> |
| 142 |
<a href="#section-4.6">4.6</a>. Single NAL Unit Packets ...................................<a href="#page-32">32</a> |
| 143 |
<a href="#section-4.7">4.7</a>. Aggregation Packets .......................................<a href="#page-33">33</a> |
| 144 |
4.7.1. Non-Interleaved Multi-Time Aggregation |
| 145 |
Packets (NI-MTAPs) .................................<a href="#page-33">33</a> |
| 146 |
<a href="#section-4.8">4.8</a>. Fragmentation Units (FUs) .................................<a href="#page-35">35</a> |
| 147 |
<a href="#section-4.9">4.9</a>. Payload Content Scalability Information (PACSI) NAL Unit ..35 |
| 148 |
<a href="#section-4.10">4.10</a>. Empty NAL unit ...........................................<a href="#page-43">43</a> |
| 149 |
<a href="#section-4.11">4.11</a>. Decoding Order Number (DON) ..............................<a href="#page-43">43</a> |
| 150 |
4.11.1. Cross-Session DON (CS-DON) for |
| 151 |
Multi-Session Transmission ........................<a href="#page-43">43</a> |
| 152 |
<a href="#section-5">5</a>. Packetization Rules ............................................<a href="#page-45">45</a> |
| 153 |
<a href="#section-5.1">5.1</a>. Packetization Rules for Single-Session Transmission .......<a href="#page-45">45</a> |
| 154 |
<a href="#section-5.2">5.2</a>. Packetization Rules for Multi-Session Transmission ........<a href="#page-46">46</a> |
| 155 |
<a href="#section-5.2.1">5.2.1</a>. NI-T/NI-TC Packetization Rules .....................<a href="#page-47">47</a> |
| 156 |
<a href="#section-5.2.2">5.2.2</a>. NI-C/NI-TC Packetization Rules .....................<a href="#page-49">49</a> |
| 157 |
<a href="#section-5.2.3">5.2.3</a>. I-C Packetization Rules ............................<a href="#page-50">50</a> |
| 158 |
<a href="#section-5.2.4">5.2.4</a>. Packetization Rules for Non-VCL NAL Units ..........<a href="#page-50">50</a> |
| 159 |
<a href="#section-5.2.5">5.2.5</a>. Packetization Rules for Prefix NAL Units ...........<a href="#page-51">51</a> |
| 160 |
|
| 161 |
|
| 162 |
|
| 163 |
|
| 164 |
<span class="grey">Wenger, et al. Standards Track [Page 3]</span></pre> |
| 165 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-4" ></span> |
| 166 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 167 |
|
| 168 |
|
| 169 |
<a href="#section-6">6</a>. De-Packetization Process .......................................<a href="#page-51">51</a> |
| 170 |
<a href="#section-6.1">6.1</a>. De-Packetization Process for Single-Session Transmission ..51 |
| 171 |
<a href="#section-6.2">6.2</a>. De-Packetization Process for Multi-Session Transmission ...<a href="#page-51">51</a> |
| 172 |
6.2.1. Decoding Order Recovery for the NI-T and |
| 173 |
NI-TC Modes ........................................<a href="#page-52">52</a> |
| 174 |
6.2.1.1. Informative Algorithm for NI-T |
| 175 |
Decoding Order Recovery within |
| 176 |
an Access Unit ............................<a href="#page-55">55</a> |
| 177 |
6.2.2. Decoding Order Recovery for the NI-C, |
| 178 |
NI-TC, and I-C Modes ...............................<a href="#page-57">57</a> |
| 179 |
<a href="#section-7">7</a>. Payload Format Parameters ......................................<a href="#page-59">59</a> |
| 180 |
<a href="#section-7.1">7.1</a>. Media Type Registration ...................................<a href="#page-60">60</a> |
| 181 |
<a href="#section-7.2">7.2</a>. SDP Parameters ............................................<a href="#page-75">75</a> |
| 182 |
<a href="#section-7.2.1">7.2.1</a>. Mapping of Payload Type Parameters to SDP ..........<a href="#page-75">75</a> |
| 183 |
<a href="#section-7.2.2">7.2.2</a>. Usage with the SDP Offer/Answer Model ..............<a href="#page-76">76</a> |
| 184 |
7.2.3. Dependency Signaling in Multi-Session |
| 185 |
Transmission .......................................<a href="#page-84">84</a> |
| 186 |
<a href="#section-7.2.4">7.2.4</a>. Usage in Declarative Session Descriptions ..........<a href="#page-85">85</a> |
| 187 |
<a href="#section-7.3">7.3</a>. Examples ..................................................<a href="#page-86">86</a> |
| 188 |
<a href="#section-7.3.1">7.3.1</a>. Example for Offering a Single SVC Session ..........<a href="#page-86">86</a> |
| 189 |
7.3.2. Example for Offering a Single SVC Session Using |
| 190 |
scalable-layer-id ..................................<a href="#page-87">87</a> |
| 191 |
<a href="#section-7.3.3">7.3.3</a>. Example for Offering Multiple Sessions in MST ......<a href="#page-87">87</a> |
| 192 |
7.3.4. Example for Offering Multiple Sessions in |
| 193 |
MST Including Operation with Answerer Using |
| 194 |
scalable-layer-id ..................................<a href="#page-89">89</a> |
| 195 |
7.3.5. Example for Negotiating an SVC Stream with |
| 196 |
a Constrained Base Layer in SST ....................<a href="#page-90">90</a> |
| 197 |
<a href="#section-7.4">7.4</a>. Parameter Set Considerations ..............................<a href="#page-91">91</a> |
| 198 |
<a href="#section-8">8</a>. Security Considerations ........................................<a href="#page-91">91</a> |
| 199 |
<a href="#section-9">9</a>. Congestion Control .............................................<a href="#page-92">92</a> |
| 200 |
<a href="#section-10">10</a>. IANA Considerations ...........................................<a href="#page-93">93</a> |
| 201 |
<a href="#section-11">11</a>. Informative Appendix: Application Examples ....................<a href="#page-93">93</a> |
| 202 |
<a href="#section-11.1">11.1</a>. Introduction .............................................<a href="#page-93">93</a> |
| 203 |
<a href="#section-11.2">11.2</a>. Layered Multicast ........................................<a href="#page-93">93</a> |
| 204 |
<a href="#section-11.3">11.3</a>. Streaming ................................................<a href="#page-94">94</a> |
| 205 |
11.4. Videoconferencing (Unicast to MANE, Unicast to |
| 206 |
Endpoints) ...............................................<a href="#page-95">95</a> |
| 207 |
<a href="#section-11.5">11.5</a>. Mobile TV (Multicast to MANE, Unicast to Endpoint) .......<a href="#page-96">96</a> |
| 208 |
<a href="#section-12">12</a>. Acknowledgements ..............................................<a href="#page-97">97</a> |
| 209 |
<a href="#section-13">13</a>. References ....................................................<a href="#page-97">97</a> |
| 210 |
<a href="#section-13.1">13.1</a>. Normative References .....................................<a href="#page-97">97</a> |
| 211 |
<a href="#section-13.2">13.2</a>. Informative References ...................................<a href="#page-98">98</a> |
| 212 |
|
| 213 |
|
| 214 |
|
| 215 |
|
| 216 |
|
| 217 |
|
| 218 |
|
| 219 |
|
| 220 |
<span class="grey">Wenger, et al. Standards Track [Page 4]</span></pre> |
| 221 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-5" ></span> |
| 222 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 223 |
|
| 224 |
|
| 225 |
<span class="h2"><a class="selflink" id="section-1" href="#section-1">1</a>. Introduction</span> |
| 226 |
|
| 227 |
This memo specifies an RTP [<a href="./rfc3550" title=""RTP: A Transport Protocol for Real-Time Applications"">RFC3550</a>] payload format for the Scalable |
| 228 |
Video Coding (SVC) extension of the H.264/AVC video coding standard. |
| 229 |
SVC is specified in Amendment 3 to ISO/IEC 14496 Part 10 |
| 230 |
[ISO/IEC14496-10] and equivalently in Annex G of ITU-T Rec. H.264 |
| 231 |
[<a href="#ref-H.264" title=""Advanced video coding for generic audiovisual services"">H.264</a>]. In this memo, unless explicitly stated otherwise, |
| 232 |
"H.264/AVC" refers to the specification of [<a href="#ref-H.264" title=""Advanced video coding for generic audiovisual services"">H.264</a>] excluding Annex G. |
| 233 |
|
| 234 |
SVC covers the entire application range of H.264/AVC, from low- |
| 235 |
bitrate mobile applications, to High-Definition Television (HDTV) |
| 236 |
broadcasting, and even Digital Cinema that requires nearly lossless |
| 237 |
coding and hundreds of megabits per second. The scalability features |
| 238 |
that SVC adds to H.264/AVC enable several system-level |
| 239 |
functionalities related to the ability of a system to adapt the |
| 240 |
signal to different system conditions with no or minimal processing. |
| 241 |
The adaptation relates both to the capabilities of potentially |
| 242 |
heterogeneous receivers (differing in screen resolution, processing |
| 243 |
speed, etc.), and to differing or time-varying network conditions. |
| 244 |
The adaptation can be performed at the source, the destination, or in |
| 245 |
intermediate media-aware network elements (MANEs). The payload |
| 246 |
format specified in this memo exposes these system-level |
| 247 |
functionalities so that system designers can take direct advantage of |
| 248 |
these features. |
| 249 |
|
| 250 |
Informative note: Since SVC streams contain, by design, a sub- |
| 251 |
stream that is compliant with H.264/AVC, it is trivial for a MANE |
| 252 |
to filter the stream so that all SVC-specific information is |
| 253 |
removed. This memo, in fact, defines a media type parameter |
| 254 |
(sprop-avc-ready, <a href="#section-7.2">Section 7.2</a>) that indicates whether or not the |
| 255 |
stream can be converted to one compliant with [<a href="./rfc6184" title=""RTP Payload Format for H.264 Video"">RFC6184</a>] by |
| 256 |
eliminating RTP packets, and rewriting RTP Control Protocol (RTCP) |
| 257 |
to match the changes to the RTP packet stream as specified in |
| 258 |
<a href="./rfc3550#section-7">Section 7 of [RFC3550]</a>. |
| 259 |
|
| 260 |
This memo defines two basic modes for transmission of SVC data, |
| 261 |
single-session transmission (SST) and multi-session transmission |
| 262 |
(MST). In SST, a single RTP session is used for the transmission of |
| 263 |
all scalability layers comprising an SVC bitstream; in MST, the |
| 264 |
scalability layers are transported on different RTP sessions. In |
| 265 |
SST, packetization is a straightforward extension of [<a href="./rfc6184" title=""RTP Payload Format for H.264 Video"">RFC6184</a>]. For |
| 266 |
MST, four different modes are defined in this memo. They differ on |
| 267 |
whether or not they allow interleaving, i.e., transmitting Network |
| 268 |
Abstraction Layer (NAL) units in an order different than the decoding |
| 269 |
order, and by the technique used to effect inter-session NAL unit |
| 270 |
decoding order recovery. Decoding order recovery is performed using |
| 271 |
either inter-session timestamp alignment [<a href="./rfc3550" title=""RTP: A Transport Protocol for Real-Time Applications"">RFC3550</a>] or cross-session |
| 272 |
decoding order numbers (CS-DONs). One of the MST modes supports both |
| 273 |
|
| 274 |
|
| 275 |
|
| 276 |
<span class="grey">Wenger, et al. Standards Track [Page 5]</span></pre> |
| 277 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-6" ></span> |
| 278 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 279 |
|
| 280 |
|
| 281 |
decoding order recovery techniques, so that receivers can select |
| 282 |
their preferred technique. More details can be found in <a href="#section-1.2.2">Section</a> |
| 283 |
<a href="#section-1.2.2">1.2.2</a>. |
| 284 |
|
| 285 |
This memo further defines three new NAL unit types. The first type |
| 286 |
is the payload content scalability information (PACSI) NAL unit, |
| 287 |
which is used to provide an informative summary of the scalability |
| 288 |
information of the data contained in an RTP packet, as well as |
| 289 |
ancillary data (e.g., CS-DON values). The second and third new NAL |
| 290 |
unit types are the empty NAL unit and the non-interleaved multi-time |
| 291 |
aggregation packet (NI-MTAP) NAL unit. The empty NAL unit is used to |
| 292 |
ensure inter-session timestamp alignment required for decoding order |
| 293 |
recovery in MST. The NI-MTAP is used as a new payload structure |
| 294 |
allowing the grouping of NAL units of different time instances in |
| 295 |
decoding order. More details about the new packet structures can be |
| 296 |
found in <a href="#section-1.2.3">Section 1.2.3</a>. |
| 297 |
|
| 298 |
This memo also defines the signaling support for SVC transport over |
| 299 |
RTP, including a new media subtype name (H264-SVC). |
| 300 |
|
| 301 |
A non-normative overview of the SVC codec and the payload is given in |
| 302 |
the remainder of this section. |
| 303 |
|
| 304 |
<span class="h3"><a class="selflink" id="section-1.1" href="#section-1.1">1.1</a>. The SVC Codec</span> |
| 305 |
|
| 306 |
<span class="h4"><a class="selflink" id="section-1.1.1" href="#section-1.1.1">1.1.1</a>. Overview</span> |
| 307 |
|
| 308 |
SVC defines a coded video representation in which a given bitstream |
| 309 |
offers representations of the source material at different levels of |
| 310 |
fidelity (hence the term "scalable"). Scalable video coding |
| 311 |
bitstreams, or scalable bitstreams, are constructed in a pyramidal |
| 312 |
fashion: the coding process creates bitstream components that improve |
| 313 |
the fidelity of hierarchically lower components. |
| 314 |
|
| 315 |
The fidelity dimensions offered by SVC are spatial (picture size), |
| 316 |
quality (or Signal-to-Noise Ratio (SNR)), and temporal (pictures per |
| 317 |
second). Bitstream components associated with a given level of |
| 318 |
spatial, quality, and temporal fidelity are identified using |
| 319 |
corresponding parameters in the bitstream: dependency_id, quality_id, |
| 320 |
and temporal_id (see also <a href="#section-1.1.3">Section 1.1.3</a>). The fidelity identifiers |
| 321 |
have integer values, where higher values designate components that |
| 322 |
are higher in the hierarchy. It is noted that SVC offers significant |
| 323 |
flexibility in terms of how an encoder may choose to structure the |
| 324 |
dependencies between the various components. Decoding of a |
| 325 |
particular component requires the availability of all the components |
| 326 |
it depends upon, either directly, or indirectly. An operation point |
| 327 |
|
| 328 |
|
| 329 |
|
| 330 |
|
| 331 |
|
| 332 |
<span class="grey">Wenger, et al. Standards Track [Page 6]</span></pre> |
| 333 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-7" ></span> |
| 334 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 335 |
|
| 336 |
|
| 337 |
of an SVC bitstream consists of the bitstream components required to |
| 338 |
be able to decode a particular dependency_id, quality_id, and |
| 339 |
temporal_id combination. |
| 340 |
|
| 341 |
The term "layer" is used in various contexts in this memo. For |
| 342 |
example, in the terms "Video Coding Layer" and "Network Abstraction |
| 343 |
Layer" it refers to conceptual organization levels. When referring |
| 344 |
to bitstream syntax elements such as block layer or macroblock layer, |
| 345 |
it refers to hierarchical bitstream structure levels. When used in |
| 346 |
the context of bitstream scalability, e.g., "AVC base layer", it |
| 347 |
refers to a level of representation fidelity of the source signal |
| 348 |
with a specific set of NAL units included. The correct |
| 349 |
interpretation is supported by providing the appropriate context. |
| 350 |
|
| 351 |
SVC maintains the bitstream organization introduced in H.264/AVC. |
| 352 |
Specifically, all bitstream components are encapsulated in Network |
| 353 |
Abstraction Layer (NAL) units, which are organized as Access Units |
| 354 |
(AUs). An AU is associated with a single sampling instance in time. |
| 355 |
A subset of the NAL unit types correspond to the Video Coding Layer |
| 356 |
(VCL), and contain the coded picture data associated with the source |
| 357 |
content. Non-VCL NAL units carry ancillary data that may be |
| 358 |
necessary for decoding (e.g., parameter sets as explained below) or |
| 359 |
that facilitate certain system operations but are not needed by the |
| 360 |
decoding process itself. Coded picture data at the various fidelity |
| 361 |
dimensions are organized in slices. Within one AU, a coded picture |
| 362 |
of an operation point consists of all the coded slices required for |
| 363 |
decoding up to the particular combination of dependency_id and |
| 364 |
quality_id values at the time instance corresponding to the AU. |
| 365 |
|
| 366 |
It is noted that the concept of temporal scalability is already |
| 367 |
present in H.264/AVC, as profiles defined in Annex A of [<a href="#ref-H.264" title=""Advanced video coding for generic audiovisual services"">H.264</a>] |
| 368 |
already support it. Specifically, in H.264/AVC, the concept of sub- |
| 369 |
sequences has been introduced to allow optional use of temporal |
| 370 |
layers through Supplemental Enhancement Information (SEI) messages. |
| 371 |
SVC extends this approach by exposing the temporal scalability |
| 372 |
information using the temporal_id parameter, alongside (and unified |
| 373 |
with) the dependency_id and quality_id values that are used for |
| 374 |
spatial and quality scalability, respectively. For coded picture |
| 375 |
data defined in Annex G of [<a href="#ref-H.264" title=""Advanced video coding for generic audiovisual services"">H.264</a>], this is accomplished by using a |
| 376 |
new type of NAL unit, namely, coded slice in scalable extension NAL |
| 377 |
unit (type 20), where the fidelity parameters are part of its header. |
| 378 |
For coded picture data that follow H.264/AVC, and to ensure |
| 379 |
compatibility with existing H.264/AVC decoders, another new type of |
| 380 |
NAL unit, namely, prefix NAL unit (type 14), has been defined to |
| 381 |
carry this header information. SVC additionally specifies a third |
| 382 |
new type of NAL unit, namely, subset sequence parameter set NAL unit |
| 383 |
(type 15), to contain sequence parameter set information for quality |
| 384 |
and spatial enhancement layers. All these three newly specified NAL |
| 385 |
|
| 386 |
|
| 387 |
|
| 388 |
<span class="grey">Wenger, et al. Standards Track [Page 7]</span></pre> |
| 389 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-8" ></span> |
| 390 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 391 |
|
| 392 |
|
| 393 |
unit types (14, 15, and 20) are among those reserved in H.264/AVC and |
| 394 |
are to be ignored by decoders conforming to one or more of the |
| 395 |
profiles specified in Annex A of [<a href="#ref-H.264" title=""Advanced video coding for generic audiovisual services"">H.264</a>]. |
| 396 |
|
| 397 |
Within an AU, the VCL NAL units associated with a given dependency_id |
| 398 |
and quality_id are referred to as a "layer representation". The |
| 399 |
layer representation corresponding to the lowest values of |
| 400 |
dependency_id and quality_id (i.e., zero for both) is compliant by |
| 401 |
design to H.264/AVC. The set of VCL and associated non-VCL NAL units |
| 402 |
across all AUs in a bitstream associated with a particular |
| 403 |
combination of values of dependency_id and quality_id, and regardless |
| 404 |
of the value of temporal_id, is conceptually a scalable layer. For |
| 405 |
backward compatibility with H.264/AVC, it is important to |
| 406 |
differentiate, however, whether or not SVC-specific NAL units are |
| 407 |
present in a given bitstream. This is particularly important for the |
| 408 |
lowest fidelity values in terms of dependency_id and quality_id (zero |
| 409 |
for both), as the corresponding VCL data are compliant with |
| 410 |
H.264/AVC, and may or may not be accompanied by associated prefix NAL |
| 411 |
units. This memo therefore uses the term "AVC base layer" to |
| 412 |
designate the layer that does not contain SVC-specific NAL units, and |
| 413 |
"SVC base layer" to designate the same layer but with the addition of |
| 414 |
the associated SVC prefix NAL units. Note that the SVC specification |
| 415 |
uses the term "base layer" for what in this memo will be referred to |
| 416 |
as "AVC base layer". Similarly, it is also important to be able to |
| 417 |
differentiate, within a layer, the temporal fidelity components it |
| 418 |
contains. This memo uses the term "T0" to indicate, within a |
| 419 |
particular layer, the subset that contains the NAL units associated |
| 420 |
with temporal_id equal to 0. |
| 421 |
|
| 422 |
SNR scalability in SVC is offered in two different ways. In what is |
| 423 |
called coarse-grain scalability (CGS), scalability is provided by |
| 424 |
including or excluding a complete layer when decoding a particular |
| 425 |
bitstream. In contrast, in medium-grain scalability (MGS), |
| 426 |
scalability is provided by selectively omitting the decoding of |
| 427 |
specific NAL units belonging to MGS layers. The selection of the NAL |
| 428 |
units to omit can be based on fixed-length fields present in the NAL |
| 429 |
unit header (see also Sections <a href="#section-1.1.3">1.1.3</a> and <a href="#section-4.2">4.2</a>). |
| 430 |
|
| 431 |
<span class="h4"><a class="selflink" id="section-1.1.2" href="#section-1.1.2">1.1.2</a>. Parameter Sets</span> |
| 432 |
|
| 433 |
SVC maintains the parameter sets concept in H.264/AVC and introduces |
| 434 |
a new type of sequence parameter set, referred to as the subset |
| 435 |
sequence parameter set [<a href="#ref-H.264" title=""Advanced video coding for generic audiovisual services"">H.264</a>]. Subset sequence parameter sets have |
| 436 |
NAL unit type equal to 15, which is different from the NAL unit type |
| 437 |
value (7) of sequence parameter sets. VCL NAL units of NAL unit type |
| 438 |
1 to 5 must only (indirectly) refer to sequence parameter sets, while |
| 439 |
VCL NAL units of NAL unit type 20 must only (indirectly) refer to |
| 440 |
subset sequence parameter sets. The references are indirect because |
| 441 |
|
| 442 |
|
| 443 |
|
| 444 |
<span class="grey">Wenger, et al. Standards Track [Page 8]</span></pre> |
| 445 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-9" ></span> |
| 446 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 447 |
|
| 448 |
|
| 449 |
VCL NAL units refer to picture parameter sets (in their slice |
| 450 |
header), which in turn refer to regular or subset sequence parameter |
| 451 |
sets. Subset sequence parameter sets use a separate identifier value |
| 452 |
space than sequence parameter sets. |
| 453 |
|
| 454 |
In SVC, coded picture data from different layers may use the same or |
| 455 |
different sequence and picture parameter sets. Let the variable DQId |
| 456 |
be equal to dependency_id * 16 + quality_id. At any time instant |
| 457 |
during the decoding process there is one active sequence parameter |
| 458 |
set for the layer representation with the highest value of DQId and |
| 459 |
one or more active layer SVC sequence parameter set(s) for layer |
| 460 |
representations with lower values of DQId. The active sequence |
| 461 |
parameter set or an active layer SVC sequence parameter set remains |
| 462 |
unchanged throughout a coded video sequence in the scalable layer in |
| 463 |
which the active sequence parameter set or active layer SVC sequence |
| 464 |
parameter set is referred to. This means that the referred sequence |
| 465 |
parameter set or subset sequence parameter set can only change at |
| 466 |
instantaneous decoding refresh (IDR) access units for any layer. At |
| 467 |
any time instant during the decoding process there may be one active |
| 468 |
picture parameter set (for the layer representation with the highest |
| 469 |
value of DQId) and one or more active layer picture parameter set(s) |
| 470 |
(for layer representations with lower values of DQId). The active |
| 471 |
picture parameter set or an active layer picture parameter set |
| 472 |
remains unchanged throughout a layer representation in which the |
| 473 |
active picture parameter set or active layer picture parameter set is |
| 474 |
referred to, but may change from one AU to the next. |
| 475 |
|
| 476 |
<span class="h4"><a class="selflink" id="section-1.1.3" href="#section-1.1.3">1.1.3</a>. NAL Unit Header</span> |
| 477 |
|
| 478 |
SVC extends the one-byte H.264/AVC NAL unit header by three |
| 479 |
additional octets for NAL units of types 14 and 20. The header |
| 480 |
indicates the type of the NAL unit, the (potential) presence of bit |
| 481 |
errors or syntax violations in the NAL unit payload, information |
| 482 |
regarding the relative importance of the NAL unit for the decoding |
| 483 |
process, the layer identification information, and other fields as |
| 484 |
discussed below. |
| 485 |
|
| 486 |
The syntax and semantics of the NAL unit header are specified in |
| 487 |
[<a href="#ref-H.264" title=""Advanced video coding for generic audiovisual services"">H.264</a>], but the essential properties of the NAL unit header are |
| 488 |
summarized below for convenience. |
| 489 |
|
| 490 |
The first byte of the NAL unit header has the following format (the |
| 491 |
bit fields are the same as defined for the one-byte H.264/AVC NAL |
| 492 |
unit header, while the semantics of some fields have changed |
| 493 |
slightly, in a backward-compatible way): |
| 494 |
|
| 495 |
|
| 496 |
|
| 497 |
|
| 498 |
|
| 499 |
|
| 500 |
<span class="grey">Wenger, et al. Standards Track [Page 9]</span></pre> |
| 501 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-10" ></span> |
| 502 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 503 |
|
| 504 |
|
| 505 |
+---------------+ |
| 506 |
|0|1|2|3|4|5|6|7| |
| 507 |
+-+-+-+-+-+-+-+-+ |
| 508 |
|F|NRI| Type | |
| 509 |
+---------------+ |
| 510 |
|
| 511 |
The semantics of the components of the NAL unit type octet, as |
| 512 |
specified in [<a href="#ref-H.264" title=""Advanced video coding for generic audiovisual services"">H.264</a>], are described briefly below. In addition to |
| 513 |
the name and size of each field, the corresponding syntax element |
| 514 |
name in [<a href="#ref-H.264" title=""Advanced video coding for generic audiovisual services"">H.264</a>] is also provided. |
| 515 |
|
| 516 |
F: 1 bit |
| 517 |
forbidden_zero_bit. H.264/AVC declares a value of 1 as a |
| 518 |
syntax violation. |
| 519 |
|
| 520 |
NRI: 2 bits |
| 521 |
nal_ref_idc. A value of "00" (in binary form) indicates that |
| 522 |
the content of the NAL unit is not used to reconstruct |
| 523 |
reference pictures for future prediction. Such NAL units can |
| 524 |
be discarded without risking the integrity of the reference |
| 525 |
pictures in the same layer. A value greater than "00" |
| 526 |
indicates that the decoding of the NAL unit is required to |
| 527 |
maintain the integrity of reference pictures in the same layer |
| 528 |
or that the NAL unit contains parameter sets. |
| 529 |
|
| 530 |
Type: 5 bits |
| 531 |
nal_unit_type. This component specifies the NAL unit type as |
| 532 |
defined in Table 7-1 of [<a href="#ref-H.264" title=""Advanced video coding for generic audiovisual services"">H.264</a>], and later within this memo. |
| 533 |
For a reference of all currently defined NAL unit types and |
| 534 |
their semantics, please refer to Section 7.4.1 in [<a href="#ref-H.264" title=""Advanced video coding for generic audiovisual services"">H.264</a>]. |
| 535 |
|
| 536 |
In H.264/AVC, NAL unit types 14, 15, and 20 are reserved for |
| 537 |
future extensions. SVC uses these three NAL unit types as |
| 538 |
follows: NAL unit type 14 is used for prefix NAL unit, NAL unit |
| 539 |
type 15 is used for subset sequence parameter set, and NAL unit |
| 540 |
type 20 is used for coded slice in scalable extension (see |
| 541 |
Section 7.4.1 in [<a href="#ref-H.264" title=""Advanced video coding for generic audiovisual services"">H.264</a>]). NAL unit types 14 and 20 indicate |
| 542 |
the presence of three additional octets in the NAL unit header, |
| 543 |
as shown below. |
| 544 |
|
| 545 |
+---------------+---------------+---------------+ |
| 546 |
|0|1|2|3|4|5|6|7|0|1|2|3|4|5|6|7|0|1|2|3|4|5|6|7| |
| 547 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |
| 548 |
|R|I| PRID |N| DID | QID | TID |U|D|O| RR| |
| 549 |
+---------------+---------------+---------------+ |
| 550 |
|
| 551 |
|
| 552 |
|
| 553 |
|
| 554 |
|
| 555 |
|
| 556 |
<span class="grey">Wenger, et al. Standards Track [Page 10]</span></pre> |
| 557 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-11" ></span> |
| 558 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 559 |
|
| 560 |
|
| 561 |
R: 1 bit |
| 562 |
reserved_one_bit. Reserved bit for future extension. R must |
| 563 |
be equal to 1. The value of R must be ignored by decoders. |
| 564 |
|
| 565 |
I: 1 bit |
| 566 |
idr_flag. This component specifies whether the layer |
| 567 |
representation is an instantaneous decoding refresh (IDR) layer |
| 568 |
representation (when equal to 1) or not (when equal to 0). |
| 569 |
|
| 570 |
PRID: 6 bits |
| 571 |
priority_id. This flag specifies a priority identifier for the |
| 572 |
NAL unit. A lower value of PRID indicates a higher priority. |
| 573 |
|
| 574 |
N: 1 bit |
| 575 |
no_inter_layer_pred_flag. This flag specifies, when present in |
| 576 |
a coded slice NAL unit, whether inter-layer prediction may be |
| 577 |
used for decoding the coded slice (when equal to 1) or not |
| 578 |
(when equal to 0). |
| 579 |
|
| 580 |
DID: 3 bits |
| 581 |
dependency_id. This component indicates the inter-layer coding |
| 582 |
dependency level of a layer representation. At any access |
| 583 |
unit, a layer representation with a given dependency_id may be |
| 584 |
used for inter-layer prediction for coding of a layer |
| 585 |
representation with a higher dependency_id, while a layer |
| 586 |
representation with a given dependency_id shall not be used for |
| 587 |
inter-layer prediction for coding of a layer representation |
| 588 |
with a lower dependency_id. |
| 589 |
|
| 590 |
QID: 4 bits |
| 591 |
quality_id. This component indicates the quality level of an |
| 592 |
MGS layer representation. At any access unit and for identical |
| 593 |
dependency_id values, a layer representation with quality_id |
| 594 |
equal to ql uses a layer representation with quality_id equal |
| 595 |
to ql-1 for inter-layer prediction. |
| 596 |
|
| 597 |
TID: 3 bits |
| 598 |
temporal_id. This component indicates the temporal level of a |
| 599 |
layer representation. The temporal_id is associated with the |
| 600 |
frame rate, with lower values of _temporal_id corresponding to |
| 601 |
lower frame rates. A layer representation at a given |
| 602 |
temporal_id typically depends on layer representations with |
| 603 |
lower temporal_id values, but it never depends on layer |
| 604 |
representations with higher temporal_id values. |
| 605 |
|
| 606 |
|
| 607 |
|
| 608 |
|
| 609 |
|
| 610 |
|
| 611 |
|
| 612 |
<span class="grey">Wenger, et al. Standards Track [Page 11]</span></pre> |
| 613 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-12" ></span> |
| 614 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 615 |
|
| 616 |
|
| 617 |
U: 1 bit |
| 618 |
use_ref_base_pic_flag. A value of 1 indicates that only |
| 619 |
reference base pictures are used during the inter prediction |
| 620 |
process. A value of 0 indicates that the reference base |
| 621 |
pictures are not used during the inter prediction process. |
| 622 |
|
| 623 |
D: 1 bit |
| 624 |
discardable_flag. A value of 1 indicates that the current NAL |
| 625 |
unit is not used for decoding NAL units with values of |
| 626 |
dependency_id higher than the one of the current NAL unit, in |
| 627 |
the current and all subsequent access units. Such NAL units |
| 628 |
can be discarded without risking the integrity of layers with |
| 629 |
higher dependency_id values. discardable_flag equal to 0 |
| 630 |
indicates that the decoding of the NAL unit is required to |
| 631 |
maintain the integrity of layers with higher dependency_id. |
| 632 |
|
| 633 |
O: 1 bit |
| 634 |
output_flag: Affects the decoded picture output process as |
| 635 |
defined in Annex C of [<a href="#ref-H.264" title=""Advanced video coding for generic audiovisual services"">H.264</a>]. |
| 636 |
|
| 637 |
RR: 2 bits |
| 638 |
reserved_three_2bits. Reserved bits for future extension. RR |
| 639 |
MUST be equal to "11" (in binary form). The value of RR must |
| 640 |
be ignored by decoders. |
| 641 |
|
| 642 |
This memo extends the semantics of F, NRI, I, PRID, DID, QID, TID, U, |
| 643 |
and D per Annex G of [<a href="#ref-H.264" title=""Advanced video coding for generic audiovisual services"">H.264</a>] as described in <a href="#section-4.2">Section 4.2</a>. |
| 644 |
|
| 645 |
<span class="h3"><a class="selflink" id="section-1.2" href="#section-1.2">1.2</a>. Overview of the Payload Format</span> |
| 646 |
|
| 647 |
Similar to [<a href="./rfc6184" title=""RTP Payload Format for H.264 Video"">RFC6184</a>], this payload format can only be used to carry |
| 648 |
the raw NAL unit stream over RTP and not the bytestream format |
| 649 |
specified in Annex B of [<a href="#ref-H.264" title=""Advanced video coding for generic audiovisual services"">H.264</a>]. |
| 650 |
|
| 651 |
The design principles, transmission modes, and packetization modes as |
| 652 |
well as new payload structures are summarized in this section. It is |
| 653 |
assumed that the reader is familiar with the terminology and concepts |
| 654 |
defined in [<a href="./rfc6184" title=""RTP Payload Format for H.264 Video"">RFC6184</a>]. |
| 655 |
|
| 656 |
<span class="h4"><a class="selflink" id="section-1.2.1" href="#section-1.2.1">1.2.1</a>. Design Principles</span> |
| 657 |
|
| 658 |
The following design principles have been observed for this payload |
| 659 |
format: |
| 660 |
|
| 661 |
o Backward compatibility with [<a href="./rfc6184" title=""RTP Payload Format for H.264 Video"">RFC6184</a>] wherever possible. |
| 662 |
|
| 663 |
|
| 664 |
|
| 665 |
|
| 666 |
|
| 667 |
|
| 668 |
<span class="grey">Wenger, et al. Standards Track [Page 12]</span></pre> |
| 669 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-13" ></span> |
| 670 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 671 |
|
| 672 |
|
| 673 |
o The SVC base layer or any H.264/AVC compatible subset of the SVC |
| 674 |
base layer, when transmitted in its own RTP stream, must be |
| 675 |
encapsulated using [<a href="./rfc6184" title=""RTP Payload Format for H.264 Video"">RFC6184</a>]. This ensures that such an RTP |
| 676 |
stream can be understood by [<a href="./rfc6184" title=""RTP Payload Format for H.264 Video"">RFC6184</a>] receivers. |
| 677 |
|
| 678 |
o Media-aware network elements (MANEs) as defined in [<a href="./rfc6184" title=""RTP Payload Format for H.264 Video"">RFC6184</a>] are |
| 679 |
signaling-aware, rely on signaling information, and have state. |
| 680 |
|
| 681 |
o MANEs can aggregate multiple RTP streams, possibly from multiple |
| 682 |
RTP sessions. |
| 683 |
|
| 684 |
o MANEs can perform media-aware stream thinning (selective |
| 685 |
elimination of packets or portions thereof). By using the payload |
| 686 |
header information identifying layers within an RTP session, MANEs |
| 687 |
are able to remove packets or portions thereof from the incoming |
| 688 |
RTP packet stream. This implies rewriting the RTP headers of the |
| 689 |
outgoing packet stream, and rewriting of RTCP packets as specified |
| 690 |
in <a href="./rfc3550#section-7">Section 7 of [RFC3550]</a>. |
| 691 |
|
| 692 |
<span class="h4"><a class="selflink" id="section-1.2.2" href="#section-1.2.2">1.2.2</a>. Transmission Modes and Packetization Modes</span> |
| 693 |
|
| 694 |
This memo allows the packetization of SVC data for both single- |
| 695 |
session transmission (SST) and multi-session transmission (MST). In |
| 696 |
the case of SST all SVC data are carried in a single RTP session. In |
| 697 |
the case of MST two or more RTP sessions are used to carry the SVC |
| 698 |
data, in accordance with the MST-specific packetization modes defined |
| 699 |
in this memo, which are based on the packetization modes defined in |
| 700 |
[<a href="./rfc6184" title=""RTP Payload Format for H.264 Video"">RFC6184</a>]. In MST, each RTP session is associated with one RTP |
| 701 |
stream, which may carry one or more layers. |
| 702 |
|
| 703 |
The base layer is, by design, compatible to H.264/AVC. During |
| 704 |
transmission, the associated prefix NAL units, which are introduced |
| 705 |
by SVC and, when present, are ignored by H.264/AVC decoders, may be |
| 706 |
encapsulated within the same RTP packet stream as the H.264/AVC VCL |
| 707 |
NAL units or in a different RTP packet stream (when MST is used). |
| 708 |
For convenience, the term "AVC base layer" is used to refer to the |
| 709 |
base layer without prefix NAL units, while the term "SVC base layer" |
| 710 |
is used to refer to the base layer with prefix NAL units. |
| 711 |
|
| 712 |
Furthermore, the base layer may have multiple temporal components |
| 713 |
(i.e., supporting different frame rates). As a result, the lowest |
| 714 |
temporal component ("T0") of the AVC or SVC base layer is used as the |
| 715 |
starting point of the SVC bitstream hierarchy. |
| 716 |
|
| 717 |
This memo allows encapsulating in a given RTP stream any of the |
| 718 |
following three alternatives of layer combinations: |
| 719 |
|
| 720 |
|
| 721 |
|
| 722 |
|
| 723 |
|
| 724 |
<span class="grey">Wenger, et al. Standards Track [Page 13]</span></pre> |
| 725 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-14" ></span> |
| 726 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 727 |
|
| 728 |
|
| 729 |
1. the T0 AVC base layer or the T0 SVC base layer only; |
| 730 |
2. one or more enhancement layers only; or |
| 731 |
3. the T0 SVC base layer, and one or more enhancement layers. |
| 732 |
|
| 733 |
SST should be used in point-to-point unicast applications and, in |
| 734 |
general, whenever the potential benefit of using multiple RTP |
| 735 |
sessions does not justify the added complexity. When SST is used, |
| 736 |
the layer combination cases 1 and 3 above can be used. When an |
| 737 |
H.264/AVC compatible subset of the SVC base layer is transmitted |
| 738 |
using SST, the packetization of [<a href="./rfc6184" title=""RTP Payload Format for H.264 Video"">RFC6184</a>] must be used, thus ensuring |
| 739 |
compatibility with [<a href="./rfc6184" title=""RTP Payload Format for H.264 Video"">RFC6184</a>] receivers. When, however, one or more |
| 740 |
SVC quality or spatial enhancement layers are transmitted using SST, |
| 741 |
the packetization defined in this memo must be used. In SST, any of |
| 742 |
the three [<a href="./rfc6184" title=""RTP Payload Format for H.264 Video"">RFC6184</a>] packetization modes, namely, single NAL unit |
| 743 |
mode, non-interleaved mode, and interleaved mode, can be used. |
| 744 |
|
| 745 |
MST should be used in a multicast session when different receivers |
| 746 |
may request different layers of the scalable bitstream. An operation |
| 747 |
point for an SVC bitstream, as defined in this memo, corresponds to a |
| 748 |
set of layers that together conform to one of the profiles defined in |
| 749 |
Annex A or G of [<a href="#ref-H.264" title=""Advanced video coding for generic audiovisual services"">H.264</a>] and, when decoded, offer a representation of |
| 750 |
the original video at a certain fidelity. The number of streams used |
| 751 |
in MST should be at least equal to the number of operation points |
| 752 |
that may be requested by the receivers. Depending on the |
| 753 |
application, this may result in each layer being carried in its own |
| 754 |
RTP session, or in having multiple layers encapsulated within one RTP |
| 755 |
session. |
| 756 |
|
| 757 |
Informative note: Layered multicast is a term commonly used to |
| 758 |
describe the application where multicast is used to transmit |
| 759 |
layered or scalable data that has been encapsulated into more than |
| 760 |
one RTP session. This application allows different receivers in |
| 761 |
the multicast session to receive different operation points of the |
| 762 |
scalable bitstream. Layered multicast, among other application |
| 763 |
examples, is discussed in more detail in <a href="#section-11.2">Section 11.2</a>. |
| 764 |
|
| 765 |
When MST is used, any of the three layer combinations above can be |
| 766 |
used for each of the sessions. When an H.264/AVC compatible subset |
| 767 |
of the SVC base layer is transmitted in its own session in MST, the |
| 768 |
packetization of [<a href="./rfc6184" title=""RTP Payload Format for H.264 Video"">RFC6184</a>] must be used, such that [<a href="./rfc6184" title=""RTP Payload Format for H.264 Video"">RFC6184</a>] |
| 769 |
receivers can be part of the MST and receive only this session. For |
| 770 |
MST, this memo defines four different MST-specific packetization |
| 771 |
modes, namely, non-interleaved timestamp (NI-T) based mode, non- |
| 772 |
interleaved CS-DON (NI-C) based mode, non-interleaved combined |
| 773 |
timestamp and CS-DON mode (NI-TC), and interleaved CS-DON (I-C) based |
| 774 |
mode (detailed in <a href="#section-4.5.2">Section 4.5.2</a>). The modes differ depending on |
| 775 |
whether the SVC data are allowed to be interleaved, i.e., to be |
| 776 |
transmitted in an order different than the intended decoding order, |
| 777 |
|
| 778 |
|
| 779 |
|
| 780 |
<span class="grey">Wenger, et al. Standards Track [Page 14]</span></pre> |
| 781 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-15" ></span> |
| 782 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 783 |
|
| 784 |
|
| 785 |
and they also differ in the mechanisms provided in order to recover |
| 786 |
the correct decoding order of the NAL units across the multiple RTP |
| 787 |
sessions. These four MST modes reuse the packetization modes |
| 788 |
introduced in [<a href="./rfc6184" title=""RTP Payload Format for H.264 Video"">RFC6184</a>] for the packetization of NAL units in each of |
| 789 |
their individual RTP sessions. |
| 790 |
|
| 791 |
As the names of the MST packetization modes imply, the NI-T, NI-C, |
| 792 |
and NI-TC modes do not allow interleaved transmission, while the I-C |
| 793 |
mode allows interleaved transmission. With any of the three non- |
| 794 |
interleaved MST packetization modes, legacy [<a href="./rfc6184" title=""RTP Payload Format for H.264 Video"">RFC6184</a>] receivers with |
| 795 |
implementation of the non-interleaved mode specified in [<a href="./rfc6184" title=""RTP Payload Format for H.264 Video"">RFC6184</a>] can |
| 796 |
join a multi-session transmission of SVC, to receive the base RTP |
| 797 |
session encapsulated according to [<a href="./rfc6184" title=""RTP Payload Format for H.264 Video"">RFC6184</a>]. |
| 798 |
|
| 799 |
<span class="h4"><a class="selflink" id="section-1.2.3" href="#section-1.2.3">1.2.3</a>. New Payload Structures</span> |
| 800 |
|
| 801 |
[<a id="ref-RFC6184">RFC6184</a>] specifies three basic payload structures, namely, single |
| 802 |
NAL unit packet, aggregation packet, and fragmentation unit. |
| 803 |
Depending on the basic payload structure, an RTP packet may contain a |
| 804 |
NAL unit not aggregating other NAL units, one or more NAL units |
| 805 |
aggregated in another NAL unit, or a fragment of a NAL unit not |
| 806 |
aggregating other NAL units. Each NAL unit of a type specified in |
| 807 |
[<a href="#ref-H.264" title=""Advanced video coding for generic audiovisual services"">H.264</a>] (i.e., 1 to 23, inclusive) may be carried in its entirety in |
| 808 |
a single NAL unit packet, may be aggregated in an aggregation packet, |
| 809 |
or may be fragmented and carried in a number of fragmentation unit |
| 810 |
packets. To enable aggregation or fragmentation of NAL units while |
| 811 |
still ensuring that the RTP packet payload is only composed of NAL |
| 812 |
units, [<a href="./rfc6184" title=""RTP Payload Format for H.264 Video"">RFC6184</a>] introduced six new NAL unit types (24-29) to be used |
| 813 |
as payload structures, selected from the NAL unit types left |
| 814 |
unspecified in [<a href="#ref-H.264" title=""Advanced video coding for generic audiovisual services"">H.264</a>]. |
| 815 |
|
| 816 |
This memo reuses all the payload structures used in [<a href="./rfc6184" title=""RTP Payload Format for H.264 Video"">RFC6184</a>]. |
| 817 |
Furthermore, three new types of NAL units are defined: payload |
| 818 |
content scalability information (PACSI) NAL unit, empty NAL unit, and |
| 819 |
non-interleaved multi-time aggregation packet (NI-MTAP) (specified in |
| 820 |
Sections <a href="#section-4.9">4.9</a>, <a href="#section-4.10">4.10</a>, and <a href="#section-4.7.1">4.7.1</a>, respectively). |
| 821 |
|
| 822 |
PACSI NAL units may be used for the following purposes: |
| 823 |
|
| 824 |
o To enable MANEs to decide whether to forward, process, or discard |
| 825 |
aggregation packets, by checking in PACSI NAL units the |
| 826 |
scalability information and other characteristics of the |
| 827 |
aggregated NAL units, rather than looking into the aggregated NAL |
| 828 |
units themselves, which are defined by the video coding |
| 829 |
specification. |
| 830 |
|
| 831 |
|
| 832 |
|
| 833 |
|
| 834 |
|
| 835 |
|
| 836 |
<span class="grey">Wenger, et al. Standards Track [Page 15]</span></pre> |
| 837 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-16" ></span> |
| 838 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 839 |
|
| 840 |
|
| 841 |
o To enable correct decoding order recovery in MST using the NI-C or |
| 842 |
NI-TC mode, with the help of the CS-DON information included in |
| 843 |
PACSI NAL units. |
| 844 |
|
| 845 |
o To improve resilience to packet losses, e.g., by utilizing the |
| 846 |
following data or information included in PACSI NAL units: |
| 847 |
repeated Supplemental Enhancement Information (SEI) messages, |
| 848 |
information regarding the start and end of layer representations, |
| 849 |
and the indices to layer representations of the lowest temporal |
| 850 |
subset. |
| 851 |
|
| 852 |
Empty NAL units may be used to enable correct decoding order recovery |
| 853 |
in MST using the NI-T or NI-TC mode. NI-MTAP NAL units may be used |
| 854 |
to aggregate NAL units from multiple access units but without |
| 855 |
interleaving. |
| 856 |
|
| 857 |
<span class="h2"><a class="selflink" id="section-2" href="#section-2">2</a>. Conventions</span> |
| 858 |
|
| 859 |
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", |
| 860 |
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this |
| 861 |
document are to be interpreted as described in <a href="https://www.rfc-editor.org/bcp/bcp14">BCP 14</a>, <a href="./rfc2119">RFC 2119</a> |
| 862 |
[<a href="./rfc2119" title=""Key words for use in RFCs to Indicate Requirement Levels"">RFC2119</a>]. |
| 863 |
|
| 864 |
This specification uses the notion of setting and clearing a bit when |
| 865 |
bit fields are handled. Setting a bit is the same as assigning that |
| 866 |
bit the value of 1 (On). Clearing a bit is the same as assigning |
| 867 |
that bit the value of 0 (Off). |
| 868 |
|
| 869 |
<span class="h2"><a class="selflink" id="section-3" href="#section-3">3</a>. Definitions and Abbreviations</span> |
| 870 |
|
| 871 |
<span class="h3"><a class="selflink" id="section-3.1" href="#section-3.1">3.1</a>. Definitions</span> |
| 872 |
|
| 873 |
This document uses the terms and definitions of [<a href="#ref-H.264" title=""Advanced video coding for generic audiovisual services"">H.264</a>]. <a href="#section-3.1.1">Section</a> |
| 874 |
<a href="#section-3.1.1">3.1.1</a> lists relevant definitions copied from [<a href="#ref-H.264" title=""Advanced video coding for generic audiovisual services"">H.264</a>] for convenience. |
| 875 |
|
| 876 |
When there is discrepancy, the definitions in [<a href="#ref-H.264" title=""Advanced video coding for generic audiovisual services"">H.264</a>] take |
| 877 |
precedence. <a href="#section-3.1.2">Section 3.1.2</a> gives definitions specific to this memo. |
| 878 |
Some of the definitions in <a href="#section-3.1.2">Section 3.1.2</a> are also present in |
| 879 |
[<a href="./rfc6184" title=""RTP Payload Format for H.264 Video"">RFC6184</a>] and copied here with slight adaptations as needed. |
| 880 |
|
| 881 |
<span class="h4"><a class="selflink" id="section-3.1.1" href="#section-3.1.1">3.1.1</a>. Definitions from the SVC Specification</span> |
| 882 |
|
| 883 |
access unit: A set of NAL units always containing exactly one primary |
| 884 |
coded picture. In addition to the primary coded picture, an access |
| 885 |
unit may also contain one or more redundant coded pictures, one |
| 886 |
auxiliary coded picture, or other NAL units not containing slices or |
| 887 |
slice data partitions of a coded picture. The decoding of an access |
| 888 |
unit always results in a decoded picture. |
| 889 |
|
| 890 |
|
| 891 |
|
| 892 |
<span class="grey">Wenger, et al. Standards Track [Page 16]</span></pre> |
| 893 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-17" ></span> |
| 894 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 895 |
|
| 896 |
|
| 897 |
base layer: A bitstream subset that contains all the NAL units with |
| 898 |
the nal_unit_type syntax element equal to 1 or 5 of the bitstream and |
| 899 |
does not contain any NAL unit with the nal_unit_type syntax element |
| 900 |
equal to 14, 15, or 20 and conforms to one or more of the profiles |
| 901 |
specified in Annex A of [<a href="#ref-H.264" title=""Advanced video coding for generic audiovisual services"">H.264</a>]. |
| 902 |
|
| 903 |
base quality layer representation: The layer representation of the |
| 904 |
target dependency representation of an access unit that is associated |
| 905 |
with the quality_id syntax element equal to 0. |
| 906 |
|
| 907 |
coded video sequence: A sequence of access units that consists, in |
| 908 |
decoding order, of an IDR access unit followed by zero or more non- |
| 909 |
IDR access units including all subsequent access units up to but not |
| 910 |
including any subsequent IDR access unit. |
| 911 |
|
| 912 |
dependency representation: A subset of Video Coding Layer (VCL) NAL |
| 913 |
units within an access unit that are associated with the same value |
| 914 |
of the dependency_id syntax element, which is provided as part of the |
| 915 |
NAL unit header or by an associated prefix NAL unit. A dependency |
| 916 |
representation consists of one or more layer representations. |
| 917 |
|
| 918 |
IDR access unit: An access unit in which the primary coded picture is |
| 919 |
an IDR picture. |
| 920 |
|
| 921 |
IDR picture: Instantaneous decoding refresh picture. A coded picture |
| 922 |
in which all slices of the target dependency representation within |
| 923 |
the access unit are I or EI slices that causes the decoding process |
| 924 |
to mark all reference pictures as "unused for reference" immediately |
| 925 |
after decoding the IDR picture. After the decoding of an IDR picture |
| 926 |
all following coded pictures in decoding order can be decoded without |
| 927 |
inter prediction from any picture decoded prior to the IDR picture. |
| 928 |
The first picture of each coded video sequence is an IDR picture. |
| 929 |
|
| 930 |
layer representation: A subset of VCL NAL units within an access unit |
| 931 |
that are associated with the same values of the dependency_id and |
| 932 |
quality_id syntax elements, which are provided as part of the VCL NAL |
| 933 |
unit header or by an associated prefix NAL unit. One or more layer |
| 934 |
representations represent a dependency representation. |
| 935 |
|
| 936 |
prefix NAL unit: A NAL unit with nal_unit_type equal to 14 that |
| 937 |
immediately precedes in decoding order a NAL unit with nal_unit_type |
| 938 |
equal to 1, 5, or 12. The NAL unit that immediately succeeds in |
| 939 |
decoding order the prefix NAL unit is referred to as the associated |
| 940 |
NAL unit. The prefix NAL unit contains data associated with the |
| 941 |
associated NAL unit, which are considered to be part of the |
| 942 |
associated NAL unit. |
| 943 |
|
| 944 |
|
| 945 |
|
| 946 |
|
| 947 |
|
| 948 |
<span class="grey">Wenger, et al. Standards Track [Page 17]</span></pre> |
| 949 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-18" ></span> |
| 950 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 951 |
|
| 952 |
|
| 953 |
reference base picture: A reference picture that is obtained by |
| 954 |
decoding a base quality layer representation with the nal_ref_idc |
| 955 |
syntax element not equal to 0 and the store_ref_base_pic_flag syntax |
| 956 |
element equal to 1 of an access unit and all layer representations of |
| 957 |
the access unit that are referred to by inter-layer prediction of the |
| 958 |
base quality layer representation. A reference base picture is not |
| 959 |
an output of the decoding process, but the samples of a reference |
| 960 |
base picture may be used for inter prediction in the decoding process |
| 961 |
of subsequent pictures in decoding order. Reference base picture is |
| 962 |
a collective term for a reference base field or a reference base |
| 963 |
frame. |
| 964 |
|
| 965 |
scalable bitstream: A bitstream with the property that one or more |
| 966 |
bitstream subsets that are not identical to the scalable bitstream |
| 967 |
form another bitstream that conforms to the SVC specification |
| 968 |
[<a href="#ref-H.264" title=""Advanced video coding for generic audiovisual services"">H.264</a>]. |
| 969 |
|
| 970 |
target dependency representation: The dependency representation of an |
| 971 |
access unit that is associated with the largest value of the |
| 972 |
dependency_id syntax element for all dependency representations of |
| 973 |
the access unit. |
| 974 |
|
| 975 |
target layer representation: The layer representation of the target |
| 976 |
dependency representation of an access unit that is associated with |
| 977 |
the largest value of the quality_id syntax element for all layer |
| 978 |
representations of the target dependency representation of the access |
| 979 |
unit. |
| 980 |
|
| 981 |
<span class="h4"><a class="selflink" id="section-3.1.2" href="#section-3.1.2">3.1.2</a>. Definitions Specific to This Memo</span> |
| 982 |
|
| 983 |
anchor layer representation: An anchor layer representation is such a |
| 984 |
layer representation that, if decoding of the operation point |
| 985 |
corresponding to the layer starts from the access unit containing |
| 986 |
this layer representation, all the following layer representations of |
| 987 |
the layer, in output order, can be correctly decoded. The output |
| 988 |
order is defined in [<a href="#ref-H.264" title=""Advanced video coding for generic audiovisual services"">H.264</a>] as the order in which decoded pictures |
| 989 |
are output from the decoded picture buffer of the decoder. As H.264 |
| 990 |
does not specify the picture display process, this more general term |
| 991 |
is used instead of display order. An anchor layer representation is |
| 992 |
a random access point to the layer the anchor layer representation |
| 993 |
belongs. However, some layer representations, succeeding an anchor |
| 994 |
layer representation in decoding order but preceding the anchor layer |
| 995 |
representation in output order, may refer to earlier layer |
| 996 |
representations for inter prediction, and hence the decoding may be |
| 997 |
incorrect if random access is performed at the anchor layer |
| 998 |
representation. |
| 999 |
|
| 1000 |
|
| 1001 |
|
| 1002 |
|
| 1003 |
|
| 1004 |
<span class="grey">Wenger, et al. Standards Track [Page 18]</span></pre> |
| 1005 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-19" ></span> |
| 1006 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 1007 |
|
| 1008 |
|
| 1009 |
AVC base layer: The subset of the SVC base layer in which all prefix |
| 1010 |
NAL units (type 14) are removed. Note that this is equivalent to the |
| 1011 |
term "base layer" as defined in Annex G of [<a href="#ref-H.264" title=""Advanced video coding for generic audiovisual services"">H.264</a>]. |
| 1012 |
|
| 1013 |
base RTP session: When multi-session transmission is used, the RTP |
| 1014 |
session that carries the RTP stream containing the T0 AVC base layer |
| 1015 |
or the T0 SVC base layer, and zero or more enhancement layers. This |
| 1016 |
RTP session does not depend on any other RTP session as indicated by |
| 1017 |
mechanisms defined in <a href="#section-7.2.3">Section 7.2.3</a>. The base RTP session may carry |
| 1018 |
NAL units of NAL unit type equal to 14 and 15. |
| 1019 |
|
| 1020 |
decoding order number (DON): A field in the payload structure or a |
| 1021 |
derived variable indicating NAL unit decoding order. Values of DON |
| 1022 |
are in the range of 0 to 65535, inclusive. After reaching the |
| 1023 |
maximum value, the value of DON wraps around to 0. Note that this |
| 1024 |
definition also exists in [<a href="./rfc6184" title=""RTP Payload Format for H.264 Video"">RFC6184</a>] in exactly the same form. |
| 1025 |
|
| 1026 |
Empty NAL unit: A NAL unit with NAL unit type equal to 31 and sub- |
| 1027 |
type equal to 1. An empty NAL unit consists of only the two-byte NAL |
| 1028 |
unit header with an empty payload. |
| 1029 |
|
| 1030 |
enhancement RTP session: When multi-session transmission is used, an |
| 1031 |
RTP session that is not the base RTP session. An enhancement RTP |
| 1032 |
session typically contains an RTP stream that depends on at least one |
| 1033 |
other RTP session as indicated by mechanisms defined in <a href="#section-7.2.3">Section</a> |
| 1034 |
<a href="#section-7.2.3">7.2.3</a>. A lower RTP session to an enhancement RTP session is an RTP |
| 1035 |
session on which the enhancement RTP session depends. The lowest RTP |
| 1036 |
session for a receiver is the RTP session that does not depend on any |
| 1037 |
other RTP session received by the receiver. The highest RTP session |
| 1038 |
for a receiver is the RTP session on which no other RTP session |
| 1039 |
received by the receiver depends. |
| 1040 |
|
| 1041 |
cross-session decoding order number (CS-DON): A derived variable |
| 1042 |
indicating NAL unit decoding order number over all NAL units within |
| 1043 |
all the session-multiplexed RTP sessions that carry the same SVC |
| 1044 |
bitstream. |
| 1045 |
|
| 1046 |
default level: The level indicated by the profile-level-id parameter. |
| 1047 |
In Session Description Protocol (SDP) Offer/Answer, the level is |
| 1048 |
downgradable, i.e., the answer may either use the default level or a |
| 1049 |
lower level. Note that this definition also exists in [<a href="./rfc6184" title=""RTP Payload Format for H.264 Video"">RFC6184</a>] in a |
| 1050 |
slightly different form. |
| 1051 |
|
| 1052 |
default sub-profile: The subset of coding tools, which may be all |
| 1053 |
coding tools of one profile or the common subset of coding tools of |
| 1054 |
more than one profile, indicated by the profile-level-id parameter. |
| 1055 |
In SDP Offer/Answer, the default sub-profile must be used in a |
| 1056 |
|
| 1057 |
|
| 1058 |
|
| 1059 |
|
| 1060 |
<span class="grey">Wenger, et al. Standards Track [Page 19]</span></pre> |
| 1061 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-20" ></span> |
| 1062 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 1063 |
|
| 1064 |
|
| 1065 |
symmetric manner, i.e., the answer must either use the same sub- |
| 1066 |
profile as the offer or reject the offer. Note that this definition |
| 1067 |
also exists in [<a href="./rfc6184" title=""RTP Payload Format for H.264 Video"">RFC6184</a>] in a slightly different form. |
| 1068 |
|
| 1069 |
enhancement layer: A layer in which at least one of the values of |
| 1070 |
dependency_id or quality_id is higher than 0, or a layer in which |
| 1071 |
none of the NAL units is associated with the value of temporal_id |
| 1072 |
equal to 0. An operation point constructed using the maximum |
| 1073 |
temporal_id, dependency_id, and quality_id values associated with an |
| 1074 |
enhancement layer may or may not conform to one or more of the |
| 1075 |
profiles specified in Annex A of [<a href="#ref-H.264" title=""Advanced video coding for generic audiovisual services"">H.264</a>]. |
| 1076 |
|
| 1077 |
H.264/AVC compatible: The property of a bitstream subset of |
| 1078 |
conforming to one or more of the profiles specified in Annex A of |
| 1079 |
[<a href="#ref-H.264" title=""Advanced video coding for generic audiovisual services"">H.264</a>]. |
| 1080 |
|
| 1081 |
intra layer representation: A layer representation that contains |
| 1082 |
only slices that use intra prediction, and hence do not refer to any |
| 1083 |
earlier layer representation in decoding order in the same layer. |
| 1084 |
Note that in SVC intra prediction includes intra-layer intra |
| 1085 |
prediction as well as inter-layer intra prediction. |
| 1086 |
|
| 1087 |
layer: A bitstream subset in which all NAL units of type 1, 5, 12, |
| 1088 |
14, or 20 have the same values of dependency_id and quality_id, |
| 1089 |
either directly through their NAL unit header (for NAL units of type |
| 1090 |
14 or 20) or through association to a prefix (type 14) NAL unit (for |
| 1091 |
NAL unit type 1, 5, or 12). A layer may contain NAL units associated |
| 1092 |
with more than one values of temporal_id. |
| 1093 |
|
| 1094 |
media-aware network element (MANE): A network element, such as a |
| 1095 |
middlebox or application layer gateway that is capable of parsing |
| 1096 |
certain aspects of the RTP payload headers or the RTP payload and |
| 1097 |
reacting to their contents. Note that this definition also exists in |
| 1098 |
[<a href="./rfc6184" title=""RTP Payload Format for H.264 Video"">RFC6184</a>] in exactly the same form. |
| 1099 |
|
| 1100 |
Informative note: The concept of a MANE goes beyond normal routers |
| 1101 |
or gateways in that a MANE has to be aware of the signaling (e.g., |
| 1102 |
to learn about the payload type mappings of the media streams), |
| 1103 |
and in that it has to be trusted when working with Secure Real- |
| 1104 |
time Transport Protocol (SRTP). The advantage of using MANEs is |
| 1105 |
that they allow packets to be dropped according to the needs of |
| 1106 |
the media coding. For example, if a MANE has to drop packets due |
| 1107 |
to congestion on a certain link, it can identify and remove those |
| 1108 |
packets whose elimination produces the least adverse effect on the |
| 1109 |
user experience. After dropping packets, MANEs must rewrite RTCP |
| 1110 |
packets to match the changes to the RTP packet stream as specified |
| 1111 |
in <a href="./rfc3550#section-7">Section 7 of [RFC3550]</a>. |
| 1112 |
|
| 1113 |
|
| 1114 |
|
| 1115 |
|
| 1116 |
<span class="grey">Wenger, et al. Standards Track [Page 20]</span></pre> |
| 1117 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-21" ></span> |
| 1118 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 1119 |
|
| 1120 |
|
| 1121 |
multi-session transmission: The transmission mode in which the SVC |
| 1122 |
stream is transmitted over multiple RTP sessions. Dependency between |
| 1123 |
RTP sessions MUST be signaled according to <a href="#section-7.2.3">Section 7.2.3</a> of this |
| 1124 |
memo. |
| 1125 |
|
| 1126 |
NAL unit decoding order: A NAL unit order that conforms to the |
| 1127 |
constraints on NAL unit order given in Section G.7.4.1.2 in [<a href="#ref-H.264" title=""Advanced video coding for generic audiovisual services"">H.264</a>]. |
| 1128 |
Note that this definition also exists in [<a href="./rfc6184" title=""RTP Payload Format for H.264 Video"">RFC6184</a>] in a slightly |
| 1129 |
different form. |
| 1130 |
|
| 1131 |
NALU-time: The value that the RTP timestamp would have if the NAL |
| 1132 |
unit would be transported in its own RTP packet. Note that this |
| 1133 |
definition also exists in [<a href="./rfc6184" title=""RTP Payload Format for H.264 Video"">RFC6184</a>] in exactly the same form. |
| 1134 |
|
| 1135 |
operation point: An operation point is identified by a set of values |
| 1136 |
of temporal_id, dependency_id, and quality_id. A bitstream |
| 1137 |
corresponding to an operation point can be constructed by removing |
| 1138 |
all NAL units associated with a higher value of dependency_id, and |
| 1139 |
all NAL units associated with the same value of dependency_id but |
| 1140 |
higher values of quality_id or temporal_id. An operation point |
| 1141 |
bitstream conforms to at least one of the profiles defined in Annex A |
| 1142 |
or G of [<a href="#ref-H.264" title=""Advanced video coding for generic audiovisual services"">H.264</a>], and offers a representation of the original video |
| 1143 |
signal at a certain fidelity. |
| 1144 |
|
| 1145 |
Informative note: Additional NAL units may be removed (with lower |
| 1146 |
dependency_id or same dependency_id but lower quality_id) if they |
| 1147 |
are not required for decoding the bitstream at the particular |
| 1148 |
operation point. The resulting bitstream, however, may no longer |
| 1149 |
conform to any of the profiles defined in Annex A or G of [<a href="#ref-H.264" title=""Advanced video coding for generic audiovisual services"">H.264</a>]. |
| 1150 |
|
| 1151 |
operation point representation: The set of all NAL units of an |
| 1152 |
operation point within the same access unit. |
| 1153 |
|
| 1154 |
RTP packet stream: A sequence of RTP packets with increasing sequence |
| 1155 |
numbers (except for wrap-around), identical payload type and |
| 1156 |
identical SSRC (Synchronization Source), carried in one RTP session. |
| 1157 |
Within the scope of this memo, one RTP packet stream is utilized to |
| 1158 |
transport one or more layers. |
| 1159 |
|
| 1160 |
single-session transmission: The transmission mode in which the SVC |
| 1161 |
bitstream is transmitted over a single RTP session. |
| 1162 |
|
| 1163 |
SVC base layer: The layer that includes all NAL units associated with |
| 1164 |
dependency_id and quality_id values both equal to 0, including prefix |
| 1165 |
NAL units (NAL unit type 14). |
| 1166 |
|
| 1167 |
|
| 1168 |
|
| 1169 |
|
| 1170 |
|
| 1171 |
|
| 1172 |
<span class="grey">Wenger, et al. Standards Track [Page 21]</span></pre> |
| 1173 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-22" ></span> |
| 1174 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 1175 |
|
| 1176 |
|
| 1177 |
SVC enhancement layer: A layer in which at least one of the values of |
| 1178 |
dependency_id or quality_id is higher than 0. An operation point |
| 1179 |
constructed using the maximum dependency_id and quality_id values and |
| 1180 |
any temporal_id value associated with an SVC enhancement layer does |
| 1181 |
not conform to any of the profiles specified in Annex A of [<a href="#ref-H.264" title=""Advanced video coding for generic audiovisual services"">H.264</a>]. |
| 1182 |
|
| 1183 |
SVC NAL unit: A NAL unit of NAL unit type 14, 15, or 20 as specified |
| 1184 |
in Annex G of [<a href="#ref-H.264" title=""Advanced video coding for generic audiovisual services"">H.264</a>]. |
| 1185 |
|
| 1186 |
SVC NAL unit header: A four-byte header resulting from the addition |
| 1187 |
of a three-byte SVC-specific header extension added in NAL unit types |
| 1188 |
14 and 20. |
| 1189 |
|
| 1190 |
SVC RTP session: Either the base RTP session or an enhancement RTP |
| 1191 |
session. |
| 1192 |
|
| 1193 |
T0 AVC base layer: A subset of the AVC base layer constructed by |
| 1194 |
removing all VCL NAL units associated with temporal_id values higher |
| 1195 |
than 0 and non-VCL NAL units and SEI messages associated only with |
| 1196 |
the VCL NAL units being removed. |
| 1197 |
|
| 1198 |
T0 SVC base layer: A subset of the SVC base layer constructed by |
| 1199 |
removing all VCL NAL units associated with temporal_id values higher |
| 1200 |
than 0 as well as prefix NAL units, non-VCL NAL units, and SEI |
| 1201 |
messages associated only with the VCL NAL units being removed. |
| 1202 |
|
| 1203 |
transmission order: The order of packets in ascending RTP sequence |
| 1204 |
number order (in modulo arithmetic). Within an aggregation packet, |
| 1205 |
the NAL unit transmission order is the same as the order of |
| 1206 |
appearance of NAL units in the packet. Note that this definition |
| 1207 |
also exists in [<a href="./rfc6184" title=""RTP Payload Format for H.264 Video"">RFC6184</a>] in exactly the same form. |
| 1208 |
|
| 1209 |
<span class="h3"><a class="selflink" id="section-3.2" href="#section-3.2">3.2</a>. Abbreviations</span> |
| 1210 |
|
| 1211 |
In addition to the abbreviations defined in [<a href="./rfc6184" title=""RTP Payload Format for H.264 Video"">RFC6184</a>], the following |
| 1212 |
abbreviations are used in this memo. |
| 1213 |
|
| 1214 |
CGS: Coarse-Grain Scalability |
| 1215 |
CS-DON: Cross-Session Decoding Order Number |
| 1216 |
MGS: Medium-Grain Scalability |
| 1217 |
MST: Multi-Session Transmission |
| 1218 |
PACSI: Payload Content Scalability Information |
| 1219 |
SST: Single-Session Transmission |
| 1220 |
SNR: Signal-to-Noise Ratio |
| 1221 |
SVC: Scalable Video Coding |
| 1222 |
|
| 1223 |
|
| 1224 |
|
| 1225 |
|
| 1226 |
|
| 1227 |
|
| 1228 |
<span class="grey">Wenger, et al. Standards Track [Page 22]</span></pre> |
| 1229 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-23" ></span> |
| 1230 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 1231 |
|
| 1232 |
|
| 1233 |
<span class="h2"><a class="selflink" id="section-4" href="#section-4">4</a>. RTP Payload Format</span> |
| 1234 |
|
| 1235 |
<span class="h3"><a class="selflink" id="section-4.1" href="#section-4.1">4.1</a>. RTP Header Usage</span> |
| 1236 |
|
| 1237 |
In addition to <a href="./rfc6184#section-5.1">Section 5.1 of [RFC6184]</a>, the following rules apply. |
| 1238 |
|
| 1239 |
o Setting of the M bit: |
| 1240 |
|
| 1241 |
The M bit of an RTP packet for which the packet payload is an NI-MTAP |
| 1242 |
MUST be equal to 1 if the last NAL unit, in decoding order, of the |
| 1243 |
access unit associated with the RTP timestamp is contained in the |
| 1244 |
packet. |
| 1245 |
|
| 1246 |
o Setting of the RTP timestamp: |
| 1247 |
|
| 1248 |
For an RTP packet for which the packet payload is an empty NAL unit, |
| 1249 |
the RTP timestamp must be set according to <a href="#section-4.10">Section 4.10</a>. |
| 1250 |
|
| 1251 |
For an RTP packet for which the packet payload is a PACSI NAL unit, |
| 1252 |
the RTP timestamp MUST be equal to the NALU-time of the next non- |
| 1253 |
PACSI NAL unit in transmission order. Recall that the NALU-time of a |
| 1254 |
NAL unit in an MTAP is defined in [<a href="./rfc6184" title=""RTP Payload Format for H.264 Video"">RFC6184</a>] as the value that the RTP |
| 1255 |
timestamp would have if that NAL unit would be transported in its own |
| 1256 |
RTP packet. |
| 1257 |
|
| 1258 |
o Setting of the SSRC: |
| 1259 |
|
| 1260 |
For both SST and MST, the SSRC values MUST be set according to |
| 1261 |
[<a href="./rfc3550" title=""RTP: A Transport Protocol for Real-Time Applications"">RFC3550</a>]. |
| 1262 |
|
| 1263 |
<span class="h3"><a class="selflink" id="section-4.2" href="#section-4.2">4.2</a>. NAL Unit Extension and Header Usage</span> |
| 1264 |
|
| 1265 |
<span class="h4"><a class="selflink" id="section-4.2.1" href="#section-4.2.1">4.2.1</a>. NAL Unit Extension</span> |
| 1266 |
|
| 1267 |
This memo specifies a NAL unit extension mechanism to allow for |
| 1268 |
introduction of new types of NAL units, beyond the three NAL unit |
| 1269 |
types left undefined in [<a href="./rfc6184" title=""RTP Payload Format for H.264 Video"">RFC6184</a>] (i.e., 0, 30, and 31). The |
| 1270 |
extension mechanism utilizes the NAL unit type value 31 and is |
| 1271 |
specified as follows. When the NAL unit type value is equal to 31, |
| 1272 |
the one-byte NAL unit header consisting of the F, NRI, and Type |
| 1273 |
fields as specified in <a href="#section-1.1.3">Section 1.1.3</a> is extended by one additional |
| 1274 |
octet, which consists of a 5-bit field named Subtype and three 1-bit |
| 1275 |
fields named J, K, and L, respectively. The additional octet is |
| 1276 |
shown in the following figure. |
| 1277 |
|
| 1278 |
|
| 1279 |
|
| 1280 |
|
| 1281 |
|
| 1282 |
|
| 1283 |
|
| 1284 |
<span class="grey">Wenger, et al. Standards Track [Page 23]</span></pre> |
| 1285 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-24" ></span> |
| 1286 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 1287 |
|
| 1288 |
|
| 1289 |
+---------------+ |
| 1290 |
|0|1|2|3|4|5|6|7| |
| 1291 |
+-+-+-+-+-+-+-+-+ |
| 1292 |
| Subtype |J|K|L| |
| 1293 |
+---------------+ |
| 1294 |
|
| 1295 |
The Subtype value determines the (extended) NAL unit type of this NAL |
| 1296 |
unit. The interpretation of the fields J, K, and L depends on the |
| 1297 |
Subtype. The semantics of the fields are as follows. |
| 1298 |
|
| 1299 |
When Subtype is equal to 1, the NAL unit is an empty NAL unit as |
| 1300 |
specified in <a href="#section-4.10">Section 4.10</a>. When Subtype is equal to 2, the NAL unit |
| 1301 |
is an NI-MTAP NAL unit as specified in <a href="#section-4.7.1">Section 4.7.1</a>. All other |
| 1302 |
values of Subtype (0, 3-31) are reserved for future extensions, and |
| 1303 |
receivers MUST ignore the entire NAL unit when Subtype is equal to |
| 1304 |
any of these reserved values. |
| 1305 |
|
| 1306 |
<span class="h4"><a class="selflink" id="section-4.2.2" href="#section-4.2.2">4.2.2</a>. NAL Unit Header Usage</span> |
| 1307 |
|
| 1308 |
The structure and semantics of the NAL unit header according to the |
| 1309 |
H.264 specification [<a href="#ref-H.264" title=""Advanced video coding for generic audiovisual services"">H.264</a>] were introduced in <a href="#section-1.1.3">Section 1.1.3</a>. This |
| 1310 |
section specifies the extended semantics of the NAL unit header |
| 1311 |
fields F, NRI, I, PRID, DID, QID, TID, U, and D, according to this |
| 1312 |
memo. When the Type field is equal to 31, the semantics of the |
| 1313 |
fields in the extension NAL unit header were specified in <a href="#section-4.2.1">Section</a> |
| 1314 |
<a href="#section-4.2.1">4.2.1</a>. |
| 1315 |
|
| 1316 |
The semantics of F specified in <a href="./rfc6184#section-5.3">Section 5.3 of [RFC6184]</a> also apply |
| 1317 |
in this memo. That is, a value of 0 for F indicates that the NAL |
| 1318 |
unit type octet and payload should not contain bit errors or other |
| 1319 |
syntax violations, whereas a value of 1 for F indicates that the NAL |
| 1320 |
unit type octet and payload may contain bit errors or other syntax |
| 1321 |
violations. MANEs SHOULD set the F bit to indicate bit errors in the |
| 1322 |
NAL unit. |
| 1323 |
|
| 1324 |
For NRI, for a bitstream conforming to one of the profiles defined in |
| 1325 |
Annex A of [<a href="#ref-H.264" title=""Advanced video coding for generic audiovisual services"">H.264</a>] and transported using [<a href="./rfc6184" title=""RTP Payload Format for H.264 Video"">RFC6184</a>], the semantics |
| 1326 |
specified in <a href="./rfc6184#section-5.3">Section 5.3 of [RFC6184]</a> apply, i.e., NRI also indicates |
| 1327 |
the relative importance of NAL units. For a bitstream conforming to |
| 1328 |
one of the profiles defined in Annex G of [<a href="#ref-H.264" title=""Advanced video coding for generic audiovisual services"">H.264</a>] and transported |
| 1329 |
using this memo, in addition to the semantics specified in Annex G of |
| 1330 |
[<a href="#ref-H.264" title=""Advanced video coding for generic audiovisual services"">H.264</a>], NRI also indicates the relative importance of NAL units |
| 1331 |
within a layer. |
| 1332 |
|
| 1333 |
For I, in addition to the semantics specified in Annex G of [<a href="#ref-H.264" title=""Advanced video coding for generic audiovisual services"">H.264</a>], |
| 1334 |
according to this memo, MANEs MAY use this information to protect NAL |
| 1335 |
units with I equal to 1 better than NAL units with I equal to 0. |
| 1336 |
MANEs MAY also utilize information of NAL units with I equal to 1 to |
| 1337 |
|
| 1338 |
|
| 1339 |
|
| 1340 |
<span class="grey">Wenger, et al. Standards Track [Page 24]</span></pre> |
| 1341 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-25" ></span> |
| 1342 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 1343 |
|
| 1344 |
|
| 1345 |
decide when to forward more packets for an RTP packet stream. For |
| 1346 |
example, when it is detected that spatial layer switching has |
| 1347 |
happened such that the operation point has changed to a higher value |
| 1348 |
of DID, MANEs MAY start to forward NAL units with the higher value of |
| 1349 |
DID only after forwarding a NAL unit with I equal to 1 with the |
| 1350 |
higher value of DID. |
| 1351 |
|
| 1352 |
Note that, in the context of this section, "protecting a NAL unit" |
| 1353 |
means any RTP or network transport mechanism that could improve the |
| 1354 |
probability of successful delivery of the packet conveying the NAL |
| 1355 |
unit, including applying a Quality of Service (QoS) enabled network, |
| 1356 |
Forward Error Correction (FEC), retransmissions, and advanced |
| 1357 |
scheduling behavior, whenever possible. |
| 1358 |
|
| 1359 |
For PRID, the semantics specified in Annex G of [<a href="#ref-H.264" title=""Advanced video coding for generic audiovisual services"">H.264</a>] apply. Note |
| 1360 |
that MANEs implementing unequal error protection MAY use this |
| 1361 |
information to protect NAL units with smaller PRID values better than |
| 1362 |
those with larger PRID values, for example, by including only the |
| 1363 |
more important NAL units in a FEC protection mechanism. The |
| 1364 |
importance for the decoding process decreases as the PRID value |
| 1365 |
increases. |
| 1366 |
|
| 1367 |
For DID, QID, or TID, in addition to the semantics specified in Annex |
| 1368 |
G of [<a href="#ref-H.264" title=""Advanced video coding for generic audiovisual services"">H.264</a>], according to this memo, values of DID, QID, or TID |
| 1369 |
indicate the relative importance in their respective dimension. A |
| 1370 |
lower value of DID, QID, or TID indicates a higher importance if the |
| 1371 |
other two components are identical. MANEs MAY use this information |
| 1372 |
to protect more important NAL units better than less important NAL |
| 1373 |
units. |
| 1374 |
|
| 1375 |
For U, in addition to the semantics specified in Annex G of [<a href="#ref-H.264" title=""Advanced video coding for generic audiovisual services"">H.264</a>], |
| 1376 |
according to this memo, MANEs MAY use this information to protect NAL |
| 1377 |
units with U equal to 1 better than NAL units with U equal to 0. |
| 1378 |
|
| 1379 |
For D, in addition to the semantics specified in Annex G of [<a href="#ref-H.264" title=""Advanced video coding for generic audiovisual services"">H.264</a>], |
| 1380 |
according to this memo, MANEs MAY use this information to determine |
| 1381 |
whether a given NAL unit is required for successfully decoding a |
| 1382 |
certain Operation Point of the SVC bitstream, hence to decide whether |
| 1383 |
to forward the NAL unit. |
| 1384 |
|
| 1385 |
<span class="h3"><a class="selflink" id="section-4.3" href="#section-4.3">4.3</a>. Payload Structures</span> |
| 1386 |
|
| 1387 |
The NAL unit structure is central to H.264/AVC, [<a href="./rfc6184" title=""RTP Payload Format for H.264 Video"">RFC6184</a>], as well as |
| 1388 |
SVC and this memo. In H.264/AVC and SVC, all coded bits for |
| 1389 |
representing a video signal are encapsulated in NAL units. In |
| 1390 |
[<a href="./rfc6184" title=""RTP Payload Format for H.264 Video"">RFC6184</a>], each RTP packet payload is structured as a NAL unit, which |
| 1391 |
contains one or a part of one NAL unit specified in H.264/AVC, or |
| 1392 |
aggregates one or more NAL units specified in H.264/AVC. |
| 1393 |
|
| 1394 |
|
| 1395 |
|
| 1396 |
<span class="grey">Wenger, et al. Standards Track [Page 25]</span></pre> |
| 1397 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-26" ></span> |
| 1398 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 1399 |
|
| 1400 |
|
| 1401 |
[<a id="ref-RFC6184">RFC6184</a>] specifies three basic payload structures (in <a href="./rfc6184#section-5.2">Section 5.2 of |
| 1402 |
[RFC6184]</a>): single NAL unit packet, aggregation packet, fragmentation |
| 1403 |
unit, and six new types (24 to 29) of NAL units. The value of the |
| 1404 |
Type field of the RTP packet payload header (i.e., the first byte of |
| 1405 |
the payload) may be equal to any value from 1 to 23 for a single NAL |
| 1406 |
unit packet, any value from 24 to 27 for an aggregation packet, and |
| 1407 |
28 or 29 for a fragmentation unit. |
| 1408 |
|
| 1409 |
In addition to the NAL unit types defined originally for H.264/AVC, |
| 1410 |
SVC defines three new NAL unit types specifically for SVC: coded |
| 1411 |
slice in scalable extension NAL units (type 20), prefix NAL units |
| 1412 |
(type 14), and subset sequence parameter set NAL units (type 15), as |
| 1413 |
described in <a href="#section-1.1">Section 1.1</a>. |
| 1414 |
|
| 1415 |
This memo further introduces three new types of NAL units, PACSI NAL |
| 1416 |
unit (NAL unit type 30) as specified in <a href="#section-4.9">Section 4.9</a>, empty NAL unit |
| 1417 |
(type 31, subtype 1) as specified in <a href="#section-4.10">Section 4.10</a>, and NI-MTAP NAL |
| 1418 |
unit (type 31, subtype 2) as specified in <a href="#section-4.7.1">Section 4.7.1</a>. |
| 1419 |
|
| 1420 |
The RTP packet payload structure in [<a href="./rfc6184" title=""RTP Payload Format for H.264 Video"">RFC6184</a>] is maintained with |
| 1421 |
slight extensions in this memo, as follows. Each RTP packet payload |
| 1422 |
is still structured as a NAL unit, which contains one or a part of |
| 1423 |
one NAL unit specified in H.264/AVC and SVC, or contains one PACSI |
| 1424 |
NAL unit or one empty NAL unit, or aggregates zero or more NAL units |
| 1425 |
specified in H.264/AVC and SVC, zero or one PACSI NAL unit, and zero |
| 1426 |
or more empty NAL units. |
| 1427 |
|
| 1428 |
In this memo, one of the three basic payload structures, |
| 1429 |
fragmentation unit, remains the same as in [<a href="./rfc6184" title=""RTP Payload Format for H.264 Video"">RFC6184</a>], and the other |
| 1430 |
two, single NAL unit packet and aggregation packet, are extended as |
| 1431 |
follows. The value of the Type field of the payload header may be |
| 1432 |
equal to any value from 1 to 23, inclusive, and 30 to 31, inclusive, |
| 1433 |
for a single NAL unit packet, and any value from 24 to 27, inclusive, |
| 1434 |
and 31, for an aggregation packet. When the Type field of the |
| 1435 |
payload header is equal to 31 and the Subtype field of the payload |
| 1436 |
header is equal to 2, the packet is an aggregation packet (containing |
| 1437 |
an NI-MTAP NAL unit). When the Type field of the payload header is |
| 1438 |
equal to 31 and the Subtype field of the payload header is equal to |
| 1439 |
1, the packet is a single NAL unit packet (containing an empty NAL |
| 1440 |
unit). |
| 1441 |
|
| 1442 |
Note that, in this memo, the length of the payload header varies |
| 1443 |
depending on the value of the Type field in the first byte of the RTP |
| 1444 |
packet payload. If the value is equal to 14, 20, or 30, the first |
| 1445 |
four bytes of the packet payload form the payload header; otherwise, |
| 1446 |
if the value is equal to 31, the first two bytes of the payload form |
| 1447 |
the payload header; otherwise, the payload header is the first byte |
| 1448 |
of the packet payload. |
| 1449 |
|
| 1450 |
|
| 1451 |
|
| 1452 |
<span class="grey">Wenger, et al. Standards Track [Page 26]</span></pre> |
| 1453 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-27" ></span> |
| 1454 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 1455 |
|
| 1456 |
|
| 1457 |
Table 1 lists the NAL unit types introduced in SVC and this memo and |
| 1458 |
where they are described in this memo. Table 2 summarizes the basic |
| 1459 |
payload structure types for all NAL unit types when they are directly |
| 1460 |
used as RTP packet payloads according to this memo. Table 3 |
| 1461 |
summarizes the NAL unit types allowed to be aggregated (i.e., used as |
| 1462 |
aggregation units in aggregation packets) or fragmented (i.e., |
| 1463 |
carried in fragmentation units) according to this memo. |
| 1464 |
|
| 1465 |
Table 1. NAL unit types introduced in SVC and this memo |
| 1466 |
|
| 1467 |
Type Subtype NAL Unit Name Section Numbers |
| 1468 |
----------------------------------------------------------- |
| 1469 |
14 - Prefix NAL unit 1.1 |
| 1470 |
15 - Subset sequence parameter set 1.1 |
| 1471 |
20 - Coded slice in scalable extension 1.1 |
| 1472 |
30 - PACSI NAL unit 4.9 |
| 1473 |
31 0 reserved 4.2.1 |
| 1474 |
31 1 Empty NAL unit 4.10 |
| 1475 |
31 2 NI-MTAP 4.7.1 |
| 1476 |
31 3-31 reserved 4.2.1 |
| 1477 |
|
| 1478 |
Table 2. Basic payload structure types for all NAL unit |
| 1479 |
types when they are directly used as RTP packet payloads |
| 1480 |
|
| 1481 |
Type Subtype Basic Payload Structure |
| 1482 |
------------------------------------------ |
| 1483 |
0 - reserved |
| 1484 |
1-23 - Single NAL Unit Packet |
| 1485 |
24-27 - Aggregation Packet |
| 1486 |
28-29 - Fragmentation Unit |
| 1487 |
30 - Single NAL Unit Packet |
| 1488 |
31 0 reserved |
| 1489 |
31 1 Single NAL Unit Packet |
| 1490 |
31 2 Aggregation Packet |
| 1491 |
31 3-31 reserved |
| 1492 |
|
| 1493 |
|
| 1494 |
|
| 1495 |
|
| 1496 |
|
| 1497 |
|
| 1498 |
|
| 1499 |
|
| 1500 |
|
| 1501 |
|
| 1502 |
|
| 1503 |
|
| 1504 |
|
| 1505 |
|
| 1506 |
|
| 1507 |
|
| 1508 |
<span class="grey">Wenger, et al. Standards Track [Page 27]</span></pre> |
| 1509 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-28" ></span> |
| 1510 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 1511 |
|
| 1512 |
|
| 1513 |
Table 3. Summary of the NAL unit types allowed to be |
| 1514 |
aggregated or fragmented (yes = allowed, no = disallowed, |
| 1515 |
- = not applicable/not specified) |
| 1516 |
|
| 1517 |
Type Subtype STAP-A STAP-B MTAP16 MTAP24 FU-A FU-B NI-MTAP |
| 1518 |
------------------------------------------------------------- |
| 1519 |
0 - - - - - - - - |
| 1520 |
1-23 - yes yes yes yes yes yes yes |
| 1521 |
24-29 - no no no no no no no |
| 1522 |
30 - yes yes yes yes no no yes |
| 1523 |
31 0 - - - - - - - |
| 1524 |
31 1 yes no no no no no yes |
| 1525 |
31 2 no no no no no no no |
| 1526 |
31 3-31 - - - - - - - |
| 1527 |
|
| 1528 |
<span class="h3"><a class="selflink" id="section-4.4" href="#section-4.4">4.4</a>. Transmission Modes</span> |
| 1529 |
|
| 1530 |
This memo enables transmission of an SVC bitstream over one or more |
| 1531 |
RTP sessions. If only one RTP session is used for transmission of |
| 1532 |
the SVC bitstream, the transmission mode is referred to as single- |
| 1533 |
session transmission (SST); otherwise (more than one RTP session is |
| 1534 |
used for transmission of the SVC bitstream), the transmission mode is |
| 1535 |
referred to as multi-session transmission (MST). |
| 1536 |
|
| 1537 |
SST SHOULD be used for point-to-point unicast scenarios, while MST |
| 1538 |
SHOULD be used for point-to-multipoint multicast scenarios where |
| 1539 |
different receivers requires different operation points of the same |
| 1540 |
SVC bitstream, to improve bandwidth utilizing efficiency. |
| 1541 |
|
| 1542 |
If the OPTIONAL mst-mode media type parameter (see <a href="#section-7.1">Section 7.1</a>) is |
| 1543 |
not present, SST MUST be used; otherwise (mst-mode is present), MST |
| 1544 |
MUST be used. |
| 1545 |
|
| 1546 |
<span class="h3"><a class="selflink" id="section-4.5" href="#section-4.5">4.5</a>. Packetization Modes</span> |
| 1547 |
|
| 1548 |
<span class="h4"><a class="selflink" id="section-4.5.1" href="#section-4.5.1">4.5.1</a>. Packetization Modes for Single-Session Transmission</span> |
| 1549 |
|
| 1550 |
When SST is in use, <a href="./rfc6184#section-5.4">Section 5.4 of [RFC6184]</a> applies with the |
| 1551 |
following extensions. |
| 1552 |
|
| 1553 |
The packetization modes specified in <a href="./rfc6184#section-5.4">Section 5.4 of [RFC6184]</a>, |
| 1554 |
namely, single NAL unit mode, non-interleaved mode, and interleaved |
| 1555 |
mode, are also referred to as session packetization modes. Table 4 |
| 1556 |
summarizes the allowed session packetization modes for SST. |
| 1557 |
|
| 1558 |
|
| 1559 |
|
| 1560 |
|
| 1561 |
|
| 1562 |
|
| 1563 |
|
| 1564 |
<span class="grey">Wenger, et al. Standards Track [Page 28]</span></pre> |
| 1565 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-29" ></span> |
| 1566 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 1567 |
|
| 1568 |
|
| 1569 |
Table 4. Summary of allowed session packetization modes |
| 1570 |
(denoted as "Session Mode" for simplicity) for SST (yes = |
| 1571 |
allowed, no = disallowed) |
| 1572 |
|
| 1573 |
Session Mode Allowed |
| 1574 |
------------------------------------- |
| 1575 |
Single NAL Unit Mode yes |
| 1576 |
Non-Interleaved Mode yes |
| 1577 |
Interleaved Mode yes |
| 1578 |
|
| 1579 |
For NAL unit types in the range of 0 to 29, inclusive, the NAL unit |
| 1580 |
types allowed to be directly used as packet payloads for each session |
| 1581 |
packetization mode are the same as specified in <a href="./rfc6184#section-5.4">Section 5.4 of |
| 1582 |
[RFC6184]</a>. For other NAL unit types, which are newly introduced in |
| 1583 |
this memo, the NAL unit types allowed to be directly used as packet |
| 1584 |
payloads for each session packetization mode are summarized in Table |
| 1585 |
5. |
| 1586 |
|
| 1587 |
Table 5. New NAL unit types allowed to be directly used |
| 1588 |
as packet payloads for each session packetization mode |
| 1589 |
(yes = allowed, no = disallowed, - = not applicable/not specified) |
| 1590 |
|
| 1591 |
Type Subtype Single NAL Non-Interleaved Interleaved |
| 1592 |
Unit Mode Mode Mode |
| 1593 |
------------------------------------------------------------- |
| 1594 |
30 - yes no no |
| 1595 |
31 0 - - - |
| 1596 |
31 1 yes yes no |
| 1597 |
31 2 no yes no |
| 1598 |
31 3-31 - - - |
| 1599 |
|
| 1600 |
<span class="h4"><a class="selflink" id="section-4.5.2" href="#section-4.5.2">4.5.2</a>. Packetization Modes for Multi-Session Transmission</span> |
| 1601 |
|
| 1602 |
For MST, this memo specifies four MST packetization modes: |
| 1603 |
|
| 1604 |
o Non-interleaved timestamp based mode (NI-T); |
| 1605 |
|
| 1606 |
o Non-interleaved cross-session decoding order number (CS-DON) based |
| 1607 |
mode (NI-C); |
| 1608 |
|
| 1609 |
o Non-interleaved combined timestamp and CS-DON mode (NI-TC); and |
| 1610 |
|
| 1611 |
o Interleaved CS-DON (I-C) mode. |
| 1612 |
|
| 1613 |
These four modes differ in two ways. First, they differ in terms of |
| 1614 |
whether NAL units are required to be transmitted within each RTP |
| 1615 |
session in decoding order (i.e., non-interleaved), or they are |
| 1616 |
allowed to be transmitted in a different order (i.e., interleaved). |
| 1617 |
|
| 1618 |
|
| 1619 |
|
| 1620 |
<span class="grey">Wenger, et al. Standards Track [Page 29]</span></pre> |
| 1621 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-30" ></span> |
| 1622 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 1623 |
|
| 1624 |
|
| 1625 |
Second, they differ in the mechanisms they provide in order to |
| 1626 |
recover the correct decoding order of the NAL units across all RTP |
| 1627 |
sessions involved. |
| 1628 |
|
| 1629 |
The NI-T, NI-C, and NI-TC modes do not allow interleaving, and are |
| 1630 |
thus targeted for systems that require relatively low end-to-end |
| 1631 |
latency, e.g., conversational systems. The I-C mode allows |
| 1632 |
interleaving and is thus targeted for systems that do not require |
| 1633 |
very low end-to-end latency. The benefits of interleaving are the |
| 1634 |
same as that of the interleaved mode specified in [<a href="./rfc6184" title=""RTP Payload Format for H.264 Video"">RFC6184</a>]. |
| 1635 |
|
| 1636 |
The NI-T mode uses timestamps to recover the decoding order of NAL |
| 1637 |
units, whereas the NI-C and I-C modes both use the CS-DON mechanism |
| 1638 |
(explained later) to do so. The NI-TC mode provides both timestamps |
| 1639 |
and the CS-DON method; receivers in this case may choose to use |
| 1640 |
either method for performing decoding order recovery. The MST |
| 1641 |
packetization mode in use MUST be signaled by the value of the |
| 1642 |
OPTIONAL mst-mode media type parameter. The used MST packetization |
| 1643 |
mode governs which session packetization modes are allowed in the |
| 1644 |
associated RTP sessions, which in turn govern which NAL unit types |
| 1645 |
are allowed to be directly used as RTP packet payloads. |
| 1646 |
|
| 1647 |
Table 6 summarizes the allowed session packetization modes for NI-T, |
| 1648 |
NI-C, and NI-TC. Table 7 summarizes the allowed session |
| 1649 |
packetization modes for I-C. |
| 1650 |
|
| 1651 |
Table 6. Summary of allowed session packetization modes |
| 1652 |
(denoted as "Session Mode" for simplicity) for NI-T, NI-C, and |
| 1653 |
NI-TC (yes = allowed, no = disallowed) |
| 1654 |
|
| 1655 |
Session Mode Base Session Enhancement Session |
| 1656 |
----------------------------------------------------------- |
| 1657 |
Single NAL Unit Mode yes no |
| 1658 |
Non-Interleaved Mode yes yes |
| 1659 |
Interleaved Mode no no |
| 1660 |
|
| 1661 |
Table 7. Summary of allowed session packetization modes |
| 1662 |
(denoted as "Session Mode" for simplicity) for I-C |
| 1663 |
(yes = allowed, no = disallowed) |
| 1664 |
|
| 1665 |
Session Mode Base Session Enhancement Session |
| 1666 |
----------------------------------------------------------- |
| 1667 |
Single NAL Unit Mode no no |
| 1668 |
Non-Interleaved Mode no no |
| 1669 |
Interleaved Mode yes yes |
| 1670 |
|
| 1671 |
|
| 1672 |
|
| 1673 |
|
| 1674 |
|
| 1675 |
|
| 1676 |
<span class="grey">Wenger, et al. Standards Track [Page 30]</span></pre> |
| 1677 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-31" ></span> |
| 1678 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 1679 |
|
| 1680 |
|
| 1681 |
For NAL unit types in the range of 0 to 29, inclusive, the NAL unit |
| 1682 |
types allowed to be directly used as packet payloads for each session |
| 1683 |
packetization mode are the same as specified in <a href="./rfc6184#section-5.4">Section 5.4 of |
| 1684 |
[RFC6184]</a>. For other NAL unit types, which are newly introduced in |
| 1685 |
this memo, the NAL unit types allowed to be directly used as packet |
| 1686 |
payloads for each allowed session packetization mode for NI-T, NI-C, |
| 1687 |
NI-TC, and I-C are summarized in Tables 8, 9, 10, and 11, |
| 1688 |
respectively. |
| 1689 |
|
| 1690 |
Table 8. New NAL unit types allowed to be directly used |
| 1691 |
as packet payloads for each allowed session packetization |
| 1692 |
mode when NI-T is in use (yes = allowed, no = disallowed, |
| 1693 |
- = not applicable/not specified) |
| 1694 |
|
| 1695 |
Type Subtype Single NAL Non-Interleaved |
| 1696 |
Unit Mode Mode |
| 1697 |
--------------------------------------------------- |
| 1698 |
30 - yes no |
| 1699 |
31 0 - - |
| 1700 |
31 1 yes yes |
| 1701 |
31 2 no yes |
| 1702 |
31 3-31 - - |
| 1703 |
|
| 1704 |
Table 9. New NAL unit types allowed to be directly used |
| 1705 |
as packet payloads for each allowed session packetization |
| 1706 |
mode when NI-C is in use (yes = allowed, no = disallowed, |
| 1707 |
- = not applicable/not specified) |
| 1708 |
|
| 1709 |
Type Subtype Single NAL Non-Interleaved |
| 1710 |
Unit Mode Mode |
| 1711 |
--------------------------------------------------- |
| 1712 |
30 - yes yes |
| 1713 |
31 0 - - |
| 1714 |
31 1 no no |
| 1715 |
31 2 no yes |
| 1716 |
31 3-31 - - |
| 1717 |
|
| 1718 |
|
| 1719 |
|
| 1720 |
|
| 1721 |
|
| 1722 |
|
| 1723 |
|
| 1724 |
|
| 1725 |
|
| 1726 |
|
| 1727 |
|
| 1728 |
|
| 1729 |
|
| 1730 |
|
| 1731 |
|
| 1732 |
<span class="grey">Wenger, et al. Standards Track [Page 31]</span></pre> |
| 1733 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-32" ></span> |
| 1734 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 1735 |
|
| 1736 |
|
| 1737 |
Table 10. New NAL unit types allowed to be directly used |
| 1738 |
as packet payloads for each allowed session packetization |
| 1739 |
mode when NI-TC is in use (yes = allowed, no = disallowed, |
| 1740 |
- = not applicable/not specified) |
| 1741 |
|
| 1742 |
Type Subtype Single NAL Non-Interleaved |
| 1743 |
Unit Mode Mode |
| 1744 |
--------------------------------------------------- |
| 1745 |
30 - yes yes |
| 1746 |
31 0 - - |
| 1747 |
31 1 yes yes |
| 1748 |
31 2 no yes |
| 1749 |
31 3-31 - - |
| 1750 |
|
| 1751 |
Table 11. New NAL unit types allowed to be directly used |
| 1752 |
as packet payloads for the allowed session packetization |
| 1753 |
mode when I-C is in use (yes = allowed, no = disallowed, |
| 1754 |
- = not applicable/not specified) |
| 1755 |
|
| 1756 |
Type Subtype Interleaved Mode |
| 1757 |
------------------------------------ |
| 1758 |
30 - no |
| 1759 |
31 0 - |
| 1760 |
31 1 no |
| 1761 |
31 2 no |
| 1762 |
31 3-31 - |
| 1763 |
|
| 1764 |
When MST is in use and the MST packetization mode in use is NI-C, |
| 1765 |
empty NAL units (type 31, subtype 1) MUST NOT be used, i.e., no RTP |
| 1766 |
packet is allowed to contain one or more empty NAL units. |
| 1767 |
|
| 1768 |
When MST is in use and the MST packetization mode in use is I-C, both |
| 1769 |
empty NAL units (type 31, subtype 1) and NI-MTAP NAL units (type 31, |
| 1770 |
subtype 2) MUST NOT be used, i.e., no RTP packet is allowed to |
| 1771 |
contain one or more empty NAL units or an NI-MTAP NAL unit. |
| 1772 |
|
| 1773 |
<span class="h3"><a class="selflink" id="section-4.6" href="#section-4.6">4.6</a>. Single NAL Unit Packets</span> |
| 1774 |
|
| 1775 |
<a href="./rfc6184#section-5.6">Section 5.6 of [RFC6184]</a> applies with the following extensions. |
| 1776 |
|
| 1777 |
The payload of a single NAL unit packet MAY be a PACSI NAL unit (Type |
| 1778 |
30) or an empty NAL unit (Type 31 and Subtype 1), in addition to a |
| 1779 |
NAL unit with NAL unit type equal to any value from 1 to 23, |
| 1780 |
inclusive. |
| 1781 |
|
| 1782 |
|
| 1783 |
|
| 1784 |
|
| 1785 |
|
| 1786 |
|
| 1787 |
|
| 1788 |
<span class="grey">Wenger, et al. Standards Track [Page 32]</span></pre> |
| 1789 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-33" ></span> |
| 1790 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 1791 |
|
| 1792 |
|
| 1793 |
If the Type field of the first byte of the payload is not equal to |
| 1794 |
31, the payload header is the first byte of the payload. Otherwise, |
| 1795 |
(the Type field of the first byte of the payload is equal to 31), the |
| 1796 |
payload header is the first two bytes of the payload. |
| 1797 |
|
| 1798 |
<span class="h3"><a class="selflink" id="section-4.7" href="#section-4.7">4.7</a>. Aggregation Packets</span> |
| 1799 |
|
| 1800 |
In addition to <a href="./rfc6184#section-5.7">Section 5.7 of [RFC6184]</a>, the following applies in |
| 1801 |
this memo. |
| 1802 |
|
| 1803 |
<span class="h4"><a class="selflink" id="section-4.7.1" href="#section-4.7.1">4.7.1</a>. Non-Interleaved Multi-Time Aggregation Packets (NI-MTAPs)</span> |
| 1804 |
|
| 1805 |
One new NAL unit type introduced in this memo is the non-interleaved |
| 1806 |
multi-time aggregation packet (NI-MTAP). An NI-MTAP consists of one |
| 1807 |
or more non-interleaved multi-time aggregation units. |
| 1808 |
|
| 1809 |
The NAL units contained in NI-MTAPs MUST be aggregated in decoding |
| 1810 |
order. |
| 1811 |
|
| 1812 |
A non-interleaved multi-time aggregation unit for the NI-MTAP |
| 1813 |
consists of 16 bits of unsigned size information of the following NAL |
| 1814 |
unit (in network byte order), and 16 bits (in network byte order) of |
| 1815 |
timestamp offset (TS offset) for the NAL unit. The structure is |
| 1816 |
presented in Figure 1. The starting or ending position of an |
| 1817 |
aggregation unit within a packet may or may not be on a 32-bit word |
| 1818 |
boundary. The NAL units in the NI-MTAP are ordered in NAL unit |
| 1819 |
decoding order. |
| 1820 |
|
| 1821 |
The Type field of the NI-MTAP MUST be set equal to "31". |
| 1822 |
|
| 1823 |
The F bit MUST be set to 0 if all the F bits of the aggregated NAL |
| 1824 |
units are zero; otherwise, it MUST be set to 1. |
| 1825 |
|
| 1826 |
The value of NRI MUST be the maximum value of NRI across all NAL |
| 1827 |
units carried in the NI-MTAP packet. |
| 1828 |
|
| 1829 |
The field Subtype MUST be equal to 2. |
| 1830 |
|
| 1831 |
If the field J is equal to 1, the optional DON field MUST be present |
| 1832 |
for each of the non-interleaved multi-time aggregation units. For |
| 1833 |
SST, the J field MUST be equal to 0. For MST, in the NI-T mode the J |
| 1834 |
field MUST be equal to 0, whereas in the NI-C or NI-TC mode the J |
| 1835 |
field MUST be equal to 1. When the NI-C or NI-TC mode is in use, the |
| 1836 |
DON field, when present, MUST represent the CS-DON value for the |
| 1837 |
particular NAL unit as defined in <a href="#section-6.2.2">Section 6.2.2</a>. |
| 1838 |
|
| 1839 |
The fields K and L MUST be both equal to 0. |
| 1840 |
|
| 1841 |
|
| 1842 |
|
| 1843 |
|
| 1844 |
<span class="grey">Wenger, et al. Standards Track [Page 33]</span></pre> |
| 1845 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-34" ></span> |
| 1846 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 1847 |
|
| 1848 |
|
| 1849 |
0 1 2 3 |
| 1850 |
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 |
| 1851 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |
| 1852 |
: NAL unit size | TS offset | |
| 1853 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |
| 1854 |
| DON (optional) | | |
| 1855 |
|-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ NAL unit | |
| 1856 |
| | |
| 1857 |
| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |
| 1858 |
| : |
| 1859 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |
| 1860 |
|
| 1861 |
Figure 1. Non-interleaved multi-time aggregation unit for NI-MTAP |
| 1862 |
|
| 1863 |
Let TS be the RTP timestamp of the packet carrying the NAL unit. |
| 1864 |
Recall that the NALU-time of a NAL unit in an MTAP is defined in |
| 1865 |
[<a href="./rfc6184" title=""RTP Payload Format for H.264 Video"">RFC6184</a>] as the value that the RTP timestamp would have if that NAL |
| 1866 |
unit would be transported in its own RTP packet. The timestamp |
| 1867 |
offset field MUST be set to a value equal to the value of the |
| 1868 |
following formula: |
| 1869 |
|
| 1870 |
if NALU-time >= TS, TS offset = NALU-time - TS |
| 1871 |
else, TS offset = NALU-time + (2^32 - TS) |
| 1872 |
|
| 1873 |
For the "earliest" multi-time aggregation unit in an NI-MTAP, the |
| 1874 |
timestamp offset MUST be zero. Hence, the RTP timestamp of the NI- |
| 1875 |
MTAP itself is identical to the earliest NALU-time. |
| 1876 |
|
| 1877 |
Informative note: The "earliest" multi-time aggregation unit is |
| 1878 |
the one that would have the smallest extended RTP timestamp among |
| 1879 |
all the aggregation units of an NI-MTAP if the aggregation units |
| 1880 |
were encapsulated in single NAL unit packets. An extended |
| 1881 |
timestamp is a timestamp that has more than 32 bits and is capable |
| 1882 |
of counting the wraparound of the timestamp field, thus enabling |
| 1883 |
one to determine the smallest value if the timestamp wraps. Such |
| 1884 |
an "earliest" aggregation unit may or may not be the first one in |
| 1885 |
the order in which the aggregation units are encapsulated in an |
| 1886 |
NI-MTAP. The "earliest" NAL unit need not be the same as the |
| 1887 |
first NAL unit in the NAL unit decoding order either. |
| 1888 |
|
| 1889 |
Figure 2 presents an example of an RTP packet that contains an NI- |
| 1890 |
MTAP that contains two non-interleaved multi-time aggregation units, |
| 1891 |
labeled as 1 and 2 in the figure. |
| 1892 |
|
| 1893 |
|
| 1894 |
|
| 1895 |
|
| 1896 |
|
| 1897 |
|
| 1898 |
|
| 1899 |
|
| 1900 |
<span class="grey">Wenger, et al. Standards Track [Page 34]</span></pre> |
| 1901 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-35" ></span> |
| 1902 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 1903 |
|
| 1904 |
|
| 1905 |
0 1 2 3 |
| 1906 |
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 |
| 1907 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |
| 1908 |
| RTP Header | |
| 1909 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |
| 1910 |
|F|NRI| Type | Subtype |J|K|L| | |
| 1911 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | |
| 1912 |
| | |
| 1913 |
| Non-interleaved multi-time aggregation unit #1 | |
| 1914 |
: : |
| 1915 |
| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |
| 1916 |
| | Non-interleaved multi-time | |
| 1917 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | |
| 1918 |
| aggregation unit #2 | |
| 1919 |
: : |
| 1920 |
| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |
| 1921 |
| :...OPTIONAL RTP padding | |
| 1922 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |
| 1923 |
|
| 1924 |
Figure 2. An RTP packet including an NI-MTAP containing two |
| 1925 |
non-interleaved multi-time aggregation units |
| 1926 |
|
| 1927 |
<span class="h3"><a class="selflink" id="section-4.8" href="#section-4.8">4.8</a>. Fragmentation Units (FUs)</span> |
| 1928 |
|
| 1929 |
<a href="./rfc6184#section-5.8">Section 5.8 of [RFC6184]</a> applies. |
| 1930 |
|
| 1931 |
Informative note: In case a NAL unit with the four-byte SVC NAL |
| 1932 |
unit header is fragmented, the three-byte SVC-specific header |
| 1933 |
extension is considered as part of the NAL unit payload. That is, |
| 1934 |
the three-byte SVC-specific header extension is only available in |
| 1935 |
the first fragment of the fragmented NAL unit. |
| 1936 |
|
| 1937 |
<span class="h3"><a class="selflink" id="section-4.9" href="#section-4.9">4.9</a>. Payload Content Scalability Information (PACSI) NAL Unit</span> |
| 1938 |
|
| 1939 |
Another new type of NAL unit specified in this memo is the payload |
| 1940 |
content scalability information (PACSI) NAL unit. The Type field of |
| 1941 |
PACSI NAL units MUST be equal to 30 (a NAL unit type value left |
| 1942 |
unspecified in [<a href="#ref-H.264" title=""Advanced video coding for generic audiovisual services"">H.264</a>] and [<a href="./rfc6184" title=""RTP Payload Format for H.264 Video"">RFC6184</a>]). A PACSI NAL unit MAY be |
| 1943 |
carried in a single NAL unit packet or an aggregation packet, and |
| 1944 |
MUST NOT be fragmented. |
| 1945 |
|
| 1946 |
PACSI NAL units may be used for the following purposes: |
| 1947 |
|
| 1948 |
o To enable MANEs to decide whether to forward, process, or discard |
| 1949 |
aggregation packets, by checking in PACSI NAL units the |
| 1950 |
scalability information and other characteristics of the |
| 1951 |
|
| 1952 |
|
| 1953 |
|
| 1954 |
|
| 1955 |
|
| 1956 |
<span class="grey">Wenger, et al. Standards Track [Page 35]</span></pre> |
| 1957 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-36" ></span> |
| 1958 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 1959 |
|
| 1960 |
|
| 1961 |
aggregated NAL units, rather than looking into the aggregated NAL |
| 1962 |
units themselves, which are defined by the video coding |
| 1963 |
specification; |
| 1964 |
|
| 1965 |
o To enable correct decoding order recovery in MST using the NI-C or |
| 1966 |
NI-TC mode, with the help of the CS-DON information included in |
| 1967 |
PACSI NAL units; and |
| 1968 |
|
| 1969 |
o To improve resilience to packet losses, e.g., by utilizing the |
| 1970 |
following data or information included in PACSI NAL units: |
| 1971 |
repeated Supplemental Enhancement Information (SEI) messages, |
| 1972 |
information regarding the start and end of layer representations, |
| 1973 |
and the indices to layer representations of the lowest temporal |
| 1974 |
subset. |
| 1975 |
|
| 1976 |
PACSI NAL units MAY be ignored in the NI-T mode without affecting the |
| 1977 |
decoding order recovery process. |
| 1978 |
|
| 1979 |
When a PACSI NAL unit is present in an aggregation packet, the |
| 1980 |
following applies. |
| 1981 |
|
| 1982 |
o The PACSI NAL unit MUST be the first aggregated NAL unit in the |
| 1983 |
aggregation packet. |
| 1984 |
|
| 1985 |
o There MUST be at least one additional aggregated NAL unit in the |
| 1986 |
aggregation packet. |
| 1987 |
|
| 1988 |
o The RTP header fields and the payload header fields of the |
| 1989 |
aggregation packet are set as if the PACSI NAL unit was not |
| 1990 |
included in the aggregation packet. |
| 1991 |
|
| 1992 |
o If the aggregation packet is an MTAP16, MTAP24, or NI-MTAP with |
| 1993 |
the J field equal to 1, the decoding order number (DON) for the |
| 1994 |
PACSI NAL unit MUST be set to indicate that the PACSI NAL unit has |
| 1995 |
an identical DON to the first NAL unit in decoding order among the |
| 1996 |
remaining NAL units in the aggregation packet. |
| 1997 |
|
| 1998 |
When a PACSI NAL unit is included in a single NAL unit packet, it is |
| 1999 |
associated with the next non-PACSI NAL unit in transmission order, |
| 2000 |
and the RTP header fields of the packet are set as if the next non- |
| 2001 |
PACSI NAL unit in transmission order was included in a single NAL |
| 2002 |
unit packet. |
| 2003 |
|
| 2004 |
The PACSI NAL unit structure is as follows. The first four octets |
| 2005 |
are exactly the same as the four-byte SVC NAL unit header discussed |
| 2006 |
in <a href="#section-1.1.3">Section 1.1.3</a>. They are followed by one octet containing several |
| 2007 |
flags, then five optional octets, and finally zero or more SEI NAL |
| 2008 |
units. Each SEI NAL unit is preceded by a 16-bit unsigned size field |
| 2009 |
|
| 2010 |
|
| 2011 |
|
| 2012 |
<span class="grey">Wenger, et al. Standards Track [Page 36]</span></pre> |
| 2013 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-37" ></span> |
| 2014 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 2015 |
|
| 2016 |
|
| 2017 |
(in network byte order) that indicates the size of the following NAL |
| 2018 |
unit in bytes (excluding these two octets, but including the NAL unit |
| 2019 |
header octet of the SEI NAL unit). Figure 3 illustrates the PACSI |
| 2020 |
NAL unit structure and an example of a PACSI NAL unit containing two |
| 2021 |
SEI NAL units. |
| 2022 |
|
| 2023 |
0 1 2 3 |
| 2024 |
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 |
| 2025 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |
| 2026 |
|F|NRI| Type |R|I| PRID |N| DID | QID | TID |U|D|O| RR| |
| 2027 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |
| 2028 |
|X|Y|T|A|P|C|S|E| TL0PICIDX (o) | IDRPICID (o) | |
| 2029 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |
| 2030 |
| DONC (o) | NAL unit size 1 | |
| 2031 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |
| 2032 |
| | |
| 2033 |
| SEI NAL unit 1 | |
| 2034 |
| | |
| 2035 |
| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |
| 2036 |
| | NAL unit size 2 | | |
| 2037 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | |
| 2038 |
| | |
| 2039 |
| SEI NAL unit 2 | |
| 2040 |
| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |
| 2041 |
| | |
| 2042 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |
| 2043 |
|
| 2044 |
Figure 3. PACSI NAL unit structure. Fields suffixed by |
| 2045 |
"(o)" are OPTIONAL. |
| 2046 |
|
| 2047 |
The bits A, P, and C are specified only if the bit X is equal to 1. |
| 2048 |
The bits S and E are specified, and the fields TL0PICIDX and IDRPICID |
| 2049 |
are present, only if the bit Y is equal to 1. The field DONC is |
| 2050 |
present only if the bit T is equal to 1. The field T MUST be equal |
| 2051 |
to 0 if the PACSI NAL unit is contained in an STAP-B, MTAP16, MTAP24, |
| 2052 |
or NI-MTAP with the J field equal to 1. |
| 2053 |
|
| 2054 |
The values of the fields in PACSI NAL unit MUST be set as follows. |
| 2055 |
|
| 2056 |
o The F bit MUST be set to 1 if the F bit in at least one of the |
| 2057 |
remaining NAL units in the aggregation packet is equal to 1 (when |
| 2058 |
the PACSI NAL unit is included in an aggregation packet) or if the |
| 2059 |
next non-PACSI NAL unit in transmission order has the F bit equal |
| 2060 |
to 1 (when the PACSI NAL unit is included in a single NAL unit |
| 2061 |
packet). Otherwise, the F bit MUST be set to 0. |
| 2062 |
|
| 2063 |
|
| 2064 |
|
| 2065 |
|
| 2066 |
|
| 2067 |
|
| 2068 |
<span class="grey">Wenger, et al. Standards Track [Page 37]</span></pre> |
| 2069 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-38" ></span> |
| 2070 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 2071 |
|
| 2072 |
|
| 2073 |
o The NRI field MUST be set to the highest value of NRI field among |
| 2074 |
all the remaining NAL units in the aggregation packet (when the |
| 2075 |
PACSI NAL unit is included in an aggregation packet) or the value |
| 2076 |
of the NRI field of the next non-PACSI NAL unit in transmission |
| 2077 |
order (when the PACSI NAL unit is included in a single NAL unit |
| 2078 |
packet). |
| 2079 |
|
| 2080 |
o The Type field MUST be set to 30. |
| 2081 |
|
| 2082 |
o The R bit MUST be set to 1. Receivers MUST ignore the value of R. |
| 2083 |
|
| 2084 |
o The I bit MUST be set to 1 if the I bit of at least one of the |
| 2085 |
remaining NAL units in the aggregation packet is equal to 1 (when |
| 2086 |
the PACSI NAL unit is included in an aggregation packet) or if the |
| 2087 |
I bit of the next non-PACSI NAL unit in transmission order is |
| 2088 |
equal to 1 (when the PACSI NAL unit is included in a single NAL |
| 2089 |
unit packet). Otherwise, the I bit MUST be set to 0. |
| 2090 |
|
| 2091 |
o The PRID field MUST be set to the lowest value of the PRID values |
| 2092 |
of the remaining NAL units in the aggregation packet (when the |
| 2093 |
PACSI NAL unit is included in an aggregation packet) or the PRID |
| 2094 |
value of the next non-PACSI NAL unit in transmission order (when |
| 2095 |
the PACSI NAL unit is included in a single NAL unit packet). |
| 2096 |
|
| 2097 |
o The N bit MUST be set to 1 if the N bit of all the remaining NAL |
| 2098 |
units in the aggregation packet is equal to 1 (when the PACSI NAL |
| 2099 |
unit is included in an aggregation packet) or if the N bit of the |
| 2100 |
next non-PACSI NAL unit in transmission order is equal to 1 (when |
| 2101 |
the PACSI NAL unit is included in a single NAL unit packet). |
| 2102 |
Otherwise, the N bit MUST be set to 0. |
| 2103 |
|
| 2104 |
o The DID field MUST be set to the lowest value of the DID values of |
| 2105 |
the remaining NAL units in the aggregation packet (when the PACSI |
| 2106 |
NAL unit is included in an aggregation packet) or the DID value of |
| 2107 |
the next non-PACSI NAL unit in transmission order (when the PACSI |
| 2108 |
NAL unit is included in a single NAL unit packet). |
| 2109 |
|
| 2110 |
o The QID field MUST be set to the lowest value of the QID values of |
| 2111 |
the remaining NAL units with the lowest value of DID in the |
| 2112 |
aggregation packet (when the PACSI NAL unit is included in an |
| 2113 |
aggregation packet) or the QID value of the next non-PACSI NAL |
| 2114 |
unit in transmission order (when the PACSI NAL unit is included in |
| 2115 |
a single NAL unit packet). |
| 2116 |
|
| 2117 |
o The TID field MUST be set to the lowest value of the TID values of |
| 2118 |
the remaining NAL units with the lowest value of DID in the |
| 2119 |
aggregation packet (when the PACSI NAL unit is included in an |
| 2120 |
|
| 2121 |
|
| 2122 |
|
| 2123 |
|
| 2124 |
<span class="grey">Wenger, et al. Standards Track [Page 38]</span></pre> |
| 2125 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-39" ></span> |
| 2126 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 2127 |
|
| 2128 |
|
| 2129 |
aggregation packet) or the TID value of the next non-PACSI NAL |
| 2130 |
unit in transmission order (when the PACSI NAL unit is included in |
| 2131 |
a single NAL unit packet). |
| 2132 |
|
| 2133 |
o The U bit MUST be set to 1 if the U bit of at least one of the |
| 2134 |
remaining NAL units in the aggregation packet is equal to 1 (when |
| 2135 |
the PACSI NAL unit is included in an aggregation packet) or if the |
| 2136 |
U bit of the next non-PACSI NAL unit in transmission order is |
| 2137 |
equal to 1 (when the PACSI NAL unit is included in a single NAL |
| 2138 |
unit packet). Otherwise, the U bit MUST be set to 0. |
| 2139 |
|
| 2140 |
o The D bit MUST be set to 1 if the D value of all the remaining NAL |
| 2141 |
units in the aggregation packet is equal to 1 (when the PACSI NAL |
| 2142 |
unit is included in an aggregation packet) or if the D bit of the |
| 2143 |
next non-PACSI NAL unit in transmission order is equal to 1 (when |
| 2144 |
the PACSI NAL unit is included in a single NAL unit packet). |
| 2145 |
Otherwise, the D bit MUST be set to 0. |
| 2146 |
|
| 2147 |
o The O bit MUST be set to 1 if the O bit of at least one of the |
| 2148 |
remaining NAL units in the aggregation packet is equal to 1 (when |
| 2149 |
the PACSI NAL unit is included in an aggregation packet) or if the |
| 2150 |
O bit of the next non-PACSI NAL unit in transmission order is |
| 2151 |
equal to 1 (when the PACSI NAL unit is included in a single NAL |
| 2152 |
unit packet). Otherwise, the O bit MUST be set to 0. |
| 2153 |
|
| 2154 |
o The RR field MUST be set to "11" (in binary form). Receivers MUST |
| 2155 |
ignore the value of RR. |
| 2156 |
|
| 2157 |
o If the X bit is equal to 1, the bits A, P, and C are specified as |
| 2158 |
below. Otherwise, the bits A, P, and C are unspecified, and |
| 2159 |
receivers MUST ignore the values of these bits. The X bit SHOULD |
| 2160 |
be identical for all the PACSI NAL units in all the RTP sessions |
| 2161 |
carrying the same SVC bitstream. |
| 2162 |
|
| 2163 |
o If the Y bit is equal to 1, the OPTIONAL fields TL0PICIDX and |
| 2164 |
IDRPICID MUST be present and specified as below, and the bits S |
| 2165 |
and E are also specified as below. Otherwise, the fields |
| 2166 |
TL0PICIDX and IDRPICID MUST NOT be present, while the S and E bits |
| 2167 |
are unspecified and receivers MUST ignore the values of these |
| 2168 |
bits. The Y bit MUST be identical for all the PACSI NAL units in |
| 2169 |
all the RTP sessions carrying the same SVC bitstream. The Y bit |
| 2170 |
MUST be equal to 0 when the parameter packetization-mode is equal |
| 2171 |
to 2. |
| 2172 |
|
| 2173 |
o If the T bit is equal to 1, the OPTIONAL field DONC MUST be |
| 2174 |
present and specified as below. Otherwise, the field DONC MUST |
| 2175 |
NOT be present. The field T MUST be equal to 0 if the PACSI NAL |
| 2176 |
unit is contained in an STAP-B, MTAP16, MTAP24, or NI-MTAP. |
| 2177 |
|
| 2178 |
|
| 2179 |
|
| 2180 |
<span class="grey">Wenger, et al. Standards Track [Page 39]</span></pre> |
| 2181 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-40" ></span> |
| 2182 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 2183 |
|
| 2184 |
|
| 2185 |
o The A bit MUST be set to 1 if at least one of the remaining NAL |
| 2186 |
units in the aggregation packet belongs to an anchor layer |
| 2187 |
representation (when the PACSI NAL unit is included in an |
| 2188 |
aggregation packet) or if the next non-PACSI NAL unit in |
| 2189 |
transmission order belongs to an anchor layer representation (when |
| 2190 |
the PACSI NAL unit is included in a single NAL unit packet). |
| 2191 |
Otherwise, the A bit MUST be set to 0. |
| 2192 |
|
| 2193 |
Informative note: The A bit indicates whether CGS or spatial layer |
| 2194 |
switching at a non-IDR layer representation (a layer |
| 2195 |
representation with nal_unit_type not equal to 5 and idr_flag not |
| 2196 |
equal to 1) can be performed. With some picture coding structures |
| 2197 |
a non-IDR intra layer representation can be used for random |
| 2198 |
access. Compared to using only IDR layer representations, higher |
| 2199 |
coding efficiency can be achieved. The H.264/AVC or SVC solution |
| 2200 |
to indicate the random accessibility of a non-IDR intra layer |
| 2201 |
representation is using a recovery point SEI message. The A bit |
| 2202 |
offers direct access to this information, without having to parse |
| 2203 |
the recovery point SEI message, which may be buried deeply in an |
| 2204 |
SEI NAL unit. Furthermore, the SEI message may or may not be |
| 2205 |
present in the bitstream. |
| 2206 |
|
| 2207 |
o The P bit MUST be set to 1 if all the remaining NAL units in the |
| 2208 |
aggregation packet have redundant_pic_cnt greater than 0 (when the |
| 2209 |
PACSI NAL unit is included in an aggregation packet) or the next |
| 2210 |
non-PACSI NAL unit in transmission order has redundant_pic_cnt |
| 2211 |
greater than 0 (when the PACSI NAL unit is included in a single |
| 2212 |
NAL unit packet). Otherwise, the P bit MUST be set to 0. |
| 2213 |
|
| 2214 |
Informative note: The P bit indicates whether a packet can be |
| 2215 |
discarded because it contains only redundant slice NAL units. |
| 2216 |
Without this bit, the corresponding information can be obtained |
| 2217 |
from the syntax element redundant_pic_cnt, which is contained in |
| 2218 |
the variable-length coded slice header. |
| 2219 |
|
| 2220 |
o The C bit MUST be set to 1 if at least one of the remaining NAL |
| 2221 |
units in the aggregation packet belongs to an intra layer |
| 2222 |
representation (when the PACSI NAL unit is included in an |
| 2223 |
aggregation packet) or if the next non-PACSI NAL unit in |
| 2224 |
transmission order belongs to an intra layer representation (when |
| 2225 |
the PACSI NAL unit is included in a single NAL unit packet). |
| 2226 |
Otherwise, the C bit MUST be set to 0. |
| 2227 |
|
| 2228 |
Informative note: The C bit indicates whether a packet contains |
| 2229 |
intra slices, which may be the only packets to be forwarded, e.g., |
| 2230 |
when the network conditions are particularly adverse. |
| 2231 |
|
| 2232 |
|
| 2233 |
|
| 2234 |
|
| 2235 |
|
| 2236 |
<span class="grey">Wenger, et al. Standards Track [Page 40]</span></pre> |
| 2237 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-41" ></span> |
| 2238 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 2239 |
|
| 2240 |
|
| 2241 |
o The S bit MUST be set to 1, if the first NAL unit following the |
| 2242 |
PACSI NAL unit in an aggregation packet is the first VCL NAL unit, |
| 2243 |
in decoding order, of a layer representation (when the PACSI NAL |
| 2244 |
unit is included in an aggregation packet) or if the next non- |
| 2245 |
PACSI NAL unit in transmission order is the first VCL NAL unit, in |
| 2246 |
decoding order, of a layer representation(when the PACSI NAL unit |
| 2247 |
is included in a single NAL unit packet). Otherwise, the S bit |
| 2248 |
MUST be set to 0. |
| 2249 |
|
| 2250 |
o The E bit MUST be set to 1, if the last NAL unit following the |
| 2251 |
PACSI NAL unit in an aggregation packet is the last VCL NAL unit, |
| 2252 |
in decoding order, of a layer representation (when the PACSI NAL |
| 2253 |
unit is included in an aggregation packet) or if the next non- |
| 2254 |
PACSI NAL unit in transmission order is the last VCL NAL unit, in |
| 2255 |
decoding order, of a layer representation (when the PACSI NAL unit |
| 2256 |
is included in a single NAL unit packet). Otherwise, the E bit |
| 2257 |
MUST be set to 0. |
| 2258 |
|
| 2259 |
Informative note: In an aggregation packet it is always possible |
| 2260 |
to detect the beginning or end of a layer representation by |
| 2261 |
detecting changes in the values of dependency_id, quality_id, and |
| 2262 |
temporal_id in NAL unit headers, except from the first and last |
| 2263 |
NAL units of a packet. The S or E bits are used to provide this |
| 2264 |
information, for both single NAL unit and aggregation packets, so |
| 2265 |
that previous or following packets do not have to be examined. |
| 2266 |
This enables MANEs to detect slice loss and take proper action |
| 2267 |
such as requesting a retransmission as soon as possible, as well |
| 2268 |
as to allow efficient playout buffer handling similarly to the M |
| 2269 |
bit present in the RTP header. The M bit in the RTP header still |
| 2270 |
indicates the end of an access unit, not the end of a layer |
| 2271 |
representation. |
| 2272 |
|
| 2273 |
o When present, the TL0PICIDX field MUST be set to equal to |
| 2274 |
tl0_dep_rep_idx as specified in Annex G of [<a href="#ref-H.264" title=""Advanced video coding for generic audiovisual services"">H.264</a>] for the layer |
| 2275 |
representation containing the first NAL unit following the PACSI |
| 2276 |
NAL unit in the aggregation packet (when the PACSI NAL unit is |
| 2277 |
included in an aggregation packet) or containing the next non- |
| 2278 |
PACSI NAL unit in transmission order (when the PACSI NAL unit is |
| 2279 |
included in a single NAL unit packet). |
| 2280 |
|
| 2281 |
o When present, the IDRPICID field MUST be set to equal to |
| 2282 |
effective_idr_pic_id as specified in Annex G of [<a href="#ref-H.264" title=""Advanced video coding for generic audiovisual services"">H.264</a>] for the |
| 2283 |
layer representation containing the first NAL unit following the |
| 2284 |
PACSI NAL unit in the aggregation packet (when the PACSI NAL unit |
| 2285 |
is included in an aggregation packet) or containing the next non- |
| 2286 |
PACSI NAL unit in transmission order (when the PACSI NAL unit is |
| 2287 |
included in a single NAL unit packet). |
| 2288 |
|
| 2289 |
|
| 2290 |
|
| 2291 |
|
| 2292 |
<span class="grey">Wenger, et al. Standards Track [Page 41]</span></pre> |
| 2293 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-42" ></span> |
| 2294 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 2295 |
|
| 2296 |
|
| 2297 |
Informative note: The TL0PICIDX and IDRPICID fields enable the |
| 2298 |
detection of the loss of layer representations in the most |
| 2299 |
important temporal layer (with temporal_id equal to 0) by |
| 2300 |
receivers as well as MANEs. SVC provides a solution that uses SEI |
| 2301 |
messages, which are harder to parse and may or may not be present |
| 2302 |
in the bitstream. When the PACSI NAL unit is part of an NI-MTAP |
| 2303 |
packet, it is possible to infer the correct values of |
| 2304 |
tl0_dep_rep_idx and idr_pic_id for all layer representations |
| 2305 |
contained in the NI-MTAP by following the rules that specify how |
| 2306 |
these parameters are set as given in Annex G of [<a href="#ref-H.264" title=""Advanced video coding for generic audiovisual services"">H.264</a>] and by |
| 2307 |
detecting the different layer representations contained in the NI- |
| 2308 |
MTAP packet by detecting changes in the values of dependency_id_, |
| 2309 |
quality_id, and temporal_id in the NAL unit headers as well as |
| 2310 |
using the S and E flags. The only exception is if NAL units of an |
| 2311 |
IDR picture are present in the NI-MTAP in a position other than |
| 2312 |
the first NAL unit following the PACSI NAL unit, in which case the |
| 2313 |
value of idr_pic_id cannot be inferred. In this case the NAL unit |
| 2314 |
has to be partially parsed to obtain the idr_pic_id. Note that, |
| 2315 |
due to the large size of IDR pictures, their inclusion in an NI- |
| 2316 |
MTAP, and especially in a position other than the first NAL unit |
| 2317 |
following the PACSI NAL unit, may be neither practical nor useful. |
| 2318 |
|
| 2319 |
o When present, the field DONC indicates the cross-session decoding |
| 2320 |
order number (CS-DON) for the first of the remaining NAL units in |
| 2321 |
the aggregation packet (when the PACSI NAL unit is included in an |
| 2322 |
aggregation packet) or the CS-DON of the next non-PACSI NAL unit |
| 2323 |
in transmission order (when the PACSI NAL unit is included in a |
| 2324 |
single NAL unit packet). CS-DON is further discussed in <a href="#section-4.11">Section</a> |
| 2325 |
<a href="#section-4.11">4.11</a>. |
| 2326 |
|
| 2327 |
The PACSI NAL unit MAY include a subset of the SEI NAL units |
| 2328 |
associated with the access unit to which the first non-PACSI NAL unit |
| 2329 |
in the aggregation packet belongs, and MUST NOT contain SEI NAL units |
| 2330 |
associated with any other access unit. |
| 2331 |
|
| 2332 |
Informative note: In H.264/AVC and SVC, within each access unit, |
| 2333 |
SEI NAL units must appear before any VCL NAL unit in decoding |
| 2334 |
order. Therefore, without using PACSI NAL units, SEI messages are |
| 2335 |
typically only conveyed in the first of the packets carrying an |
| 2336 |
access unit. Senders may repeat SEI NAL units in PACSI NAL units, |
| 2337 |
so that they are repeated in more than one packet and thus |
| 2338 |
increase robustness against packet losses. Receivers may use the |
| 2339 |
repeated SEI messages in place of missing SEI messages. |
| 2340 |
|
| 2341 |
For a PACSI NAL unit included in an aggregation packet, an SEI |
| 2342 |
message SHOULD NOT be included in the PACSI NAL unit and also |
| 2343 |
included in one of the remaining NAL units contained in the same |
| 2344 |
aggregation packet. |
| 2345 |
|
| 2346 |
|
| 2347 |
|
| 2348 |
<span class="grey">Wenger, et al. Standards Track [Page 42]</span></pre> |
| 2349 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-43" ></span> |
| 2350 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 2351 |
|
| 2352 |
|
| 2353 |
<span class="h3"><a class="selflink" id="section-4.10" href="#section-4.10">4.10</a>. Empty NAL unit</span> |
| 2354 |
|
| 2355 |
An empty NAL unit MAY be included in a single NAL unit packet, an |
| 2356 |
STAP-A or an NI-MTAP packet. Empty NAL units MUST have an RTP |
| 2357 |
timestamp (when transported in a single NAL unit packet) or NALU- |
| 2358 |
time (when transported in an aggregation packet) that is associated |
| 2359 |
with an access unit for which there exists at least one NAL unit of |
| 2360 |
type 1, 5, or 20. When MST is used, the type 1, 5, or 20 NAL unit |
| 2361 |
may be in a different RTP session. Empty NAL units may be used in |
| 2362 |
the decoding order recovery process of the NI-T mode as described in |
| 2363 |
<a href="#section-5.2.1">Section 5.2.1</a>. |
| 2364 |
|
| 2365 |
The packet structure is shown in the following figure. |
| 2366 |
|
| 2367 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |
| 2368 |
|F|NRI| Type | Subtype |J|K|L| |
| 2369 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |
| 2370 |
|
| 2371 |
Figure 4. Empty NAL unit structure. |
| 2372 |
|
| 2373 |
The fields MUST be set as follows: |
| 2374 |
|
| 2375 |
F MUST be equal to 0 |
| 2376 |
NRI MUST be equal to 3 |
| 2377 |
Type MUST be equal to 31 |
| 2378 |
Subtype MUST be equal to 1 |
| 2379 |
J MUST be equal to 0 |
| 2380 |
K MUST be equal to 0 |
| 2381 |
L MUST be equal to 0 |
| 2382 |
|
| 2383 |
<span class="h3"><a class="selflink" id="section-4.11" href="#section-4.11">4.11</a>. Decoding Order Number (DON)</span> |
| 2384 |
|
| 2385 |
The DON concept is introduced in [<a href="./rfc6184" title=""RTP Payload Format for H.264 Video"">RFC6184</a>] and is used to recover the |
| 2386 |
decoding order when interleaving is used within a single session. |
| 2387 |
<a href="./rfc6184#section-5.5">Section 5.5 of [RFC6184]</a> applies when using SST. |
| 2388 |
|
| 2389 |
When using MST, it is necessary to recover the decoding order across |
| 2390 |
the various RTP sessions regardless if interleaving is used or not. |
| 2391 |
In addition to the timestamp mechanism described later, the CS-DON |
| 2392 |
mechanism is an extension of the DON facility that can be used for |
| 2393 |
this purpose, and is defined in the following section. |
| 2394 |
|
| 2395 |
<span class="h4"><a class="selflink" id="section-4.11.1" href="#section-4.11.1">4.11.1</a>. Cross-Session DON (CS-DON) for Multi-Session Transmission</span> |
| 2396 |
|
| 2397 |
The cross-session decoding order number (CS-DON) is a number that |
| 2398 |
indicates the decoding order of NAL units across all RTP sessions |
| 2399 |
involved in MST. It is similar to the DON concept in [<a href="./rfc6184" title=""RTP Payload Format for H.264 Video"">RFC6184</a>], but |
| 2400 |
contrary to [<a href="./rfc6184" title=""RTP Payload Format for H.264 Video"">RFC6184</a>] where the DON was used only for interleaved |
| 2401 |
|
| 2402 |
|
| 2403 |
|
| 2404 |
<span class="grey">Wenger, et al. Standards Track [Page 43]</span></pre> |
| 2405 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-44" ></span> |
| 2406 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 2407 |
|
| 2408 |
|
| 2409 |
packetization, in this memo it is used not only in the interleaved |
| 2410 |
MST mode (I-C) but also in two of the non-interleaved MST modes (NI-C |
| 2411 |
and NI-TC). |
| 2412 |
|
| 2413 |
When the NI-C or NI-TC MST modes are in use, the packetization of |
| 2414 |
each session MUST be as specified in <a href="#section-5.2.2">Section 5.2.2</a>. In PACSI NAL |
| 2415 |
units the CS-DON value is explicitly coded in the field DONC. For |
| 2416 |
non-PACSI NAL units the CS-DON value is derived as follows. Let SN |
| 2417 |
indicate the RTP sequence number of a packet. |
| 2418 |
|
| 2419 |
o For each non-PACSI NAL unit carried in a session using the single |
| 2420 |
NAL unit session packetization mode, the CS-DON value of the NAL |
| 2421 |
unit is equal to (DONC_prev_PACSI + SN_diff - 1) % 65536, wherein |
| 2422 |
"%" is the modulo operation, DONC_prev_PACSI is the DONC value of |
| 2423 |
the previous PACSI NAL unit with the same NALU-time as the current |
| 2424 |
NAL unit, and SN_diff is calculated as follows: |
| 2425 |
|
| 2426 |
if SN1 > SN2, SN_diff = SN1 - SN2 |
| 2427 |
else SN_diff = SN2 + 65536 - SN1 |
| 2428 |
|
| 2429 |
where SN1 and SN2 are the SNs of the current NAL unit and the |
| 2430 |
previous PACSI NAL unit with the same NALU-time, respectively. |
| 2431 |
|
| 2432 |
o For non-PACSI NAL units carried in a session using the non- |
| 2433 |
interleaved session packetization mode, the CS-DON value of each |
| 2434 |
non-PACSI NAL unit is derived as follows. |
| 2435 |
|
| 2436 |
For a non-PACSI NAL unit in a single NAL unit packet, the |
| 2437 |
following applies. |
| 2438 |
|
| 2439 |
If the previous PACSI NAL unit is contained in a single NAL |
| 2440 |
unit packet, the CS-DON value of the NAL unit is calculated |
| 2441 |
as above; |
| 2442 |
|
| 2443 |
otherwise (the previous PACSI NAL unit is contained in an |
| 2444 |
STAP-A packet), the CS-DON value of the NAL unit is |
| 2445 |
calculated as above, with DONC_prev_PACSI being replaced by |
| 2446 |
the CS-DON value of the previous non-PACSI NAL unit in |
| 2447 |
decoding order (i.e., the CS-DON value of the last NAL unit |
| 2448 |
of the STAP-A packet). |
| 2449 |
|
| 2450 |
For a non-PACSI NAL unit in an STAP-A packet, the following |
| 2451 |
applies. |
| 2452 |
|
| 2453 |
If the non-PACSI NAL unit is the first non-PACSI NAL unit in |
| 2454 |
the STAP-A packet, the CS-DON value of the NAL unit is equal |
| 2455 |
to DONC of the PACSI NAL unit in the STAP-A packet; |
| 2456 |
|
| 2457 |
|
| 2458 |
|
| 2459 |
|
| 2460 |
<span class="grey">Wenger, et al. Standards Track [Page 44]</span></pre> |
| 2461 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-45" ></span> |
| 2462 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 2463 |
|
| 2464 |
|
| 2465 |
otherwise (the non-PACSI NAL unit is not the first non- |
| 2466 |
PACSI NAL unit in the STAP-A packet), the CS-DON value of |
| 2467 |
the NAL unit is equal to: (the CS-DON value of the previous |
| 2468 |
non-PACSI NAL unit in decoding order + 1) % 65536, wherein |
| 2469 |
"%" is the modulo operation. |
| 2470 |
|
| 2471 |
For a non-PACSI NAL unit in a number of FU-A packets, the CS- |
| 2472 |
DON value of the NAL unit is calculated the same way as when |
| 2473 |
the single NAL unit session packetization mode is in use, with |
| 2474 |
SN1 being the SN value of the first FU-A packet. |
| 2475 |
|
| 2476 |
For a non-PACSI NAL unit in an NI-MTAP packet, the CS-DON value |
| 2477 |
is equal to the value of the DON field of the non-interleaved |
| 2478 |
multi-time aggregation unit. |
| 2479 |
|
| 2480 |
When the I-C MST packetization mode is in use, the DON values derived |
| 2481 |
according to [<a href="./rfc6184" title=""RTP Payload Format for H.264 Video"">RFC6184</a>] for all the NAL units in each of the RTP |
| 2482 |
sessions MUST indicate CS-DON values. |
| 2483 |
|
| 2484 |
<span class="h2"><a class="selflink" id="section-5" href="#section-5">5</a>. Packetization Rules</span> |
| 2485 |
|
| 2486 |
<a href="./rfc6184#section-6">Section 6 of [RFC6184]</a> applies in this memo, with the following |
| 2487 |
additions. |
| 2488 |
|
| 2489 |
<span class="h3"><a class="selflink" id="section-5.1" href="#section-5.1">5.1</a>. Packetization Rules for Single-Session Transmission</span> |
| 2490 |
|
| 2491 |
All receivers MUST support the single NAL unit packetization mode to |
| 2492 |
provide backward compatibility to endpoints supporting only the |
| 2493 |
single NAL unit mode of [<a href="./rfc6184" title=""RTP Payload Format for H.264 Video"">RFC6184</a>]. However, the use of single NAL |
| 2494 |
unit packetization mode (packetization-mode equal to 0) SHOULD be |
| 2495 |
avoided whenever possible, because encapsulating NAL units of small |
| 2496 |
sizes in their own packets (e.g., small NAL units containing |
| 2497 |
parameter sets, prefix NAL units, or SEI messages) is less efficient |
| 2498 |
due to the packet header overhead. |
| 2499 |
|
| 2500 |
All receivers MUST support the non-interleaved mode. |
| 2501 |
|
| 2502 |
Informative note: The non-interleaved mode of [<a href="./rfc6184" title=""RTP Payload Format for H.264 Video"">RFC6184</a>] does allow |
| 2503 |
an application to encapsulate a single NAL unit in a single RTP |
| 2504 |
packet. Historically, the single NAL unit mode has been included |
| 2505 |
in [<a href="./rfc6184" title=""RTP Payload Format for H.264 Video"">RFC6184</a>] only for compatibility with ITU-T Rec. H.241 Annex A |
| 2506 |
[<a href="#ref-H.241" title=""Extended video procedures and control signals for H.300-series terminals"">H.241</a>]. There is no point in carrying this historic ballast |
| 2507 |
towards a new application space such as the one provided with SVC. |
| 2508 |
The implementation complexity increase for supporting the |
| 2509 |
additional mechanisms of the non-interleaved mode (namely, STAP-A |
| 2510 |
and FU-A) is minor, whereas the benefits are significant. As a |
| 2511 |
result, the support of STAP-A and FU-A is required. Additionally, |
| 2512 |
|
| 2513 |
|
| 2514 |
|
| 2515 |
|
| 2516 |
<span class="grey">Wenger, et al. Standards Track [Page 45]</span></pre> |
| 2517 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-46" ></span> |
| 2518 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 2519 |
|
| 2520 |
|
| 2521 |
support for two of the three NAL unit types defined in this memo, |
| 2522 |
namely, empty NAL units and NI-MTAP is needed, as specified in |
| 2523 |
<a href="#section-4.5.1">Section 4.5.1</a>. |
| 2524 |
|
| 2525 |
A NAL unit of small size SHOULD be encapsulated in an aggregation |
| 2526 |
packet together with one or more other NAL units. For example, non- |
| 2527 |
VCL NAL units such as access unit delimiters, parameter sets, or SEI |
| 2528 |
NAL units are typically small. |
| 2529 |
|
| 2530 |
A prefix NAL unit and the NAL unit with which it is associated, and |
| 2531 |
which follows the prefix NAL unit in decoding order, SHOULD be |
| 2532 |
included in the same aggregation packet whenever an aggregation |
| 2533 |
packet is used for the associated NAL unit, unless this would violate |
| 2534 |
session MTU constraints or if fragmentation units are used for the |
| 2535 |
associated NAL unit. |
| 2536 |
|
| 2537 |
Informative note: Although the prefix NAL unit is ignored by an |
| 2538 |
H.264/AVC decoder, it is necessary in the SVC decoding process. |
| 2539 |
|
| 2540 |
Given the small size of the prefix NAL unit, it is best if it is |
| 2541 |
transported in the same RTP packet as its associated NAL unit. |
| 2542 |
|
| 2543 |
When only an H.264/AVC compatible subset of the SVC base layer is |
| 2544 |
transmitted in an RTP session, the subset MUST be encapsulated |
| 2545 |
according to [<a href="./rfc6184" title=""RTP Payload Format for H.264 Video"">RFC6184</a>]. This way, an [<a href="./rfc6184" title=""RTP Payload Format for H.264 Video"">RFC6184</a>] receiver will be able |
| 2546 |
to receive the H.264/AVC compatible bitstream subset. |
| 2547 |
|
| 2548 |
When a set of layers including one or more SVC enhancement layers is |
| 2549 |
transmitted in an RTP session, the set SHOULD be carried in one RTP |
| 2550 |
stream that SHOULD be encapsulated according to this memo. |
| 2551 |
|
| 2552 |
<span class="h3"><a class="selflink" id="section-5.2" href="#section-5.2">5.2</a>. Packetization Rules for Multi-Session Transmission</span> |
| 2553 |
|
| 2554 |
When MST is used, the packetization rules specified in <a href="#section-5.1">Section 5.1</a> |
| 2555 |
still apply. In addition, the following packetization rules MUST be |
| 2556 |
followed, to ensure that decoding order of NAL units carried in the |
| 2557 |
sessions can be correctly recovered for each of the MST packetization |
| 2558 |
modes using the de-packetization process specified in <a href="#section-6.2">Section 6.2</a>. |
| 2559 |
|
| 2560 |
The NI-T and NI-TC modes both use timestamps to recover the decoding |
| 2561 |
order. In order to be able to do so, it is necessary for the RTP |
| 2562 |
packet stream to contain data for all sampling instances of a given |
| 2563 |
RTP session in all enhancement RTP sessions that depend on the given |
| 2564 |
RTP session. The NI-C and I-C modes do not have this limitation, and |
| 2565 |
use the CS-DON values as a means to explicitly indicate decoding |
| 2566 |
order, either directly coded in PACSI NAL units, or inferred from |
| 2567 |
|
| 2568 |
|
| 2569 |
|
| 2570 |
|
| 2571 |
|
| 2572 |
<span class="grey">Wenger, et al. Standards Track [Page 46]</span></pre> |
| 2573 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-47" ></span> |
| 2574 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 2575 |
|
| 2576 |
|
| 2577 |
them using the packetization rules. It is noted that the NI-TC mode |
| 2578 |
offers both alternatives and it is up to the receiver to select which |
| 2579 |
one to use. |
| 2580 |
|
| 2581 |
<span class="h4"><a class="selflink" id="section-5.2.1" href="#section-5.2.1">5.2.1</a>. NI-T/NI-TC Packetization Rules</span> |
| 2582 |
|
| 2583 |
When using the NI-T mode and a PACSI NAL unit is present, the T bit |
| 2584 |
MUST be equal to 0, i.e., the DONC field MUST NOT be present. |
| 2585 |
|
| 2586 |
When using the NI-T mode, the optional parameters sprop-mst-remux- |
| 2587 |
buf-size, sprop-remux-buf-req, remux-buf-cap, sprop-remux-init-buf- |
| 2588 |
time, sprop-mst-max-don-diff MUST NOT be present. |
| 2589 |
|
| 2590 |
When the NI-T or NI-TC MST mode is in use, the following applies. |
| 2591 |
|
| 2592 |
If one or more NAL units of an access unit of sampling time instance |
| 2593 |
t is present in RTP session A, then one or more NAL units of the same |
| 2594 |
access unit MUST be present in any enhancement RTP session that |
| 2595 |
depends on RTP session A. |
| 2596 |
|
| 2597 |
Informative note: The mapping between RTP and NTP format |
| 2598 |
timestamps is conveyed in RTCP SR packets. In addition, the |
| 2599 |
mechanisms for faster media timestamp synchronization discussed in |
| 2600 |
[<a href="./rfc6051" title=""Rapid Synchronisation of RTP Flows"">RFC6051</a>] may be used to speed up the acquisition of the RTP-to- |
| 2601 |
wall-clock mapping. |
| 2602 |
|
| 2603 |
Informative note: The rule above may require the insertion of NAL |
| 2604 |
units, typically when temporal scalability is used, i.e., an |
| 2605 |
enhancement RTP session does not contain any NAL units for an |
| 2606 |
access unit with a particular NTP timestamp (media timestamp), |
| 2607 |
which, however, is present in a lower enhancement RTP session or |
| 2608 |
the base RTP session. There are two ways to insert additional NAL |
| 2609 |
units in order to satisfy this rule: |
| 2610 |
|
| 2611 |
- One option for adding additional NAL units is to use empty NAL |
| 2612 |
units (defined in <a href="#section-4.10">Section 4.10</a>), which can be used by the |
| 2613 |
process described in <a href="#section-6.2.1">Section 6.2.1</a> for the access unit |
| 2614 |
reordering process. |
| 2615 |
|
| 2616 |
- Additional NAL units may also be added by the encoder itself, |
| 2617 |
for example, by transmitting coded data that simply instruct the |
| 2618 |
decoder to repeat the previous picture. This option, however, |
| 2619 |
may be difficult to use with pre-encoded content. |
| 2620 |
|
| 2621 |
If a packet must be inserted in order to satisfy the above rule, |
| 2622 |
e.g., in case of a MANE generating multiple RTP streams out of a |
| 2623 |
single RTP stream, the inserted packet must have an RTP timestamp |
| 2624 |
that maps to the same wall-clock time (in NTP format) as the one of |
| 2625 |
|
| 2626 |
|
| 2627 |
|
| 2628 |
<span class="grey">Wenger, et al. Standards Track [Page 47]</span></pre> |
| 2629 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-48" ></span> |
| 2630 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 2631 |
|
| 2632 |
|
| 2633 |
the RTP timestamp of any packet of the access unit present in any |
| 2634 |
lower enhancement RTP session or the base RTP session. This is easy |
| 2635 |
to accomplish if the NAL unit or the packet can be inserted at the |
| 2636 |
time of the RTP stream generation, since the media timestamp (NTP |
| 2637 |
timestamp) must be the same for the inserted packet and the packet of |
| 2638 |
the corresponding access unit. If there is no knowledge of the media |
| 2639 |
time at RTP stream generation or if the RTP streams are not generated |
| 2640 |
at the same instance, this can be also applied later in the |
| 2641 |
transmission process. In this case the NTP timestamp of the inserted |
| 2642 |
packet can be calculated as follows. |
| 2643 |
|
| 2644 |
Assume that a packet A2 of an access unit with RTP timestamp TS_A2 is |
| 2645 |
present in base RTP session A, and that no packet of that access unit |
| 2646 |
is present in enhancement RTP session B, as shown in Figure 5. Thus, |
| 2647 |
a packet B2 must be inserted into session B following the rule above. |
| 2648 |
The most recent RTCP sender report in session A carries NTP timestamp |
| 2649 |
NTP_A and the RTP timestamp TS_A. The sender report in session B |
| 2650 |
with a lower NTP timestamp than NTP_A is NTP_B, and carries the RTP |
| 2651 |
timestamp TS_B. |
| 2652 |
|
| 2653 |
RTP session B:..B0........B1........(B2)...................... |
| 2654 |
|
| 2655 |
RTCP session B:.....SR(NTP_B,TS_B)............................. |
| 2656 |
|
| 2657 |
RTP session A:..A0........A1........A2........................ |
| 2658 |
|
| 2659 |
RTCP session A:..................SR(NTP_A,TS_A)................ |
| 2660 |
|
| 2661 |
-----------------|--x------|-----x---|------------------------> |
| 2662 |
NTP time |
| 2663 |
--------------------+<---------->+<->+------------------------> |
| 2664 |
t1 t2 RTP TS(B) time |
| 2665 |
|
| 2666 |
Figure 5. Example calculation of RTP timestamp for packet |
| 2667 |
insertion in an enhancement layer RTP session |
| 2668 |
|
| 2669 |
The vertical bars ("|")in the NTP time line in the figure above |
| 2670 |
indicate that access unit data is present in at least one of the |
| 2671 |
sessions. The "x" marks indicate the times of the sender reports. |
| 2672 |
The RTP timestamp time line for session B, shown right below the NTP |
| 2673 |
time line, indicates two time segments, t1 and t2. t1 is the time |
| 2674 |
difference between the sender reports between the two sessions, |
| 2675 |
expressed in RTP timestamp clock ticks, and t2 is the time difference |
| 2676 |
from the session A sender report to the A2 packet, again expressed in |
| 2677 |
RTP timestamp clock ticks. The sum of these differences is added to |
| 2678 |
|
| 2679 |
|
| 2680 |
|
| 2681 |
|
| 2682 |
|
| 2683 |
|
| 2684 |
<span class="grey">Wenger, et al. Standards Track [Page 48]</span></pre> |
| 2685 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-49" ></span> |
| 2686 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 2687 |
|
| 2688 |
|
| 2689 |
the RTP timestamp of the session report from session B in order to |
| 2690 |
derive the correct RTP timestamp for the inserted packet B2. In |
| 2691 |
other words: |
| 2692 |
|
| 2693 |
TS_B2 = TS_B + t1 + t2 |
| 2694 |
|
| 2695 |
Let toRTP() be a function that calculates the RTP time difference (in |
| 2696 |
clock ticks of the used clock) given an NTP timestamp difference, and |
| 2697 |
effRTPdiff() be a function that calculates the effective difference |
| 2698 |
between two timestamps, including wraparounds: |
| 2699 |
|
| 2700 |
effRTPdiff( ts1, ts2 ): |
| 2701 |
|
| 2702 |
if( ts1 <= ts2 ) then |
| 2703 |
effRTPdiff := ts1-ts2 |
| 2704 |
else |
| 2705 |
effRTPDiff := (4294967296 + ts2) - ts1 |
| 2706 |
We have: |
| 2707 |
|
| 2708 |
t1 = toRTP(NTP_A - NTP_B) and t2 = effRTPdiff(TS_A2, TS_A) |
| 2709 |
|
| 2710 |
Hence in order to generate the RTP timestamp TS_B2 for the inserted |
| 2711 |
packet B2, the RTP timestamp for packet B2 TS_B2 can be calculated as |
| 2712 |
follows. |
| 2713 |
|
| 2714 |
TS_B2 = TS_B + toRTP(NTP_A - NTP_B) + effRTPdiff(TS_A2, TS_A) |
| 2715 |
|
| 2716 |
<span class="h4"><a class="selflink" id="section-5.2.2" href="#section-5.2.2">5.2.2</a>. NI-C/NI-TC Packetization Rules</span> |
| 2717 |
|
| 2718 |
When the NI-C or NI-TC MST mode is in use, the following applies for |
| 2719 |
each of the RTP sessions. |
| 2720 |
|
| 2721 |
o For each single NAL unit packet containing a non-PACSI NAL unit, |
| 2722 |
the previous packet, if present, MUST have the same RTP timestamp |
| 2723 |
as the single NAL unit packet, and the following applies. |
| 2724 |
|
| 2725 |
o If the NALU-time of the non-PACSI NAL unit is not equal to the |
| 2726 |
NALU-time of the previous non-PACSI NAL unit in decoding order, |
| 2727 |
the previous packet MUST contain a PACSI NAL unit containing |
| 2728 |
the DONC field. |
| 2729 |
|
| 2730 |
o In an STAP-A packet the first NAL unit in the STAP-A packet MUST |
| 2731 |
be a PACSI NAL unit containing the DONC field. |
| 2732 |
|
| 2733 |
o For an FU-A packet the previous packet MUST have the same RTP |
| 2734 |
timestamp as the FU-A packet, and the following applies. |
| 2735 |
|
| 2736 |
|
| 2737 |
|
| 2738 |
|
| 2739 |
|
| 2740 |
<span class="grey">Wenger, et al. Standards Track [Page 49]</span></pre> |
| 2741 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-50" ></span> |
| 2742 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 2743 |
|
| 2744 |
|
| 2745 |
o If the FU-A packet is the start of the fragmented NAL unit, the |
| 2746 |
following applies. |
| 2747 |
|
| 2748 |
o If the NALU-time of the fragmented NAL unit is not equal to |
| 2749 |
the NALU-time of the previous non-PACSI NAL unit in decoding |
| 2750 |
order, the previous packet MUST contain a PACSI NAL unit |
| 2751 |
containing the DONC field; |
| 2752 |
|
| 2753 |
o Otherwise, (the NALU-time of the fragmented NAL unit is |
| 2754 |
equal to the NALU-time of the previous non-PACSI NAL unit in |
| 2755 |
decoding order), the previous packet MAY contain a PACSI NAL |
| 2756 |
unit containing the DONC field. |
| 2757 |
|
| 2758 |
o Otherwise, if the FU-A packet is the end of the fragmented NAL |
| 2759 |
unit, the following applies. |
| 2760 |
|
| 2761 |
o If the next non-PACSI NAL unit in decoding order has NALU- |
| 2762 |
time equal to the NALU-time of the fragmented NAL unit, and |
| 2763 |
is carried in a number of FU-A packets or a single NAL unit |
| 2764 |
packet, the next packet MUST be a single NAL unit packet |
| 2765 |
containing a PACSI NAL unit containing the DONC field. |
| 2766 |
|
| 2767 |
o Otherwise (the FU-A packet is neither the start nor the end |
| 2768 |
of the fragmented NAL unit), the previous packet MUST be a |
| 2769 |
FU-A packet. |
| 2770 |
|
| 2771 |
o For each single NAL unit packet containing a PACSI NAL unit, if |
| 2772 |
present, the PACSI NAL unit MUST contain the DONC field. |
| 2773 |
|
| 2774 |
o When the optional media type parameter sprop-mst-csdon-always- |
| 2775 |
present is equal to 1, the session packetization mode in use MUST |
| 2776 |
be the non-interleaved mode, and only STAP-A and NI-MTAP packets |
| 2777 |
can be used. |
| 2778 |
|
| 2779 |
<span class="h4"><a class="selflink" id="section-5.2.3" href="#section-5.2.3">5.2.3</a>. I-C Packetization Rules</span> |
| 2780 |
|
| 2781 |
When the I-C MST packetization mode is in use, the following applies. |
| 2782 |
|
| 2783 |
o When a PACSI NAL unit is present, the T bit MUST be equal to 0, |
| 2784 |
i.e., the DONC field is not present, and the Y bit MUST be equal |
| 2785 |
to 0, i.e., the TL0PICIDX and IDRPICID are not present. |
| 2786 |
|
| 2787 |
<span class="h4"><a class="selflink" id="section-5.2.4" href="#section-5.2.4">5.2.4</a>. Packetization Rules for Non-VCL NAL Units</span> |
| 2788 |
|
| 2789 |
NAL units that do not directly encode video slices are known in H.264 |
| 2790 |
as non-VCL NAL units. Non-VCL units that are only used by, or only |
| 2791 |
relevant to, enhancement RTP sessions SHOULD be sent in the lowest |
| 2792 |
session to which they are relevant. |
| 2793 |
|
| 2794 |
|
| 2795 |
|
| 2796 |
<span class="grey">Wenger, et al. Standards Track [Page 50]</span></pre> |
| 2797 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-51" ></span> |
| 2798 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 2799 |
|
| 2800 |
|
| 2801 |
Some senders, however, such as those sending pre-encoded data, may be |
| 2802 |
unable to easily determine which non-VCL units are relevant to which |
| 2803 |
session. Thus, non-VCL NAL units MAY, instead, be sent in a session |
| 2804 |
on which the session using these non-VCL NAL units depends (e.g., the |
| 2805 |
base RTP session). |
| 2806 |
|
| 2807 |
If a non-VCL unit is relevant to more than one RTP session, neither |
| 2808 |
of which depends on the other(s), the NAL unit MAY be sent in another |
| 2809 |
session on which all these sessions depend. |
| 2810 |
|
| 2811 |
<span class="h4"><a class="selflink" id="section-5.2.5" href="#section-5.2.5">5.2.5</a>. Packetization Rules for Prefix NAL Units</span> |
| 2812 |
|
| 2813 |
<a href="#section-5.1">Section 5.1</a> of this memo applies, with the following addition. If |
| 2814 |
the base layer is sent in a base RTP session using [<a href="./rfc6184" title=""RTP Payload Format for H.264 Video"">RFC6184</a>], prefix |
| 2815 |
NAL units MAY be sent in the lowest enhancement RTP session rather |
| 2816 |
than in the base RTP session. |
| 2817 |
|
| 2818 |
<span class="h2"><a class="selflink" id="section-6" href="#section-6">6</a>. De-Packetization Process</span> |
| 2819 |
|
| 2820 |
<span class="h3"><a class="selflink" id="section-6.1" href="#section-6.1">6.1</a>. De-Packetization Process for Single-Session Transmission</span> |
| 2821 |
|
| 2822 |
For single-session transmission, where a single RTP session is used, |
| 2823 |
the de-packetization process specified in <a href="./rfc6184#section-7">Section 7 of [RFC6184]</a> |
| 2824 |
applies. |
| 2825 |
|
| 2826 |
<span class="h3"><a class="selflink" id="section-6.2" href="#section-6.2">6.2</a>. De-Packetization Process for Multi-Session Transmission</span> |
| 2827 |
|
| 2828 |
For multi-session transmission, where more than one RTP session is |
| 2829 |
used to receive data from the same SVC bitstream, the de- |
| 2830 |
packetization process is specified as follows. |
| 2831 |
|
| 2832 |
As for a single RTP session, the general concept behind the de- |
| 2833 |
packetization process is to reorder NAL units from transmission order |
| 2834 |
to the NAL unit decoding order. |
| 2835 |
|
| 2836 |
The sessions to be received MUST be identified by mechanisms |
| 2837 |
specified in <a href="#section-7.2.3">Section 7.2.3</a>. An enhancement RTP session typically |
| 2838 |
contains an RTP stream that depends on at least one other RTP |
| 2839 |
session, as indicated by mechanisms defined in <a href="#section-7.2.3">Section 7.2.3</a>. A |
| 2840 |
lower RTP session to an enhancement RTP session is an RTP session on |
| 2841 |
which the enhancement RTP session depends. The lowest RTP session |
| 2842 |
for a receiver is the base RTP session, which does not depend on any |
| 2843 |
other RTP session received by the receiver. The highest RTP session |
| 2844 |
for a receiver is the RTP session on which no other RTP session |
| 2845 |
received by the receiver depends. |
| 2846 |
|
| 2847 |
|
| 2848 |
|
| 2849 |
|
| 2850 |
|
| 2851 |
|
| 2852 |
<span class="grey">Wenger, et al. Standards Track [Page 51]</span></pre> |
| 2853 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-52" ></span> |
| 2854 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 2855 |
|
| 2856 |
|
| 2857 |
For each of the RTP sessions, the RTP reception process as specified |
| 2858 |
in <a href="./rfc3550">RFC 3550</a> is applied. Then the received packets are passed into |
| 2859 |
the payload de-packetization process as defined in this memo. |
| 2860 |
|
| 2861 |
The decoding order of the NAL units carried in all the associated RTP |
| 2862 |
sessions is then recovered by applying one of the following |
| 2863 |
subsections, depending on which of the MST packetization modes is in |
| 2864 |
use. |
| 2865 |
|
| 2866 |
<span class="h4"><a class="selflink" id="section-6.2.1" href="#section-6.2.1">6.2.1</a>. Decoding Order Recovery for the NI-T and NI-TC Modes</span> |
| 2867 |
|
| 2868 |
The following process MUST be applied when the NI-T packetization |
| 2869 |
mode is in use. The following process MAY be applied when the NI-TC |
| 2870 |
packetization mode is in use. |
| 2871 |
|
| 2872 |
The process is based on RTP session dependency signaling, RTP |
| 2873 |
sequence numbers, and timestamps. |
| 2874 |
|
| 2875 |
The decoding order of NAL units within an RTP packet stream in RTP |
| 2876 |
session is given by the ordering of sequence numbers SN of the RTP |
| 2877 |
packets that contain the NAL units, and the order of appearance of |
| 2878 |
NAL units within a packet. |
| 2879 |
|
| 2880 |
Timing information according to the media timestamp TS, i.e., the NTP |
| 2881 |
timestamp as derived from the RTP timestamp of an RTP packet, is |
| 2882 |
associated with all NAL units contained in the same RTP packet |
| 2883 |
received in an RTP session. |
| 2884 |
|
| 2885 |
For NI-MTAP packets the NALU-time is derived for each contained NAL |
| 2886 |
unit by using the "TS offset" value in the NI-MTAP packet as defined |
| 2887 |
in <a href="#section-4.10">Section 4.10</a>, and is used instead of the RTP packet timestamp to |
| 2888 |
derive the media timestamp, e.g., using the NTP wall clock as |
| 2889 |
provided via RTCP sender reports. NAL units contained in |
| 2890 |
fragmentation packets are handled as defragmented, entire NAL units |
| 2891 |
with their own media timestamps. All NAL units associated with the |
| 2892 |
same value of media timestamp TS are part of the same access unit |
| 2893 |
AU(TS). Any empty NAL units SHOULD be kept as, effectively, access |
| 2894 |
unit indicators in the reordering process. Empty NAL units and PACSI |
| 2895 |
NAL units SHOULD be removed before passing access unit data to the |
| 2896 |
decoder. |
| 2897 |
|
| 2898 |
Informative note: These empty NAL units are used to associate NAL |
| 2899 |
units present in other RTP sessions with RTP sessions not |
| 2900 |
containing any data for an access unit of a particular time |
| 2901 |
instance. They act as access unit indicators in sessions that |
| 2902 |
would otherwise contain no data for the particular access unit. |
| 2903 |
The presence of these NAL units is ensured by the packetization |
| 2904 |
rules in <a href="#section-5.2.1">Section 5.2.1</a>. |
| 2905 |
|
| 2906 |
|
| 2907 |
|
| 2908 |
<span class="grey">Wenger, et al. Standards Track [Page 52]</span></pre> |
| 2909 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-53" ></span> |
| 2910 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 2911 |
|
| 2912 |
|
| 2913 |
It is assumed that the receiver has established an operation point |
| 2914 |
(DID, QID, and TID values), and has identified the highest |
| 2915 |
enhancement RTP session for this operation point. The decoding order |
| 2916 |
of NAL units from multiple RTP streams in multiple RTP sessions MUST |
| 2917 |
be recovered into a single sequence of NAL units, grouped into access |
| 2918 |
units, by performing any process equivalent to the following steps. |
| 2919 |
The general process is described in <a href="./rfc6051#section-4.2">Section 4.2 of [RFC6051]</a>. For |
| 2920 |
convenience the instructions of [<a href="./rfc6051" title=""Rapid Synchronisation of RTP Flows"">RFC6051</a>] are repeated and applied to |
| 2921 |
NAL units rather than to full RTP packets. Additionally, SVC- |
| 2922 |
specific extensions to the procedure in <a href="./rfc6051#section-4.2">Section 4.2. of [RFC6051]</a> |
| 2923 |
are presented in the following list: |
| 2924 |
|
| 2925 |
o The process should be started with the NAL units received in |
| 2926 |
the highest RTP session with the first media timestamp TS (in |
| 2927 |
NTP format) available in the session's (de-jittering) buffer. |
| 2928 |
It is assumed that packets in the de-jittering buffer are |
| 2929 |
already stored in RTP sequence number order. |
| 2930 |
|
| 2931 |
o Collect all NAL units associated with the same value of media |
| 2932 |
timestamp TS, starting from the highest RTP session, from all |
| 2933 |
the (de-jittering) buffers of the received RTP sessions. The |
| 2934 |
collected NAL units will be those associated with the access |
| 2935 |
unit AU(TS). |
| 2936 |
|
| 2937 |
o Place the collected NAL units in the order of session |
| 2938 |
dependency as derived by the dependency indication as specified |
| 2939 |
in <a href="#section-7.2.3">Section 7.2.3</a>, starting from the lowest RTP session. |
| 2940 |
|
| 2941 |
o Place the session ordered NAL units in decoding order within |
| 2942 |
the particular access unit by satisfying the NAL unit ordering |
| 2943 |
rules for SVC access units, as described in the informative |
| 2944 |
algorithm provided in <a href="#section-6.2.1.1">Section 6.2.1.1</a>. |
| 2945 |
|
| 2946 |
o Remove NI-MTAP and any PACSI NAL units from the access unit |
| 2947 |
AU(TS). |
| 2948 |
|
| 2949 |
o The access units can then be transferred to the decoder. |
| 2950 |
Access units AU(TS) are transferred to the decoder in the order |
| 2951 |
of appearance (given by the order of RTP sequence numbers) of |
| 2952 |
media timestamp values TS in the highest RTP session associated |
| 2953 |
with access unit AU(TS). |
| 2954 |
|
| 2955 |
Informative note: Due to packet loss it is possible that not |
| 2956 |
all sessions may have NAL units present for the media |
| 2957 |
timestamp value TS present in the highest RTP session. In |
| 2958 |
such a case, an algorithm may: a) proceed to the next |
| 2959 |
complete access unit with NAL units present in all the |
| 2960 |
received RTP sessions; or b) consider a new highest RTP |
| 2961 |
|
| 2962 |
|
| 2963 |
|
| 2964 |
<span class="grey">Wenger, et al. Standards Track [Page 53]</span></pre> |
| 2965 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-54" ></span> |
| 2966 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 2967 |
|
| 2968 |
|
| 2969 |
session, the highest RTP session for which the access unit |
| 2970 |
is complete, and apply the process above. The algorithm may |
| 2971 |
return to the original highest RTP session when a complete |
| 2972 |
and error-free access unit that contains NAL units in all |
| 2973 |
the sessions is received. |
| 2974 |
|
| 2975 |
The following gives an informative example. |
| 2976 |
|
| 2977 |
The example shown in Figure 6 refers to three RTP sessions A, B, and |
| 2978 |
C containing an SVC bitstream transmitted as 3 sources. In the |
| 2979 |
example, the dependency signaling (described in <a href="#section-7.2.3">Section 7.2.3</a>) |
| 2980 |
indicates that session A is the base RTP session, B is the first |
| 2981 |
enhancement RTP session and depends on A, and C is the second |
| 2982 |
enhancement RTP session and depends on A and B. A hierarchical |
| 2983 |
picture coding prediction structure is used, in which session A has |
| 2984 |
the lowest frame rate and sessions B and C have the same but higher |
| 2985 |
frame rate. |
| 2986 |
|
| 2987 |
The figure shows NAL units contained in RTP packets that are stored |
| 2988 |
in the de-jittering buffer at the receiver for session de- |
| 2989 |
packetization. The NAL units are already reordered according to |
| 2990 |
their RTP sequence number order and, if within an aggregation packet, |
| 2991 |
according to the order of their appearance within the aggregation |
| 2992 |
packet. The figure indicates for the received NAL units the decoding |
| 2993 |
order within the sessions, as well as the associated media (NTP) |
| 2994 |
timestamps ("TS[..]"). NAL units of the same access unit within a |
| 2995 |
session are grouped by "(.,.)" and share the same media timestamp TS, |
| 2996 |
which is shown at the bottom of the figure. Note that the timestamps |
| 2997 |
are not in increasing order since, in this example, the decoding |
| 2998 |
order is different from the output/display order. |
| 2999 |
|
| 3000 |
The process first proceeds to the NAL units associated with the first |
| 3001 |
media timestamp TS[1] present in the highest session C and |
| 3002 |
removes/ignores all preceding (in decoding order) NAL units to NAL |
| 3003 |
units with TS[1] in each of the de-jittering buffers of RTP sessions |
| 3004 |
A, B, and C. Then, starting from session C, the first media |
| 3005 |
timestamp available in decoding order (TS[1]) is selected and NAL |
| 3006 |
units starting from RTP session A, and sessions B and C are placed in |
| 3007 |
order of the RTP session dependency as required by <a href="#section-7.2.3">Section 7.2.3</a> of |
| 3008 |
this memo (in the example for TS[1]: first session B and then session |
| 3009 |
C) into the access unit AU(TS[1]) associated with media timestamp |
| 3010 |
TS[1]. Then the next media timestamp TS[3] in order of appearance in |
| 3011 |
the highest RTP session C is processed and the process described |
| 3012 |
above is repeated. Note that there may be access units with no NAL |
| 3013 |
units present, e.g., in the lowest RTP session A (see, e.g., TS[1]). |
| 3014 |
With TS[8], the first access unit with NAL units present in all the |
| 3015 |
RTP sessions appears in the buffers. |
| 3016 |
|
| 3017 |
|
| 3018 |
|
| 3019 |
|
| 3020 |
<span class="grey">Wenger, et al. Standards Track [Page 54]</span></pre> |
| 3021 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-55" ></span> |
| 3022 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 3023 |
|
| 3024 |
|
| 3025 |
C: ------------(1,2)-(3,4)--(5)---(6)---(7,8)(9,10)-(11)--(12)---- |
| 3026 |
| | | | | | | | | | |
| 3027 |
B: -(1,2)-(3,4)-(5)---(6)--(7,8)-(9,10)-(11)-(12)--(13,14)(15,15)- |
| 3028 |
| | | | | | |
| 3029 |
A: -------(1)---------------(2)---(3)---------------(4)----(5)---- |
| 3030 |
---------------------------------------------------decoding order--> |
| 3031 |
|
| 3032 |
TS: [4] [2] [1] [3] [8] [6] [5] [7] [12] [10] |
| 3033 |
|
| 3034 |
Key: |
| 3035 |
A, B, C - RTP sessions |
| 3036 |
Integer values in "()" - NAL unit decoding order within RTP session |
| 3037 |
"( )" - groups the NAL units of an access unit |
| 3038 |
in an RTP session |
| 3039 |
"|" - indicates corresponding NAL units of the |
| 3040 |
same access unit AU(TS[..]) in the RTP |
| 3041 |
sessions |
| 3042 |
Integer values in "[]" - media timestamp TS, sampling time |
| 3043 |
as derived, e.g., from NTP timestamp |
| 3044 |
associated with the access unit AU(TS[..]), |
| 3045 |
consisting of NAL units in the sessions |
| 3046 |
above each TS value. |
| 3047 |
|
| 3048 |
Figure 6. Example of decoding order recovery in multi-source |
| 3049 |
transmission. |
| 3050 |
|
| 3051 |
<span class="h5"><a class="selflink" id="section-6.2.1.1" href="#section-6.2.1.1">6.2.1.1</a>. Informative Algorithm for NI-T Decoding Order Recovery within</span> |
| 3052 |
<span class="h5"> an Access Unit</span> |
| 3053 |
|
| 3054 |
Within an access unit, the [<a href="#ref-H.264" title=""Advanced video coding for generic audiovisual services"">H.264</a>] specification (Sections <a href="#section-7.4.1.2.3">7.4.1.2.3</a> |
| 3055 |
and G.7.4.1.2.3) constrains the valid decoding order of NAL units. |
| 3056 |
|
| 3057 |
These constraints make it possible to reconstruct a valid decoding |
| 3058 |
order for the NAL units of an access unit based only on the order of |
| 3059 |
NAL units in each session, the NAL unit headers, and Supplemental |
| 3060 |
Enhancement Information message headers. |
| 3061 |
|
| 3062 |
This section specifies an informative algorithm to reconstruct a |
| 3063 |
valid decoding order for NAL units within an access unit. Other NAL |
| 3064 |
unit orderings may also be valid; however, any compliant NAL unit |
| 3065 |
ordering will describe the same video stream and ancillary data as |
| 3066 |
the one produced by this algorithm. |
| 3067 |
|
| 3068 |
An actual implementation, of course, needs only to behave "as if" |
| 3069 |
this reordering is done. In particular, NAL units that are discarded |
| 3070 |
by an implementation's decoding process do not need to be reordered. |
| 3071 |
|
| 3072 |
|
| 3073 |
|
| 3074 |
|
| 3075 |
|
| 3076 |
<span class="grey">Wenger, et al. Standards Track [Page 55]</span></pre> |
| 3077 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-56" ></span> |
| 3078 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 3079 |
|
| 3080 |
|
| 3081 |
In this algorithm, NAL units within an access unit are first ordered |
| 3082 |
by NAL unit type, in the order specified in Table 12 below, except |
| 3083 |
from NAL unit type 14, which is handled specially as described in the |
| 3084 |
table. NAL units of the same type are then ordered as specified for |
| 3085 |
the type, if necessary. |
| 3086 |
|
| 3087 |
For the purposes of this algorithm, "session order" is the order of |
| 3088 |
NAL units implied by their transmission order within an RTP session. |
| 3089 |
For the non-interleaved and single NAL unit modes, this is the RTP |
| 3090 |
sequence number order coupled with the order of NAL units within an |
| 3091 |
aggregation unit. |
| 3092 |
|
| 3093 |
Table 12. Ordering of NAL unit types within an Access Unit |
| 3094 |
|
| 3095 |
Type Description / Comments |
| 3096 |
----------------------------------------------------------- |
| 3097 |
9 Access unit delimiter |
| 3098 |
|
| 3099 |
7 Sequence parameter set |
| 3100 |
|
| 3101 |
13 Sequence parameter set extension |
| 3102 |
|
| 3103 |
15 Subset sequence parameter set |
| 3104 |
|
| 3105 |
8 Picture parameter set |
| 3106 |
|
| 3107 |
16-18 Reserved |
| 3108 |
|
| 3109 |
6 Supplemental enhancement information (SEI) |
| 3110 |
If an SEI message with a first payload of 0 (Buffering |
| 3111 |
Period) is present, it must be the first SEI message. |
| 3112 |
|
| 3113 |
If SEI messages with a Scalable Nesting (30) payload and |
| 3114 |
a nested payload of 0 (Buffering Period) are present, |
| 3115 |
these then follow the first SEI message. Such an SEI |
| 3116 |
message with the all_layer_representations_in_au_flag |
| 3117 |
equal to 1 is placed first, followed by any others, |
| 3118 |
sorted in increasing order of DQId. |
| 3119 |
|
| 3120 |
All other SEI messages follow in any order. |
| 3121 |
|
| 3122 |
14 Prefix NAL unit in scalable extension |
| 3123 |
1 Coded slice of a non-IDR picture |
| 3124 |
5 Coded slice of an IDR picture |
| 3125 |
|
| 3126 |
|
| 3127 |
|
| 3128 |
|
| 3129 |
|
| 3130 |
|
| 3131 |
|
| 3132 |
<span class="grey">Wenger, et al. Standards Track [Page 56]</span></pre> |
| 3133 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-57" ></span> |
| 3134 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 3135 |
|
| 3136 |
|
| 3137 |
NAL units of type 1 or 5 will be sent within only a |
| 3138 |
single session for any given access unit. They are |
| 3139 |
placed in session order. (Note: Any given access unit |
| 3140 |
will contain only NAL units of type 1 or type 5, not |
| 3141 |
both.) |
| 3142 |
|
| 3143 |
If NAL units of type 14 are present, every NAL unit of |
| 3144 |
type 1 or 5 is prefixed by a NAL unit of type 14. (Note: |
| 3145 |
Within an access unit, every NAL unit of type 14 is |
| 3146 |
identical, so correlation of type 14 NAL units with the |
| 3147 |
other NAL units is not necessary.) |
| 3148 |
|
| 3149 |
12 Filler data |
| 3150 |
|
| 3151 |
The only restriction of filler data NAL units within an |
| 3152 |
access unit is that they shall not precede the first VCL |
| 3153 |
NAL unit with the same access unit. |
| 3154 |
|
| 3155 |
19 Coded slice of an auxiliary coded picture without |
| 3156 |
partitioning |
| 3157 |
|
| 3158 |
These NAL units will be sent within only a single |
| 3159 |
session for any given access unit, and are placed in |
| 3160 |
session order. |
| 3161 |
|
| 3162 |
20 Coded slice in scalable extension |
| 3163 |
21-23 Reserved |
| 3164 |
|
| 3165 |
Type 20 NAL units are placed in increasing order of DQId. |
| 3166 |
Within each DQId value, they are placed in session order. |
| 3167 |
|
| 3168 |
(Note: SVC slices with a given DQId value will be sent |
| 3169 |
within only a single session for any given access unit.) |
| 3170 |
|
| 3171 |
Type 21-23 NAL units are placed immediately following |
| 3172 |
the non-reserved-type VCL NAL unit they follow in |
| 3173 |
session order. |
| 3174 |
|
| 3175 |
10 End of sequence |
| 3176 |
|
| 3177 |
11 End of stream |
| 3178 |
|
| 3179 |
<span class="h4"><a class="selflink" id="section-6.2.2" href="#section-6.2.2">6.2.2</a>. Decoding Order Recovery for the NI-C, NI-TC, and I-C Modes</span> |
| 3180 |
|
| 3181 |
The following process MUST be used when either the NI-C or I-C MST |
| 3182 |
packetization mode is in use. The following process MAY be applied |
| 3183 |
when the NI-TC MST packetization mode is in use. |
| 3184 |
|
| 3185 |
|
| 3186 |
|
| 3187 |
|
| 3188 |
<span class="grey">Wenger, et al. Standards Track [Page 57]</span></pre> |
| 3189 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-58" ></span> |
| 3190 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 3191 |
|
| 3192 |
|
| 3193 |
The RTP packets output from the RTP-level reception processing for |
| 3194 |
each session are placed into a re-multiplexing buffer. |
| 3195 |
|
| 3196 |
It is RECOMMENDED to set the size of the re-multiplexing buffer (in |
| 3197 |
bytes) equal to or greater than the value of the sprop-remux-buf-req |
| 3198 |
media type parameter of the highest RTP session the receiver |
| 3199 |
receives. |
| 3200 |
|
| 3201 |
The CS-DON value is calculated and stored for each NAL unit. |
| 3202 |
|
| 3203 |
Informative note: The CS-DON value of a NAL unit may rely on |
| 3204 |
information carried in another packet than the packet containing |
| 3205 |
the NAL unit. This happens, e.g., when the CS-DON values need to |
| 3206 |
be derived for non-PACSI NAL units contained in single NAL unit |
| 3207 |
packets, as the single NAL unit packets themselves do not contain |
| 3208 |
CS-DON information. In this case, when no packet containing |
| 3209 |
required CS-DON information is received for a NAL unit, this NAL |
| 3210 |
unit has to be discarded by the receiver as it cannot be fed to |
| 3211 |
the decoder in the correct order. When the optional media type |
| 3212 |
parameter sprop-mst-csdon-always-present is equal to 1, no such |
| 3213 |
dependency exists, i.e., the CS-DON value of any particular NAL |
| 3214 |
unit can be derived solely according to information in the packet |
| 3215 |
containing the NAL unit, and therefore, the receiver does not need |
| 3216 |
to discard any received NAL units. |
| 3217 |
|
| 3218 |
The receiver operation is described below with the help of the |
| 3219 |
following functions and constants: |
| 3220 |
|
| 3221 |
o Function AbsDON is specified in <a href="./rfc6184#section-8.1">Section 8.1 of [RFC6184]</a>. |
| 3222 |
|
| 3223 |
o Function don_diff is specified in <a href="./rfc6184#section-5.5">Section 5.5 of [RFC6184]</a>. |
| 3224 |
|
| 3225 |
o Constant N is the value of the OPTIONAL sprop-mst-remux-buf-size |
| 3226 |
media type parameter of the highest RTP session incremented by 1. |
| 3227 |
|
| 3228 |
Initial buffering lasts until one of the following conditions is |
| 3229 |
fulfilled: |
| 3230 |
|
| 3231 |
o There are N or more VCL NAL units in the re-multiplexing buffer. |
| 3232 |
|
| 3233 |
o If sprop-mst-max-don-diff of the highest RTP session is present, |
| 3234 |
don_diff(m,n) is greater than the value of sprop-mst-max-don-diff |
| 3235 |
of the highest RTP session, where n corresponds to the NAL unit |
| 3236 |
having the greatest value of AbsDON among the received NAL units |
| 3237 |
and m corresponds to the NAL unit having the smallest value of |
| 3238 |
AbsDON among the received NAL units. |
| 3239 |
|
| 3240 |
|
| 3241 |
|
| 3242 |
|
| 3243 |
|
| 3244 |
<span class="grey">Wenger, et al. Standards Track [Page 58]</span></pre> |
| 3245 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-59" ></span> |
| 3246 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 3247 |
|
| 3248 |
|
| 3249 |
o Initial buffering has lasted for the duration equal to or greater |
| 3250 |
than the value of the OPTIONAL sprop-remux-init-buf-time media |
| 3251 |
type parameter of the highest RTP session. |
| 3252 |
|
| 3253 |
The NAL units to be removed from the re-multiplexing buffer are |
| 3254 |
determined as follows: |
| 3255 |
|
| 3256 |
o If the re-multiplexing buffer contains at least N VCL NAL units, |
| 3257 |
NAL units are removed from the re-multiplexing buffer and passed |
| 3258 |
to the decoder in the order specified below until the buffer |
| 3259 |
contains N-1 VCL NAL units. |
| 3260 |
|
| 3261 |
o If sprop-mst-max-don-diff of the highest RTP session is present, |
| 3262 |
all NAL units m for which don_diff(m,n) is greater than sprop- |
| 3263 |
max-don-diff of the highest RTP session are removed from the re- |
| 3264 |
multiplexing buffer and passed to the decoder in the order |
| 3265 |
specified below. Herein, n corresponds to the NAL unit having the |
| 3266 |
greatest value of AbsDON among the NAL units in the re- |
| 3267 |
multiplexing buffer. |
| 3268 |
|
| 3269 |
The order in which NAL units are passed to the decoder is specified |
| 3270 |
as follows: |
| 3271 |
|
| 3272 |
o Let PDON be a variable that is initialized to 0 at the beginning |
| 3273 |
of the RTP sessions. |
| 3274 |
|
| 3275 |
o For each NAL unit associated with a value of CS-DON, a CS-DON |
| 3276 |
distance is calculated as follows. If the value of CS-DON of the |
| 3277 |
NAL unit is larger than the value of PDON, the CS-DON distance is |
| 3278 |
equal to CS-DON - PDON. Otherwise, the CS-DON distance is equal |
| 3279 |
to 65535 - PDON + CS-DON + 1. |
| 3280 |
|
| 3281 |
o NAL units are delivered to the decoder in increasing order of CS- |
| 3282 |
DON distance. If several NAL units share the same value of CS- |
| 3283 |
DON distance, they can be passed to the decoder in any order. |
| 3284 |
|
| 3285 |
o When a desired number of NAL units have been passed to the |
| 3286 |
decoder, the value of PDON is set to the value of CS-DON for the |
| 3287 |
last NAL unit passed to the decoder. |
| 3288 |
|
| 3289 |
<span class="h2"><a class="selflink" id="section-7" href="#section-7">7</a>. Payload Format Parameters</span> |
| 3290 |
|
| 3291 |
This section specifies the parameters that MAY be used to select |
| 3292 |
optional features of the payload format and certain features of the |
| 3293 |
bitstream. The parameters are specified here as part of the media |
| 3294 |
type registration for the SVC codec. A mapping of the parameters |
| 3295 |
into the Session Description Protocol (SDP) [<a href="./rfc4566" title=""SDP: Session Description Protocol"">RFC4566</a>] is also |
| 3296 |
|
| 3297 |
|
| 3298 |
|
| 3299 |
|
| 3300 |
<span class="grey">Wenger, et al. Standards Track [Page 59]</span></pre> |
| 3301 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-60" ></span> |
| 3302 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 3303 |
|
| 3304 |
|
| 3305 |
provided for applications that use SDP. Equivalent parameters could |
| 3306 |
be defined elsewhere for use with control protocols that do not use |
| 3307 |
SDP. |
| 3308 |
|
| 3309 |
Some parameters provide a receiver with the properties of the stream |
| 3310 |
that will be sent. The names of all these parameters start with |
| 3311 |
"sprop" for stream properties. Some of these "sprop" parameters are |
| 3312 |
limited by other payload or codec configuration parameters. For |
| 3313 |
example, the sprop-parameter-sets parameter is constrained by the |
| 3314 |
profile-level-id parameter. The media sender selects all "sprop" |
| 3315 |
parameters rather than the receiver. This uncommon characteristic of |
| 3316 |
the "sprop" parameters may be incompatible with some signaling |
| 3317 |
protocol concepts, in which case the use of these parameters SHOULD |
| 3318 |
be avoided. |
| 3319 |
|
| 3320 |
<span class="h3"><a class="selflink" id="section-7.1" href="#section-7.1">7.1</a>. Media Type Registration</span> |
| 3321 |
|
| 3322 |
The media subtype for the SVC codec has been allocated from the IETF |
| 3323 |
tree. |
| 3324 |
|
| 3325 |
The receiver MUST ignore any unspecified parameter. |
| 3326 |
|
| 3327 |
Informative note: Requiring that the receiver ignore unspecified |
| 3328 |
parameters allows for backward compatibility of future extensions. |
| 3329 |
For example, if a future specification that is backward compatible |
| 3330 |
to this specification specifies some new parameters, then a |
| 3331 |
receiver according to this specification is capable of receiving |
| 3332 |
data per the new payload but ignoring those parameters newly |
| 3333 |
specified in the new payload specification. This provision is |
| 3334 |
also present in [<a href="./rfc6184" title=""RTP Payload Format for H.264 Video"">RFC6184</a>]. |
| 3335 |
|
| 3336 |
Media Type name: video |
| 3337 |
|
| 3338 |
Media subtype name: H264-SVC |
| 3339 |
|
| 3340 |
Required parameters: none |
| 3341 |
|
| 3342 |
OPTIONAL parameters: |
| 3343 |
|
| 3344 |
In the following definitions of parameters, "the stream" or "the |
| 3345 |
NAL unit stream" refers to all NAL units conveyed in the current |
| 3346 |
RTP session in SST, and all NAL units conveyed in the current RTP |
| 3347 |
session and all NAL units conveyed in other RTP sessions that the |
| 3348 |
current RTP session depends on in MST. |
| 3349 |
|
| 3350 |
|
| 3351 |
|
| 3352 |
|
| 3353 |
|
| 3354 |
|
| 3355 |
|
| 3356 |
<span class="grey">Wenger, et al. Standards Track [Page 60]</span></pre> |
| 3357 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-61" ></span> |
| 3358 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 3359 |
|
| 3360 |
|
| 3361 |
profile-level-id: |
| 3362 |
A base16 [<a href="./rfc4648" title=""The Base16, Base32, and Base64 Data Encodings"">RFC4648</a>] (hexadecimal) representation of the |
| 3363 |
following three bytes in the sequence parameter set or subset |
| 3364 |
sequence parameter set NAL unit specified in [<a href="#ref-H.264" title=""Advanced video coding for generic audiovisual services"">H.264</a>]: 1) |
| 3365 |
profile_idc; 2) a byte herein referred to as profile-iop, |
| 3366 |
composed of the values of constraint_set0_flag, |
| 3367 |
constraint_set1_flag, constraint_set2_flag, |
| 3368 |
constraint_set3_flag, constraint_set4_flag, |
| 3369 |
constraint_set5_flag, and reserved_zero_2bits, in bit- |
| 3370 |
significance order, starting from the most-significant bit, and |
| 3371 |
3) level_idc. Note that reserved_zero_2bits is required to be |
| 3372 |
equal to 0 in [<a href="#ref-H.264" title=""Advanced video coding for generic audiovisual services"">H.264</a>], but other values for it may be specified |
| 3373 |
in the future by ITU-T or ISO/IEC. |
| 3374 |
|
| 3375 |
The profile-level-id parameter indicates the default sub- |
| 3376 |
profile, i.e., the subset of coding tools that may have been |
| 3377 |
used to generate the stream or that the receiver supports, and |
| 3378 |
the default level of the stream or the one that the receiver |
| 3379 |
supports. |
| 3380 |
|
| 3381 |
The default sub-profile is indicated collectively by the |
| 3382 |
profile_idc byte and some fields in the profile-iop byte. |
| 3383 |
Depending on the values of the fields in the profile-iop byte, |
| 3384 |
the default sub-profile may be the same set of coding tools |
| 3385 |
supported by one profile, or a common subset of coding tools of |
| 3386 |
multiple profiles, as specified in Subsection G.7.4.2.1.1 of |
| 3387 |
[<a href="#ref-H.264" title=""Advanced video coding for generic audiovisual services"">H.264</a>]. The default level is indicated by the level_idc byte, |
| 3388 |
and, when profile_idc is equal to 66, 77, or 88 (the Baseline, |
| 3389 |
Main, or Extended profile) and level_idc is equal to 11, |
| 3390 |
additionally by bit 4 (constraint_set3_flag) of the profile-iop |
| 3391 |
byte. When profile_idc is equal to 66, 77, or 88 (the |
| 3392 |
Baseline, Main, or Extended profile) and level_idc is equal to |
| 3393 |
11, and bit 4 (constraint_set3_flag) of the profile-iop byte is |
| 3394 |
equal to 1, the default level is Level 1b. |
| 3395 |
|
| 3396 |
Table 13 lists all profiles defined in Annexes A and G of |
| 3397 |
[<a href="#ref-H.264" title=""Advanced video coding for generic audiovisual services"">H.264</a>] and, for each of the profiles, the possible |
| 3398 |
combinations of profile_idc and profile-iop that represent the |
| 3399 |
same sub-profile. |
| 3400 |
|
| 3401 |
Table 13. Combinations of profile_idc and profile-iop |
| 3402 |
representing the same sub-profile corresponding to the full set |
| 3403 |
of coding tools supported by one profile. In the following, x |
| 3404 |
may be either 0 or 1, while the profile names are indicated as |
| 3405 |
follows. CB: Constrained Baseline profile, B: Baseline |
| 3406 |
profile, M: Main profile, E: Extended profile, H: High profile, |
| 3407 |
H10: High 10 profile, H42: High 4:2:2 profile, H44: High 4:4:4 |
| 3408 |
Predictive profile, H10I: High 10 Intra profile, H42I: High |
| 3409 |
|
| 3410 |
|
| 3411 |
|
| 3412 |
<span class="grey">Wenger, et al. Standards Track [Page 61]</span></pre> |
| 3413 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-62" ></span> |
| 3414 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 3415 |
|
| 3416 |
|
| 3417 |
4:2:2 Intra profile, H44I: High 4:4:4 Intra profile, C44I: |
| 3418 |
CAVLC 4:4:4 Intra profile, SB: Scalable Baseline profile, SH: |
| 3419 |
Scalable High profile, and SHI: Scalable High Intra profile. |
| 3420 |
|
| 3421 |
Profile profile_idc profile-iop |
| 3422 |
(hexadecimal) (binary) |
| 3423 |
|
| 3424 |
CB 42 (B) x1xx0000 |
| 3425 |
same as: 4D (M) 1xxx0000 |
| 3426 |
same as: 58 (E) 11xx0000 |
| 3427 |
B 42 (B) x0xx0000 |
| 3428 |
same as: 58 (E) 10xx0000 |
| 3429 |
M 4D (M) 0x0x0000 |
| 3430 |
E 58 00xx0000 |
| 3431 |
H 64 00000000 |
| 3432 |
H10 6E 00000000 |
| 3433 |
H42 7A 00000000 |
| 3434 |
H44 F4 00000000 |
| 3435 |
H10I 6E 00010000 |
| 3436 |
H42I 7A 00010000 |
| 3437 |
H44I F4 00010000 |
| 3438 |
C44I 2C 00010000 |
| 3439 |
SB 53 x0000000 |
| 3440 |
SH 56 0x000000 |
| 3441 |
SHI 56 0x010000 |
| 3442 |
|
| 3443 |
For example, in the table above, profile_idc equal to 58 |
| 3444 |
(Extended) with profile-iop equal to 11xx0000 indicates the |
| 3445 |
same sub-profile corresponding to profile_idc equal to 42 |
| 3446 |
(Baseline) with profile-iop equal to x1xx0000. Note that other |
| 3447 |
combinations of profile_idc and profile-iop (not listed in |
| 3448 |
Table 13) may represent a sub-profile equivalent to the common |
| 3449 |
subset of coding tools for more than one profile. Note also |
| 3450 |
that a decoder conforming to a certain profile may be able to |
| 3451 |
decode bitstreams conforming to other profiles. |
| 3452 |
|
| 3453 |
If profile-level-id is used to indicate stream properties, it |
| 3454 |
indicates that, to decode the stream, the minimum subset of |
| 3455 |
coding tools a decoder has to support is the default sub- |
| 3456 |
profile, and the lowest level the decoder has to support is the |
| 3457 |
default level. |
| 3458 |
|
| 3459 |
If the profile-level-id parameter is used for capability |
| 3460 |
exchange or session setup, it indicates the subset of coding |
| 3461 |
tools, which is equal to the default sub-profile, that the |
| 3462 |
codec supports for both receiving and sending. If max-recv- |
| 3463 |
level is not present, the default level from profile-level-id |
| 3464 |
indicates the highest level the codec wishes to support. If |
| 3465 |
|
| 3466 |
|
| 3467 |
|
| 3468 |
<span class="grey">Wenger, et al. Standards Track [Page 62]</span></pre> |
| 3469 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-63" ></span> |
| 3470 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 3471 |
|
| 3472 |
|
| 3473 |
max-recv-level is present, it indicates the highest level the |
| 3474 |
codec supports for receiving. For either receiving or sending, |
| 3475 |
all levels that are lower than the highest level supported MUST |
| 3476 |
also be supported. |
| 3477 |
|
| 3478 |
Informative note: Capability exchange and session setup |
| 3479 |
procedures should provide means to list the capabilities for |
| 3480 |
each supported sub-profile separately. For example, the |
| 3481 |
one-of-N codec selection procedure of the SDP Offer/Answer |
| 3482 |
model can be used (<a href="./rfc3264#section-10.2">Section 10.2 of [RFC3264]</a>). The one-of-N |
| 3483 |
codec selection procedure may also be used to provide |
| 3484 |
different combinations of profile_idc and profile-iop that |
| 3485 |
represent the same sub-profile. When there are many |
| 3486 |
different combinations of profile_idc and profile-iop that |
| 3487 |
represent the same sub-profile, using the one-of-N codec |
| 3488 |
selection procedure may result in a fairly large SDP |
| 3489 |
message. Therefore, a receiver should understand the |
| 3490 |
different equivalent combinations of profile_idc and |
| 3491 |
profile-iop that represent the same sub-profile, and be |
| 3492 |
ready to accept an offer using any of the equivalent |
| 3493 |
combinations. |
| 3494 |
|
| 3495 |
If no profile-level-id is present, the Baseline Profile without |
| 3496 |
additional constraints at Level 1 MUST be implied. |
| 3497 |
|
| 3498 |
max-recv-level: |
| 3499 |
This parameter MAY be used to indicate the highest level a |
| 3500 |
receiver supports when the highest level is higher than the |
| 3501 |
default level (the level indicated by profile-level-id). The |
| 3502 |
value of max-recv-level is a base16 (hexadecimal) |
| 3503 |
representation of the two bytes after the syntax element |
| 3504 |
profile_idc in the sequence parameter set NAL unit specified in |
| 3505 |
[<a href="#ref-H.264" title=""Advanced video coding for generic audiovisual services"">H.264</a>]: profile-iop (as defined above) and level_idc. If (the |
| 3506 |
level_idc byte of max-recv-level is equal to 11 and bit 4 of |
| 3507 |
the profile-iop byte of max-recv-level is equal to 1) or (the |
| 3508 |
level_idc byte of max-recv-level is equal to 9 and bit 4 of the |
| 3509 |
profile-iop byte of max-recv-level is equal to 0), the highest |
| 3510 |
level the receiver supports is Level 1b. Otherwise, the |
| 3511 |
highest level the receiver supports is equal to the level_idc |
| 3512 |
byte of max-recv-level divided by 10. |
| 3513 |
|
| 3514 |
max-recv-level MUST NOT be present if the highest level the |
| 3515 |
receiver supports is not higher than the default level. |
| 3516 |
|
| 3517 |
max-recv-base-level: |
| 3518 |
This parameter MAY be used to indicate the highest level a |
| 3519 |
receiver supports for the base layer when negotiating an SVC |
| 3520 |
stream. The value of max-recv-base-level is a base16 |
| 3521 |
|
| 3522 |
|
| 3523 |
|
| 3524 |
<span class="grey">Wenger, et al. Standards Track [Page 63]</span></pre> |
| 3525 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-64" ></span> |
| 3526 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 3527 |
|
| 3528 |
|
| 3529 |
(hexadecimal) representation of the two bytes after the syntax |
| 3530 |
element profile_idc in the sequence parameter set NAL unit |
| 3531 |
specified in [<a href="#ref-H.264" title=""Advanced video coding for generic audiovisual services"">H.264</a>]: profile-iop (as defined above) and |
| 3532 |
level_idc. If (the level_idc byte of max-recv-level is equal |
| 3533 |
to 11 and bit 4 of the profile-iop byte of max-recv-level is |
| 3534 |
equal to 1) or (the level_idc byte of max-recv-level is equal |
| 3535 |
to 9 and bit 4 of the profile-iop byte of max-recv-level is |
| 3536 |
equal to 0), the highest level the receiver supports for the |
| 3537 |
base layer is Level 1b. Otherwise, the highest level the |
| 3538 |
receiver supports for the base layer is equal to the level_idc |
| 3539 |
byte of max-recv-level divided by 10. |
| 3540 |
|
| 3541 |
max-mbps, max-fs, max-cpb, max-dpb, and max-br: |
| 3542 |
The common properties of these parameters are specified in |
| 3543 |
[<a href="./rfc6184" title=""RTP Payload Format for H.264 Video"">RFC6184</a>]. |
| 3544 |
|
| 3545 |
max-mbps: This parameter is as specified in [<a href="./rfc6184" title=""RTP Payload Format for H.264 Video"">RFC6184</a>]. |
| 3546 |
|
| 3547 |
max-fs: This parameter is as specified in [<a href="./rfc6184" title=""RTP Payload Format for H.264 Video"">RFC6184</a>]. |
| 3548 |
|
| 3549 |
max-cpb: The value of max-cpb is an integer indicating the maximum |
| 3550 |
coded picture buffer size in units of 1000 bits for the VCL HRD |
| 3551 |
parameters and in units of 1200 bits for the NAL HRD |
| 3552 |
parameters. Note that this parameter does not use units of |
| 3553 |
cpbBrVclFactor and cpbBrNALFactor (see Table A-1 of [<a href="#ref-H.264" title=""Advanced video coding for generic audiovisual services"">H.264</a>]). |
| 3554 |
The max-cpb parameter signals that the receiver has more memory |
| 3555 |
than the minimum amount of coded picture buffer memory required |
| 3556 |
by the signaled highest level conveyed in the value of the |
| 3557 |
profile-level-id parameter or the max-recv-level parameter. |
| 3558 |
When max-cpb is signaled, the receiver MUST be able to decode |
| 3559 |
NAL unit streams that conform to the signaled highest level, |
| 3560 |
with the exception that the MaxCPB value in Table A-1 of |
| 3561 |
[<a href="#ref-H.264" title=""Advanced video coding for generic audiovisual services"">H.264</a>] for the signaled highest level is replaced with the |
| 3562 |
value of max-cpb (after taking cpbBrVclFactor and |
| 3563 |
cpbBrNALFactor into consideration when needed). The value of |
| 3564 |
max-cpb (after taking cpbBrVclFactor and cpbBrNALFactor into |
| 3565 |
consideration when needed) MUST be greater than or equal to the |
| 3566 |
value of MaxCPB given in Table A-1 of [<a href="#ref-H.264" title=""Advanced video coding for generic audiovisual services"">H.264</a>] for the highest |
| 3567 |
level. Senders MAY use this knowledge to construct coded video |
| 3568 |
streams with greater variation of bitrate than can be achieved |
| 3569 |
with the MaxCPB value in Table A-1 of [<a href="#ref-H.264" title=""Advanced video coding for generic audiovisual services"">H.264</a>]. |
| 3570 |
|
| 3571 |
|
| 3572 |
|
| 3573 |
|
| 3574 |
|
| 3575 |
|
| 3576 |
|
| 3577 |
|
| 3578 |
|
| 3579 |
|
| 3580 |
<span class="grey">Wenger, et al. Standards Track [Page 64]</span></pre> |
| 3581 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-65" ></span> |
| 3582 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 3583 |
|
| 3584 |
|
| 3585 |
Informative note: The coded picture buffer is used in the |
| 3586 |
Hypothetical Reference Decoder (HRD, Annex C) of [<a href="#ref-H.264" title=""Advanced video coding for generic audiovisual services"">H.264</a>]. |
| 3587 |
The use of the HRD is recommended in SVC encoders to verify |
| 3588 |
that the produced bitstream conforms to the standard and to |
| 3589 |
control the output bitrate. Thus, the coded picture buffer |
| 3590 |
is conceptually independent of any other potential buffers |
| 3591 |
in the receiver, including de-interleaving, re-multiplexing, |
| 3592 |
and de-jitter buffers. The coded picture buffer need not be |
| 3593 |
implemented in decoders as specified in Annex C of [<a href="#ref-H.264" title=""Advanced video coding for generic audiovisual services"">H.264</a>]; |
| 3594 |
standard-compliant decoders can have any buffering |
| 3595 |
arrangements provided that they can decode standard- |
| 3596 |
compliant bitstreams. Thus, in practice, the input buffer |
| 3597 |
for video decoder can be integrated with the de- |
| 3598 |
interleaving, re-multiplexing, and de-jitter buffers of the |
| 3599 |
receiver. |
| 3600 |
|
| 3601 |
max-dpb: This parameter is as specified in [<a href="./rfc6184" title=""RTP Payload Format for H.264 Video"">RFC6184</a>]. |
| 3602 |
|
| 3603 |
max-br: The value of max-br is an integer indicating the maximum |
| 3604 |
video bitrate in units of 1000 bits per second for the VCL HRD |
| 3605 |
parameters and in units of 1200 bits per second for the NAL HRD |
| 3606 |
parameters. Note that this parameter does not use units of |
| 3607 |
cpbBrVclFactor and cpbBrNALFactor (see Table A-1 of [<a href="#ref-H.264" title=""Advanced video coding for generic audiovisual services"">H.264</a>]). |
| 3608 |
|
| 3609 |
The max-br parameter signals that the video decoder of the |
| 3610 |
receiver is capable of decoding video at a higher bitrate than |
| 3611 |
is required by the signaled highest level conveyed in the value |
| 3612 |
of the profile-level-id parameter or the max-recv-level |
| 3613 |
parameter. |
| 3614 |
|
| 3615 |
When max-br is signaled, the video codec of the receiver MUST |
| 3616 |
be able to decode NAL unit streams that conform to the signaled |
| 3617 |
highest level, with the following exceptions in the limits |
| 3618 |
specified by the highest level: |
| 3619 |
|
| 3620 |
o The value of max-br (after taking cpbBrVclFactor and |
| 3621 |
cpbBrNALFactor into consideration when needed) replaces the |
| 3622 |
MaxBR value in Table A-1 of [<a href="#ref-H.264" title=""Advanced video coding for generic audiovisual services"">H.264</a>] for the highest level. |
| 3623 |
|
| 3624 |
o When the max-cpb parameter is not present, the result of the |
| 3625 |
following formula replaces the value of MaxCPB in Table A-1 |
| 3626 |
of [<a href="#ref-H.264" title=""Advanced video coding for generic audiovisual services"">H.264</a>]: (MaxCPB of the signaled level) * max-br / (MaxBR |
| 3627 |
of the signaled highest level). |
| 3628 |
|
| 3629 |
For example, if a receiver signals capability for Main profile |
| 3630 |
Level 1.2 with max-br equal to 1550, this indicates a maximum |
| 3631 |
video bitrate of 1550 kbits/sec for VCL HRD parameters, a |
| 3632 |
|
| 3633 |
|
| 3634 |
|
| 3635 |
|
| 3636 |
<span class="grey">Wenger, et al. Standards Track [Page 65]</span></pre> |
| 3637 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-66" ></span> |
| 3638 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 3639 |
|
| 3640 |
|
| 3641 |
maximum video bitrate of 1860 kbits/sec for NAL HRD parameters, |
| 3642 |
and a CPB size of 4036458 bits (1550000 / 384000 * 1000 * |
| 3643 |
1000). |
| 3644 |
|
| 3645 |
The value of max-br (after taking cpbBrVclFactor and |
| 3646 |
cpbBrNALFactor into consideration when needed) MUST be greater |
| 3647 |
than or equal to the value MaxBR given in Table A-1 of [<a href="#ref-H.264" title=""Advanced video coding for generic audiovisual services"">H.264</a>] |
| 3648 |
for the signaled highest level. |
| 3649 |
|
| 3650 |
Senders MAY use this knowledge to send higher-bitrate video as |
| 3651 |
allowed in the level definition of SVC, to achieve improved |
| 3652 |
video quality. |
| 3653 |
|
| 3654 |
Informative note: This parameter was added primarily to |
| 3655 |
complement a similar codepoint in the ITU-T Recommendation |
| 3656 |
H.245, so as to facilitate signaling gateway designs. No |
| 3657 |
assumption can be made from the value of this parameter that |
| 3658 |
the network is capable of handling such bitrates at any |
| 3659 |
given time. In particular, no conclusion can be drawn that |
| 3660 |
the signaled bitrate is possible under congestion control |
| 3661 |
constraints. |
| 3662 |
|
| 3663 |
redundant-pic-cap: |
| 3664 |
This parameter is as specified in [<a href="./rfc6184" title=""RTP Payload Format for H.264 Video"">RFC6184</a>]. |
| 3665 |
|
| 3666 |
sprop-parameter-sets: |
| 3667 |
This parameter MAY be used to convey any sequence parameter |
| 3668 |
set, subset sequence parameter set, and picture parameter set |
| 3669 |
NAL units (herein referred to as the initial parameter set NAL |
| 3670 |
units) that can be placed in the NAL unit stream to precede any |
| 3671 |
other NAL units in decoding order and that are associated with |
| 3672 |
the default level of profile-level-id. The parameter MUST NOT |
| 3673 |
be used to indicate codec capability in any capability exchange |
| 3674 |
procedure. The value of the parameter is a comma (',') |
| 3675 |
separated list of base64 [<a href="./rfc4648" title=""The Base16, Base32, and Base64 Data Encodings"">RFC4648</a>] representations of the |
| 3676 |
parameter set NAL units as specified in Sections <a href="#section-7.3.2.1">7.3.2.1</a>, |
| 3677 |
7.3.2.2, and G.7.3.2.1 of [<a href="#ref-H.264" title=""Advanced video coding for generic audiovisual services"">H.264</a>]. Note that the number of |
| 3678 |
bytes in a parameter set NAL unit is typically less than 10, |
| 3679 |
but a picture parameter set NAL unit can contain several |
| 3680 |
hundreds of bytes. |
| 3681 |
|
| 3682 |
Informative note: When several payload types are offered in |
| 3683 |
the SDP Offer/Answer model, each with its own sprop- |
| 3684 |
parameter-sets parameter, then the receiver cannot assume |
| 3685 |
that those parameter sets do not use conflicting storage |
| 3686 |
locations (i.e., identical values of parameter set |
| 3687 |
|
| 3688 |
|
| 3689 |
|
| 3690 |
|
| 3691 |
|
| 3692 |
<span class="grey">Wenger, et al. Standards Track [Page 66]</span></pre> |
| 3693 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-67" ></span> |
| 3694 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 3695 |
|
| 3696 |
|
| 3697 |
identifiers). Therefore, a receiver should buffer all |
| 3698 |
sprop-parameter-sets and make them available to the decoder |
| 3699 |
instance that decodes a certain payload type. |
| 3700 |
|
| 3701 |
sprop-level-parameter-sets: |
| 3702 |
This parameter MAY be used to convey any sequence, subset |
| 3703 |
sequence, and picture parameter set NAL units (herein referred |
| 3704 |
to as the initial parameter set NAL units) that can be placed |
| 3705 |
in the NAL unit stream to precede any other NAL units in |
| 3706 |
decoding order and that are associated with one or more levels |
| 3707 |
different than the default level of profile-level-id. The |
| 3708 |
parameter MUST NOT be used to indicate codec capability in any |
| 3709 |
capability exchange procedure. |
| 3710 |
|
| 3711 |
The sprop-level-parameter-sets parameter contains parameter |
| 3712 |
sets for one or more levels that are different than the default |
| 3713 |
level. All parameter sets targeted for use when one level of |
| 3714 |
the default sub-profile is accepted by a receiver are clustered |
| 3715 |
and prefixed with a three-byte field that has the same syntax |
| 3716 |
as profile-level-id. This enables the receiver to install the |
| 3717 |
parameter sets for the accepted level and discard the rest. |
| 3718 |
The three-byte field is named PLId, and all parameter sets |
| 3719 |
associated with one level are named PSL, which has the same |
| 3720 |
syntax as sprop-parameter-sets. Parameter sets for each level |
| 3721 |
are represented in the form of PLId:PSL, i.e., PLId followed by |
| 3722 |
a colon (':') and the base64 [<a href="./rfc4648" title=""The Base16, Base32, and Base64 Data Encodings"">RFC4648</a>] representation of the |
| 3723 |
initial parameter set NAL units for the level. Each pair of |
| 3724 |
PLId:PSL is also separated by a colon. Note that a PSL can |
| 3725 |
contain multiple parameter sets for that level, separated with |
| 3726 |
commas (','). |
| 3727 |
|
| 3728 |
The subset of coding tools indicated by each PLId field MUST be |
| 3729 |
equal to the default sub-profile, and the level indicated by |
| 3730 |
each PLId field MUST be different than the default level. |
| 3731 |
|
| 3732 |
Informative note: This parameter allows for efficient level |
| 3733 |
downgrade or upgrade in SDP Offer/Answer and out-of-band |
| 3734 |
transport of parameter sets, simultaneously. |
| 3735 |
|
| 3736 |
in-band-parameter-sets: |
| 3737 |
This parameter MAY be used to indicate a receiver capability. |
| 3738 |
The value MAY be equal to either 0 or 1. The value 1 indicates |
| 3739 |
that the receiver discards out-of-band parameter sets in sprop- |
| 3740 |
parameter-sets and sprop-level-parameter-sets, therefore the |
| 3741 |
sender MUST transmit all parameter sets in-band. The value 0 |
| 3742 |
indicates that the receiver utilizes out-of-band parameter sets |
| 3743 |
included in sprop-parameter-sets and/or sprop-level-parameter- |
| 3744 |
sets. However, in this case, the sender MAY still choose to |
| 3745 |
|
| 3746 |
|
| 3747 |
|
| 3748 |
<span class="grey">Wenger, et al. Standards Track [Page 67]</span></pre> |
| 3749 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-68" ></span> |
| 3750 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 3751 |
|
| 3752 |
|
| 3753 |
send parameter sets in-band. When the parameter is not |
| 3754 |
present, this receiver capability is not specified, and |
| 3755 |
therefore the sender MAY send out-of-band parameter sets only, |
| 3756 |
or it MAY send in-band-parameter-sets only, or it MAY send |
| 3757 |
both. |
| 3758 |
|
| 3759 |
packetization-mode: |
| 3760 |
This parameter is as specified in [<a href="./rfc6184" title=""RTP Payload Format for H.264 Video"">RFC6184</a>]. When the mst-mode |
| 3761 |
parameter is present, the value of this parameter is |
| 3762 |
additionally constrained as follows. If mst-mode is equal to |
| 3763 |
"NI-T", "NI-C", or "NI-TC", packetization-mode MUST NOT be |
| 3764 |
equal to 2. Otherwise, (mst-mode is equal to "I-C"), |
| 3765 |
packetization-mode MUST be equal to 2. |
| 3766 |
|
| 3767 |
sprop-interleaving-depth: |
| 3768 |
This parameter is as specified in [<a href="./rfc6184" title=""RTP Payload Format for H.264 Video"">RFC6184</a>]. |
| 3769 |
|
| 3770 |
sprop-deint-buf-req: |
| 3771 |
This parameter is as specified in [<a href="./rfc6184" title=""RTP Payload Format for H.264 Video"">RFC6184</a>]. |
| 3772 |
|
| 3773 |
deint-buf-cap: |
| 3774 |
This parameter is as specified in [<a href="./rfc6184" title=""RTP Payload Format for H.264 Video"">RFC6184</a>]. |
| 3775 |
|
| 3776 |
sprop-init-buf-time: |
| 3777 |
This parameter is as specified in [<a href="./rfc6184" title=""RTP Payload Format for H.264 Video"">RFC6184</a>]. |
| 3778 |
|
| 3779 |
sprop-max-don-diff: |
| 3780 |
This parameter is as specified in [<a href="./rfc6184" title=""RTP Payload Format for H.264 Video"">RFC6184</a>]. |
| 3781 |
|
| 3782 |
max-rcmd-nalu-size: |
| 3783 |
This parameter is as specified in [<a href="./rfc6184" title=""RTP Payload Format for H.264 Video"">RFC6184</a>]. |
| 3784 |
|
| 3785 |
mst-mode: |
| 3786 |
This parameter MAY be used to signal the properties of a NAL |
| 3787 |
unit stream or the capabilities of a receiver implementation. |
| 3788 |
If this parameter is present, multi-session transmission MUST |
| 3789 |
be used. Otherwise (this parameter is not present), single- |
| 3790 |
session transmission MUST be used. When this parameter is |
| 3791 |
present, the following applies. When the value of mst-mode is |
| 3792 |
equal to "NI-T", the NI-T mode MUST be used. When the value of |
| 3793 |
mst-mode is equal to "NI-C", the NI-C mode MUST be used. When |
| 3794 |
the value of mst-mode is equal to "NI-TC", the NI-TC mode MUST |
| 3795 |
be used. When the value of mst-mode is equal to "I-C", the I-C |
| 3796 |
mode MUST be used. The value of mst-mode MUST have one of the |
| 3797 |
following tokens: "NI-T", "NI-C", "NI-TC", or "I-C". |
| 3798 |
|
| 3799 |
All RTP sessions in an MST MUST have the same value of mst- |
| 3800 |
mode. |
| 3801 |
|
| 3802 |
|
| 3803 |
|
| 3804 |
<span class="grey">Wenger, et al. Standards Track [Page 68]</span></pre> |
| 3805 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-69" ></span> |
| 3806 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 3807 |
|
| 3808 |
|
| 3809 |
sprop-mst-csdon-always-present: |
| 3810 |
This parameter MUST NOT be present when mst-mode is not present |
| 3811 |
or the value of mst-mode is equal to "NI-T" or "I-C". This |
| 3812 |
parameter signals the properties of the NAL unit stream. When |
| 3813 |
sprop-mst-csdon-always-present is present and the value is |
| 3814 |
equal to 1, packetization-mode MUST be equal to 1, and all the |
| 3815 |
RTP packets carrying the NAL unit stream MUST be STAP-A packets |
| 3816 |
containing a PACSI NAL unit that further contains the DONC |
| 3817 |
field or NI-MTAP packets with the J field equal to 1. When |
| 3818 |
sprop-mst-csdon-always-present is present and the value is |
| 3819 |
equal to 1, the CS-DON value of any particular NAL unit can be |
| 3820 |
derived solely according to information in the packet |
| 3821 |
containing the NAL unit. |
| 3822 |
|
| 3823 |
When sprop-mst-csdon-always-present is present in the current |
| 3824 |
RTP session, it MUST be present also in all the RTP sessions |
| 3825 |
the current RTP session depends on and the value of sprop-mst- |
| 3826 |
csdon-always-present is identical for the current RTP session |
| 3827 |
and all the RTP sessions on which the current RTP session |
| 3828 |
depends. |
| 3829 |
|
| 3830 |
sprop-mst-remux-buf-size: |
| 3831 |
This parameter MUST NOT be present when mst-mode is not present |
| 3832 |
or the value of mst-mode is equal to "NI-T". This parameter |
| 3833 |
MUST be present when mst-mode is present and the value of mst- |
| 3834 |
mode is equal to "NI-C", "NI-TC", or "I-C". |
| 3835 |
|
| 3836 |
This parameter signals the properties of the NAL unit stream. |
| 3837 |
It MUST be set to a value one less than the minimum re- |
| 3838 |
multiplexing buffer size (in NAL units), so that it is |
| 3839 |
guaranteed that receivers can reconstruct NAL unit decoding |
| 3840 |
order as specified in Sub<a href="#section-6.2.2">section 6.2.2</a>. |
| 3841 |
|
| 3842 |
The value of sprop-mst-remux-buf-size MUST be an integer in the |
| 3843 |
range of 0 to 32767, inclusive. |
| 3844 |
|
| 3845 |
sprop-remux-buf-req: |
| 3846 |
This parameter MUST NOT be present when mst-mode is not present |
| 3847 |
or the value of mst-mode is equal to "NI-T". It MUST be |
| 3848 |
present when mst-mode is present and the value of mst-mode is |
| 3849 |
equal to "NI-C", "NI-TC", or "I-C". |
| 3850 |
|
| 3851 |
sprop-remux-buf-req signals the required size of the re- |
| 3852 |
multiplexing buffer for the NAL unit stream. It is guaranteed |
| 3853 |
that receivers can recover the decoding order of the received |
| 3854 |
NAL units from the current RTP session and the RTP sessions the |
| 3855 |
|
| 3856 |
|
| 3857 |
|
| 3858 |
|
| 3859 |
|
| 3860 |
<span class="grey">Wenger, et al. Standards Track [Page 69]</span></pre> |
| 3861 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-70" ></span> |
| 3862 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 3863 |
|
| 3864 |
|
| 3865 |
current RTP session depends on as specified in <a href="#section-6.2.2">Section 6.2.2</a>, |
| 3866 |
when the re-multiplexing buffer size is of at least the value |
| 3867 |
of sprop-remux-buf-req in units of bytes. |
| 3868 |
|
| 3869 |
The value of sprop-remux-buf-req MUST be an integer in the |
| 3870 |
range of 0 to 4294967295, inclusive. |
| 3871 |
|
| 3872 |
remux-buf-cap: |
| 3873 |
This parameter MUST NOT be present when mst-mode is not present |
| 3874 |
or the value of mst-mode is equal to "NI-T". This parameter |
| 3875 |
MAY be used to signal the capabilities of a receiver |
| 3876 |
implementation and indicates the amount of re-multiplexing |
| 3877 |
buffer space in units of bytes that the receiver has available |
| 3878 |
for recovering the NAL unit decoding order as specified in |
| 3879 |
<a href="#section-6.2.2">Section 6.2.2</a>. A receiver is able to handle any NAL unit |
| 3880 |
stream for which the value of the sprop-remux-buf-req parameter |
| 3881 |
is smaller than or equal to this parameter. |
| 3882 |
|
| 3883 |
If the parameter is not present, then a value of 0 MUST be used |
| 3884 |
for remux-buf-cap. The value of remux-buf-cap MUST be an |
| 3885 |
integer in the range of 0 to 4294967295, inclusive. |
| 3886 |
|
| 3887 |
sprop-remux-init-buf-time: |
| 3888 |
This parameter MAY be used to signal the properties of the NAL |
| 3889 |
unit stream. The parameter MUST NOT be present if mst-mode is |
| 3890 |
not present or the value of mst-mode is equal to "NI-T". |
| 3891 |
|
| 3892 |
The parameter signals the initial buffering time that a |
| 3893 |
receiver MUST wait before starting to recover the NAL unit |
| 3894 |
decoding order as specified in <a href="#section-6.2.2">Section 6.2.2</a> of this memo. |
| 3895 |
|
| 3896 |
The parameter is coded as a non-negative base10 integer |
| 3897 |
representation in clock ticks of a 90-kHz clock. If the |
| 3898 |
parameter is not present, then no initial buffering time value |
| 3899 |
is defined. Otherwise, the value of sprop-remux-init-buf-time |
| 3900 |
MUST be an integer in the range of 0 to 4294967295, inclusive. |
| 3901 |
|
| 3902 |
sprop-mst-max-don-diff: |
| 3903 |
This parameter MAY be used to signal the properties of the NAL |
| 3904 |
unit stream. It MUST NOT be used to signal transmitter or |
| 3905 |
receiver or codec capabilities. The parameter MUST NOT be |
| 3906 |
present if mst-mode is not present or the value of mst-mode is |
| 3907 |
equal to "NI-T". sprop-mst-max-don-diff is an integer in the |
| 3908 |
range of 0 to 32767, inclusive. If sprop-mst-max-don-diff is |
| 3909 |
not present, the value of the parameter is unspecified. sprop- |
| 3910 |
mst-max-don-diff is calculated same as sprop-max-don-diff as |
| 3911 |
specified in [<a href="./rfc6184" title=""RTP Payload Format for H.264 Video"">RFC6184</a>], with decoding order number being |
| 3912 |
replaced by cross-session decoding order number. |
| 3913 |
|
| 3914 |
|
| 3915 |
|
| 3916 |
<span class="grey">Wenger, et al. Standards Track [Page 70]</span></pre> |
| 3917 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-71" ></span> |
| 3918 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 3919 |
|
| 3920 |
|
| 3921 |
sprop-scalability-info: |
| 3922 |
This parameter MAY be used to convey the NAL unit containing |
| 3923 |
the scalability information SEI message as specified in Annex G |
| 3924 |
of [<a href="#ref-H.264" title=""Advanced video coding for generic audiovisual services"">H.264</a>]. This parameter MAY be used to signal the contained |
| 3925 |
layers of an SVC bitstream. The parameter MUST NOT be used to |
| 3926 |
indicate codec capability in any capability exchange procedure. |
| 3927 |
The value of the parameter is the base64 [<a href="./rfc4648" title=""The Base16, Base32, and Base64 Data Encodings"">RFC4648</a>] |
| 3928 |
representation of the NAL unit containing the scalability |
| 3929 |
information SEI message. If present, the NAL unit MUST contain |
| 3930 |
only one SEI message that is a scalability information SEI |
| 3931 |
message. |
| 3932 |
|
| 3933 |
This parameter MAY be used in an offering or declarative SDP |
| 3934 |
message to indicate what layers (operation points) can be |
| 3935 |
provided. A receiver MAY indicate its choice of one layer |
| 3936 |
using the optional media type parameter scalable-layer-id. |
| 3937 |
|
| 3938 |
scalable-layer-id: |
| 3939 |
This parameter MAY be used to signal a receiver's choice of the |
| 3940 |
offers or declared operation points or layers using sprop- |
| 3941 |
scalability-info or sprop-operation-point-info. The value of |
| 3942 |
scalable-layer-id is a base16 representation of the layer_id[ i |
| 3943 |
] syntax element in the scalability information SEI message as |
| 3944 |
specified in Annex G of [<a href="#ref-H.264" title=""Advanced video coding for generic audiovisual services"">H.264</a>] or layer-ID contained in sprop- |
| 3945 |
operation-point-info. |
| 3946 |
|
| 3947 |
sprop-operation-point-info: |
| 3948 |
This parameter MAY be used to describe the operation points of |
| 3949 |
an RTP session. The value of this parameter consists of a |
| 3950 |
comma-separated list of operation-point-description vectors. |
| 3951 |
The values given by the operation-point-description vectors are |
| 3952 |
the same as, or are derived from, the values that would be |
| 3953 |
given for a scalable layer in the scalability information SEI |
| 3954 |
message as specified in Annex G of [<a href="#ref-H.264" title=""Advanced video coding for generic audiovisual services"">H.264</a>], where the term |
| 3955 |
scalable layer in the scalability information SEI message |
| 3956 |
refers to all NAL units associated with the same values of |
| 3957 |
temporal_id, dependency_id, and quality_id. In this memo, such |
| 3958 |
a set of NAL units is called an operation point. |
| 3959 |
|
| 3960 |
Each operation-point-description vector has ten elements, |
| 3961 |
provided as a comma-separated list of values as defined below. |
| 3962 |
The first value of the operation-point-description vector is |
| 3963 |
preceded by a '<', and the last value of the operation-point- |
| 3964 |
description vector is followed by a '>'. If the sprop- |
| 3965 |
operation-point-info is followed by exactly one operation- |
| 3966 |
point-description vector, this describes the highest operation |
| 3967 |
point contained in the RTP session. If there are two or more |
| 3968 |
|
| 3969 |
|
| 3970 |
|
| 3971 |
|
| 3972 |
<span class="grey">Wenger, et al. Standards Track [Page 71]</span></pre> |
| 3973 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-72" ></span> |
| 3974 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 3975 |
|
| 3976 |
|
| 3977 |
operation-point-description vectors, the first describes the |
| 3978 |
lowest and the last describes the highest operation point |
| 3979 |
contained in the RTP session. |
| 3980 |
|
| 3981 |
The values given by the operation-point-description vector are |
| 3982 |
as follows, in the order listed: |
| 3983 |
|
| 3984 |
- layer-ID: This value specifies the layer identifier of the |
| 3985 |
operation point, which is identical to the layer_id that |
| 3986 |
would be indicated (for the same values of dependency_id, |
| 3987 |
quality_id, and temporal_id) in the scalability information |
| 3988 |
SEI message. This field MAY be empty, indicating that the |
| 3989 |
value is unspecified. When there are multiple operation- |
| 3990 |
point-description vectors with layer-ID, the values of |
| 3991 |
layer-ID do not need to be consecutive. |
| 3992 |
|
| 3993 |
- temporal-ID: This value specifies the temporal_id of the |
| 3994 |
operation point. This field MUST NOT be empty. |
| 3995 |
|
| 3996 |
- dependency-ID: This values specifies the dependency_id of |
| 3997 |
the operation point. This field MUST NOT be empty. |
| 3998 |
|
| 3999 |
- quality-ID: This values specifies the quality_id of the |
| 4000 |
operation point. This field MUST NOT be empty. |
| 4001 |
|
| 4002 |
- profile-level-ID: This value specifies the profile-level-idc |
| 4003 |
of the operation point in the base16 format. The default |
| 4004 |
sub-profile or default level indicated by the parameter |
| 4005 |
profile-level-ID in the sprop-operation-point-info vector |
| 4006 |
SHALL be equal to or lower than the default sub-profile or |
| 4007 |
default level indicated by profile-level-id, which may be |
| 4008 |
either present or the default value is taken. This field |
| 4009 |
MAY be empty, indicating that the value is unspecified. |
| 4010 |
|
| 4011 |
- avg-framerate: This value specifies the average frame rate |
| 4012 |
of the operation point. This value is given as an integer |
| 4013 |
in frames per 256 seconds. The field MAY be empty, |
| 4014 |
indicating that the value is unspecified. |
| 4015 |
|
| 4016 |
- width: This value specifies the width dimension in pixels of |
| 4017 |
decoded frames for the operation point. This parameter is |
| 4018 |
not directly given in the scalability information SEI |
| 4019 |
message. This field MAY be empty, indicating that the value |
| 4020 |
is unspecified. |
| 4021 |
|
| 4022 |
|
| 4023 |
|
| 4024 |
|
| 4025 |
|
| 4026 |
|
| 4027 |
|
| 4028 |
<span class="grey">Wenger, et al. Standards Track [Page 72]</span></pre> |
| 4029 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-73" ></span> |
| 4030 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 4031 |
|
| 4032 |
|
| 4033 |
- height: This value gives the height dimension in pixels of |
| 4034 |
decoded frames for the operation point. This parameter is |
| 4035 |
not directly given in the scalability information SEI. This |
| 4036 |
field MAY be empty, indicating that the value is |
| 4037 |
unspecified. |
| 4038 |
|
| 4039 |
- avg-bitrate: This value specifies the average bitrate of the |
| 4040 |
operation point. This parameter is given as an integer in |
| 4041 |
kbits per second over the entire stream. Note that this |
| 4042 |
parameter is provided in the scalability information SEI |
| 4043 |
message in bits per second and calculated over a variable |
| 4044 |
time window. This field MAY be empty, indicating that the |
| 4045 |
value is unspecified. |
| 4046 |
|
| 4047 |
- max-bitrate: This value specifies the maximum bitrate of the |
| 4048 |
operation point. This parameter is given as an integer in |
| 4049 |
kbits per second and describes the maximum bitrate per each |
| 4050 |
one-second window. Note that this parameter is provided in |
| 4051 |
the scalability information SEI message in bits per second |
| 4052 |
and is calculated over a variable time window. This field |
| 4053 |
MAY be empty, indicating that the value is unspecified. |
| 4054 |
|
| 4055 |
Similarly to sprop-scalability-info, this parameter MAY be |
| 4056 |
used in an offering or declarative SDP message to indicate |
| 4057 |
what layers (operation points) can be provided. A receiver |
| 4058 |
MAY indicate its choice of the highest layer it wants to |
| 4059 |
send and/or receive using the optional media type parameter |
| 4060 |
scalable-layer-id. |
| 4061 |
|
| 4062 |
sprop-no-NAL-reordering-required: |
| 4063 |
This parameter MAY be used to signal the properties of the NAL |
| 4064 |
unit stream. This parameter MUST NOT be present when mst-mode |
| 4065 |
is not present or the value of mst-mode is not equal to "NI-T". |
| 4066 |
The presence of this parameter indicates that no reordering of |
| 4067 |
non-VCL or VCL NAL units is required for the decoding order |
| 4068 |
recovery process. |
| 4069 |
|
| 4070 |
sprop-avc-ready: |
| 4071 |
This parameter MAY be used to indicate the properties of the |
| 4072 |
NAL unit stream. The presence of this parameter indicates that |
| 4073 |
the RTP session, if used in SST, or used in MST combined with |
| 4074 |
other RTP sessions also with this parameter present, can be |
| 4075 |
processed by a [<a href="./rfc6184" title=""RTP Payload Format for H.264 Video"">RFC6184</a>] receiver. This parameter MAY be used |
| 4076 |
with RTP sessions with media subtype H264-SVC. |
| 4077 |
|
| 4078 |
Encoding considerations: |
| 4079 |
This media type is framed and binary; see Section 4.8 of <a href="./rfc4288">RFC</a> |
| 4080 |
<a href="./rfc4288">4288</a> [<a href="./rfc4288" title=""Media Type Specifications and Registration Procedures"">RFC4288</a>]. |
| 4081 |
|
| 4082 |
|
| 4083 |
|
| 4084 |
<span class="grey">Wenger, et al. Standards Track [Page 73]</span></pre> |
| 4085 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-74" ></span> |
| 4086 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 4087 |
|
| 4088 |
|
| 4089 |
Security considerations: |
| 4090 |
See <a href="./rfc6190#section-8">Section 8 of RFC 6190</a>. |
| 4091 |
|
| 4092 |
Published specification: |
| 4093 |
Please refer to <a href="./rfc6190">RFC 6190</a> and its <a href="#section-13">Section 13</a>. |
| 4094 |
|
| 4095 |
Additional information: |
| 4096 |
none |
| 4097 |
|
| 4098 |
File extensions: none |
| 4099 |
|
| 4100 |
Macintosh file type code: none |
| 4101 |
|
| 4102 |
Object identifier or OID: none |
| 4103 |
|
| 4104 |
Person & email address to contact for further information: |
| 4105 |
|
| 4106 |
Ye-Kui Wang, [email protected] |
| 4107 |
|
| 4108 |
Intended usage: COMMON |
| 4109 |
|
| 4110 |
Restrictions on usage: |
| 4111 |
This media type depends on RTP framing, and hence is only |
| 4112 |
defined for transfer via RTP [<a href="./rfc3550" title=""RTP: A Transport Protocol for Real-Time Applications"">RFC3550</a>]. Transport within other |
| 4113 |
framing protocols is not defined at this time. |
| 4114 |
|
| 4115 |
Interoperability considerations: |
| 4116 |
The media subtype name contains "SVC" to avoid potential |
| 4117 |
conflict with <a href="./rfc3984">RFC 3984</a> and its potential future replacement RTP |
| 4118 |
payload format for H.264 non-SVC profiles. |
| 4119 |
|
| 4120 |
Applications that use this media type: |
| 4121 |
Real-time video applications like video streaming, video |
| 4122 |
telephony, and video conferencing. |
| 4123 |
|
| 4124 |
Author: |
| 4125 |
|
| 4126 |
Ye-Kui Wang, [email protected] |
| 4127 |
|
| 4128 |
Change controller: |
| 4129 |
IETF Audio/Video Transport working group delegated from the |
| 4130 |
IESG. |
| 4131 |
|
| 4132 |
|
| 4133 |
|
| 4134 |
|
| 4135 |
|
| 4136 |
|
| 4137 |
|
| 4138 |
|
| 4139 |
|
| 4140 |
<span class="grey">Wenger, et al. Standards Track [Page 74]</span></pre> |
| 4141 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-75" ></span> |
| 4142 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 4143 |
|
| 4144 |
|
| 4145 |
<span class="h3"><a class="selflink" id="section-7.2" href="#section-7.2">7.2</a>. SDP Parameters</span> |
| 4146 |
|
| 4147 |
<span class="h4"><a class="selflink" id="section-7.2.1" href="#section-7.2.1">7.2.1</a>. Mapping of Payload Type Parameters to SDP</span> |
| 4148 |
|
| 4149 |
The media type video/H264-SVC string is mapped to fields in the |
| 4150 |
Session Description Protocol (SDP) as follows: |
| 4151 |
|
| 4152 |
o The media name in the "m=" line of SDP MUST be video. |
| 4153 |
|
| 4154 |
o The encoding name in the "a=rtpmap" line of SDP MUST be H264-SVC |
| 4155 |
(the media subtype). |
| 4156 |
|
| 4157 |
o The clock rate in the "a=rtpmap" line MUST be 90000. |
| 4158 |
|
| 4159 |
o The OPTIONAL parameters profile-level-id, max-recv-level, max- |
| 4160 |
recv-base-level, max-mbps, max-fs, max-cpb, max-dpb, max-br, |
| 4161 |
redundant-pic-cap, in-band-parameter-sets, packetization-mode, |
| 4162 |
sprop-interleaving-depth, deint-buf-cap, sprop-deint-buf-req, |
| 4163 |
sprop-init-buf-time, sprop-max-don-diff, max-rcmd-nalu-size, mst- |
| 4164 |
mode, sprop-mst-csdon-always-present, sprop-mst-remux-buf-size, |
| 4165 |
sprop-remux-buf-req, remux-buf-cap, sprop-remux-init-buf-time, |
| 4166 |
sprop-mst-max-don-diff, and scalable-layer-id, when present, MUST |
| 4167 |
be included in the "a=fmtp" line of SDP. These parameters are |
| 4168 |
expressed as a media type string, in the form of a semicolon- |
| 4169 |
separated list of parameter=value pairs. |
| 4170 |
|
| 4171 |
o The OPTIONAL parameters sprop-parameter-sets, sprop-level- |
| 4172 |
parameter-sets, sprop-scalability-info, sprop-operation-point- |
| 4173 |
info, sprop-no-NAL-reordering-required, and sprop-avc-ready, when |
| 4174 |
present, MUST be included in the "a=fmtp" line of SDP or conveyed |
| 4175 |
using the "fmtp" source attribute as specified in <a href="./rfc5576#section-6.3">Section 6.3 of |
| 4176 |
[RFC5576]</a>. For a particular media format (i.e., RTP payload |
| 4177 |
type), a sprop-parameter-sets or sprop-level-parameter-sets MUST |
| 4178 |
NOT be both included in the "a=fmtp" line of SDP and conveyed |
| 4179 |
using the "fmtp" source attribute. When included in the "a=fmtp" |
| 4180 |
line of SDP, these parameters are expressed as a media type |
| 4181 |
string, in the form of a semicolon-separated list of |
| 4182 |
parameter=value pairs. When conveyed using the "fmtp" source |
| 4183 |
attribute, these parameters are only associated with the given |
| 4184 |
source and payload type as parts of the "fmtp" source attribute. |
| 4185 |
|
| 4186 |
Informative note: Conveyance of sprop-parameter-sets and |
| 4187 |
sprop-level-parameter-sets using the "fmtp" source attribute |
| 4188 |
allows for out-of-band transport of parameter sets in |
| 4189 |
topologies like Topo-Video-switch-MCU [<a href="./rfc5117" title=""RTP Topologies"">RFC5117</a>]. |
| 4190 |
|
| 4191 |
|
| 4192 |
|
| 4193 |
|
| 4194 |
|
| 4195 |
|
| 4196 |
<span class="grey">Wenger, et al. Standards Track [Page 75]</span></pre> |
| 4197 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-76" ></span> |
| 4198 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 4199 |
|
| 4200 |
|
| 4201 |
<span class="h4"><a class="selflink" id="section-7.2.2" href="#section-7.2.2">7.2.2</a>. Usage with the SDP Offer/Answer Model</span> |
| 4202 |
|
| 4203 |
When an SVC stream (with media subtype H264-SVC) is offered over RTP |
| 4204 |
using SDP in an Offer/Answer model [<a href="./rfc3264" title=""An Offer/Answer Model with Session Description Protocol (SDP)"">RFC3264</a>] for negotiation for |
| 4205 |
unicast usage, the following limitations and rules apply: |
| 4206 |
|
| 4207 |
o The parameters identifying a media format configuration for SVC |
| 4208 |
are profile-level-id, packetization-mode, and mst-mode. These |
| 4209 |
media configuration parameters (except for the level part of |
| 4210 |
profile-level-id) MUST be used symmetrically when the answerer |
| 4211 |
does not include scalable-layer-id in the answer; i.e., the |
| 4212 |
answerer MUST either maintain all configuration parameters or |
| 4213 |
remove the media format (payload type) completely, if one or more |
| 4214 |
of the parameter values are not supported. Note that the level |
| 4215 |
part of profile-level-id includes level_idc, and, for indication |
| 4216 |
of level 1b when profile_idc is equal to 66, 77, or 88, bit 4 |
| 4217 |
(constraint_set3_flag) of profile-iop. The level part of profile- |
| 4218 |
level-id is changeable. |
| 4219 |
|
| 4220 |
Informative note: The requirement for symmetric use does not |
| 4221 |
apply for the level part of profile-level-id, and does not |
| 4222 |
apply for the other stream properties and capability |
| 4223 |
parameters. |
| 4224 |
|
| 4225 |
Informative note: In [<a href="#ref-H.264" title=""Advanced video coding for generic audiovisual services"">H.264</a>], all the levels except for Level |
| 4226 |
1b are equal to the value of level_idc divided by 10. Level 1b |
| 4227 |
is a level higher than Level 1.0 but lower than Level 1.1, and |
| 4228 |
is signaled in an ad hoc manner. For the Baseline, Main, and |
| 4229 |
Extended profiles (with profile_idc equal to 66, 77, and 88, |
| 4230 |
respectively), Level 1b is indicated by level_idc equal to 11 |
| 4231 |
(i.e., the same as level 1.1) and constraint_set3_flag equal to |
| 4232 |
1. For other profiles, Level 1b is indicated by level_idc |
| 4233 |
equal to 9 (but note that Level 1b for these profiles is still |
| 4234 |
higher than Level 1, which has level_idc equal to 10, and lower |
| 4235 |
than Level 1.1). In SDP Offer/Answer, an answer may indicate a |
| 4236 |
level equal to or lower than the level indicated in the offer. |
| 4237 |
Due to the ad hoc indication of Level 1b, offerers and |
| 4238 |
answerers must check the value of bit 4 (constraint_set3_flag) |
| 4239 |
of the middle octet of the parameter profile-level-id, when |
| 4240 |
profile_idc is equal to 66, 77, or 88 and level_idc is equal to |
| 4241 |
11. |
| 4242 |
|
| 4243 |
To simplify handling and matching of these configurations, the |
| 4244 |
same RTP payload type number used in the offer should also be used |
| 4245 |
in the answer, as specified in [<a href="./rfc3264" title=""An Offer/Answer Model with Session Description Protocol (SDP)"">RFC3264</a>]. The same RTP payload |
| 4246 |
type number used in the offer MUST also be used in the answer when |
| 4247 |
the answer includes scalable-layer-id. When the answer does not |
| 4248 |
include scalable-layer-id, the answer MUST NOT contain a payload |
| 4249 |
|
| 4250 |
|
| 4251 |
|
| 4252 |
<span class="grey">Wenger, et al. Standards Track [Page 76]</span></pre> |
| 4253 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-77" ></span> |
| 4254 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 4255 |
|
| 4256 |
|
| 4257 |
type number used in the offer unless the configuration is exactly |
| 4258 |
the same as in the offer or the configuration in the answer only |
| 4259 |
differs from that in the offer with a level lower than the default |
| 4260 |
level offered. |
| 4261 |
|
| 4262 |
Informative note: When an offerer receives an answer that does |
| 4263 |
not include scalable-layer-id it has to compare payload types |
| 4264 |
not declared in the offer based on the media type (i.e., |
| 4265 |
video/H264-SVC) and the above media configuration parameters |
| 4266 |
with any payload types it has already declared. This will |
| 4267 |
enable it to determine whether the configuration in question is |
| 4268 |
new or if it is equivalent to configuration already offered, |
| 4269 |
since a different payload type number may be used in the |
| 4270 |
answer. |
| 4271 |
|
| 4272 |
Since an SVC stream may contain multiple operation points, a |
| 4273 |
facility is provided so that an answerer can select a different |
| 4274 |
operation point than the entire SVC stream. Specifically, |
| 4275 |
different operation points MAY be described using the sprop- |
| 4276 |
scalability-info or sprop-operation-point-info parameters. The |
| 4277 |
first one carries the entire scalability information SEI message |
| 4278 |
defined in Annex G of [<a href="#ref-H.264" title=""Advanced video coding for generic audiovisual services"">H.264</a>], whereas the second one may be |
| 4279 |
derived, e.g., as a subset of this SEI message that only contains |
| 4280 |
key information about an operation point. Operation points, in |
| 4281 |
both cases, are associated with a layer identifier. |
| 4282 |
|
| 4283 |
If such information (sprop-operation-point-info or sprop- |
| 4284 |
scalability-info) is provided in an offer, an answerer MAY select |
| 4285 |
from the various operation points offered in the sprop- |
| 4286 |
scalability-information or sprop-operation-point-info parameters |
| 4287 |
by including scalable-layer-id in the answer. By this, the |
| 4288 |
answerer indicates its selection of a particular operation point |
| 4289 |
in the received and/or in the sent stream. When such operation |
| 4290 |
point selection takes place, i.e., the answerer includes scalable- |
| 4291 |
layer-id in the answer, the media configuration parameters MUST |
| 4292 |
NOT be present in the answer. Rather, the media configuration |
| 4293 |
that the answerer will use for receiving and/or sending is the one |
| 4294 |
used for the selected operation point as indicated in the offer. |
| 4295 |
|
| 4296 |
Informative note: The ability to perform operation point |
| 4297 |
selection enables a receiver to utilize the scalable nature of |
| 4298 |
an SVC stream. |
| 4299 |
|
| 4300 |
o The parameter max-recv-level, when present, declares the highest |
| 4301 |
level supported for receiving. In case max-recv-level is not |
| 4302 |
present, the highest level supported for receiving is equal to the |
| 4303 |
|
| 4304 |
|
| 4305 |
|
| 4306 |
|
| 4307 |
|
| 4308 |
<span class="grey">Wenger, et al. Standards Track [Page 77]</span></pre> |
| 4309 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-78" ></span> |
| 4310 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 4311 |
|
| 4312 |
|
| 4313 |
default level indicated by the level part of profile-level-id. |
| 4314 |
max-recv-level, when present, MUST be higher than the default |
| 4315 |
level. |
| 4316 |
|
| 4317 |
o The parameter max-recv-base-level, when present, declares the |
| 4318 |
highest level of the base layer supported for receiving. When |
| 4319 |
max-recv-base-level is not present, the highest level supported |
| 4320 |
for the base layer is not constrained separately from the SVC |
| 4321 |
stream containing the base layer. The endpoint at the other side |
| 4322 |
MUST NOT send a scalable stream for which the base layer is of a |
| 4323 |
level higher than max-recv-base-level. Parameters declaring |
| 4324 |
receiver capabilities above the default level (max-mbps, max- |
| 4325 |
smbps, max-fs, max-cpb, max-dpb, max-br, and max-recv-level) do |
| 4326 |
not apply to the base layer when max-recv-base-level is present. |
| 4327 |
|
| 4328 |
o The parameters sprop-deint-buf-req, sprop-interleaving-depth, |
| 4329 |
sprop-max-don-diff, sprop-init-buf-time, sprop-mst-csdon-always- |
| 4330 |
present, sprop-remux-buf-req, sprop-mst-remux-buf-size, sprop- |
| 4331 |
remux-init-buf-time, sprop-mst-max-don-diff, sprop-scalability- |
| 4332 |
information, sprop-operation-point-info, sprop-no-NAL-reordering- |
| 4333 |
required, and sprop-avc-ready describe the properties of the NAL |
| 4334 |
unit stream that the offerer or answerer is sending for the media |
| 4335 |
format configuration. This differs from the normal usage of the |
| 4336 |
Offer/Answer parameters: normally such parameters declare the |
| 4337 |
properties of the stream that the offerer or the answerer is able |
| 4338 |
to receive. When dealing with SVC, the offerer assumes that the |
| 4339 |
answerer will be able to receive media encoded using the |
| 4340 |
configuration being offered. |
| 4341 |
|
| 4342 |
Informative note: The above parameters apply for any stream |
| 4343 |
sent by the declaring entity with the same configuration; i.e., |
| 4344 |
they are dependent on their source. Rather than being bound to |
| 4345 |
the payload type, the values may have to be applied to another |
| 4346 |
payload type when being sent, as they apply for the |
| 4347 |
configuration. |
| 4348 |
|
| 4349 |
o The capability parameters max-mbps, max-fs, max-cpb, max-dpb, max- |
| 4350 |
br, redundant-pic-cap, and max-rcmd-nalu-size MAY be used to |
| 4351 |
declare further capabilities of the offerer or answerer for |
| 4352 |
receiving. These parameters MUST NOT be present when the |
| 4353 |
direction attribute is sendonly, and the parameters describe the |
| 4354 |
limitations of what the offerer or answerer accepts for receiving |
| 4355 |
streams. |
| 4356 |
|
| 4357 |
o When mst-mode is not present and packetization-mode is equal to 2, |
| 4358 |
the following applies. |
| 4359 |
|
| 4360 |
|
| 4361 |
|
| 4362 |
|
| 4363 |
|
| 4364 |
<span class="grey">Wenger, et al. Standards Track [Page 78]</span></pre> |
| 4365 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-79" ></span> |
| 4366 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 4367 |
|
| 4368 |
|
| 4369 |
o An offerer has to include the size of the de-interleaving |
| 4370 |
buffer, sprop-deint-buf-req, in the offer. To enable the |
| 4371 |
offerer and answerer to inform each other about their |
| 4372 |
capabilities for de-interleaving buffering, both parties are |
| 4373 |
RECOMMENDED to include deint-buf-cap. It is also RECOMMENDED |
| 4374 |
to consider offering multiple payload types with different |
| 4375 |
buffering requirements when the capabilities of the receiver |
| 4376 |
are unknown. |
| 4377 |
|
| 4378 |
o When mst-mode is present and equal to "NI-C", "NI-TC", or "I-C", |
| 4379 |
the following applies. |
| 4380 |
|
| 4381 |
o An offerer has to include sprop-remux-buf-req in the offer. To |
| 4382 |
enable the offerer and answerer to inform each other about |
| 4383 |
their capabilities for re-multiplexing buffering, both parties |
| 4384 |
are RECOMMENDED to include remux-buf-cap. It is also |
| 4385 |
RECOMMENDED to consider offering multiple payload types with |
| 4386 |
different buffering requirements when the capabilities of the |
| 4387 |
receiver are unknown. |
| 4388 |
|
| 4389 |
o The sprop-parameter-sets or sprop-level-parameter-sets parameter, |
| 4390 |
when present (included in the "a=fmtp" line of SDP or conveyed |
| 4391 |
using the "fmtp" source attribute as specified in <a href="./rfc5576#section-6.3">Section 6.3 of |
| 4392 |
[RFC5576]</a>), is used for out-of-band transport of parameter sets. |
| 4393 |
However, when out-of-band transport of parameter sets is used, |
| 4394 |
parameter sets MAY still be additionally transported in-band. |
| 4395 |
|
| 4396 |
The answerer MAY use either out-of-band or in-band transport of |
| 4397 |
parameter sets for the stream it is sending, regardless of whether |
| 4398 |
out-of-band parameter sets transport has been used in the offerer- |
| 4399 |
to-answerer direction. Parameter sets included in an answer are |
| 4400 |
independent of those parameter sets included in the offer, as they |
| 4401 |
are used for decoding two different video streams, one from the |
| 4402 |
answerer to the offerer, and the other in the opposite direction. |
| 4403 |
|
| 4404 |
The following rules apply to transport of parameter sets in the |
| 4405 |
offerer-to-answerer direction. |
| 4406 |
|
| 4407 |
o An offer MAY include either or both of sprop-parameter- sets |
| 4408 |
and sprop-level-parameter-sets. If neither sprop-parameter- |
| 4409 |
sets nor sprop-level-parameter-sets is present in the offer, |
| 4410 |
then only in-band transport of parameter sets is used. |
| 4411 |
|
| 4412 |
o If the answer includes in-band-parameter-sets equal to 1, then |
| 4413 |
the offerer MUST transmit parameter sets in-band. Otherwise, |
| 4414 |
the following applies. |
| 4415 |
|
| 4416 |
|
| 4417 |
|
| 4418 |
|
| 4419 |
|
| 4420 |
<span class="grey">Wenger, et al. Standards Track [Page 79]</span></pre> |
| 4421 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-80" ></span> |
| 4422 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 4423 |
|
| 4424 |
|
| 4425 |
o If the level to use in the offerer-to-answerer direction is |
| 4426 |
equal to the default level in the offer, the following |
| 4427 |
applies. |
| 4428 |
|
| 4429 |
The answerer MUST be prepared to use the parameter sets |
| 4430 |
included in sprop-parameter-sets, when present, for |
| 4431 |
decoding the incoming NAL unit stream, and ignore sprop- |
| 4432 |
level-parameter-sets, when present. |
| 4433 |
|
| 4434 |
When sprop-parameter-sets is not present in the offer, |
| 4435 |
in-band transport of parameter sets MUST be used. |
| 4436 |
|
| 4437 |
o Otherwise (the level to use in the offerer-to-answerer |
| 4438 |
direction is not equal to the default level in the offer), |
| 4439 |
the following applies. |
| 4440 |
|
| 4441 |
The answerer MUST be prepared to use the parameter sets |
| 4442 |
that are included in sprop-level-parameter-sets for the |
| 4443 |
accepted level (i.e., the default level in the answer, |
| 4444 |
which is also the level to use in the offerer-to-answerer |
| 4445 |
direction), when present, for decoding the incoming NAL |
| 4446 |
unit stream, and ignore all other parameter sets included |
| 4447 |
in sprop-level-parameter-sets and sprop-parameter-sets, |
| 4448 |
when present. |
| 4449 |
|
| 4450 |
When no parameter sets for the accepted level are present |
| 4451 |
in the sprop-level-parameter-sets, in-band transport of |
| 4452 |
parameter sets MUST be used. |
| 4453 |
|
| 4454 |
The following rules apply to transport of parameter sets in the |
| 4455 |
answerer-to-offerer direction. |
| 4456 |
|
| 4457 |
o An answer MAY include either sprop-parameter-sets or sprop- |
| 4458 |
level-parameter-sets, but MUST NOT include both of the two. If |
| 4459 |
neither sprop-parameter-sets nor sprop-level-parameter-sets is |
| 4460 |
present in the answer, then only in-band transport of parameter |
| 4461 |
sets is used. |
| 4462 |
|
| 4463 |
o If the offer includes in-band-parameter-sets equal to 1, then |
| 4464 |
the answerer MUST NOT include sprop-parameter-sets or sprop- |
| 4465 |
level-parameter-sets in the answer and MUST transmit parameter |
| 4466 |
sets in-band. Otherwise, the following applies. |
| 4467 |
|
| 4468 |
o If the level to use in the answerer-to-offerer direction is |
| 4469 |
equal to the default level in the answer, the following |
| 4470 |
applies. |
| 4471 |
|
| 4472 |
|
| 4473 |
|
| 4474 |
|
| 4475 |
|
| 4476 |
<span class="grey">Wenger, et al. Standards Track [Page 80]</span></pre> |
| 4477 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-81" ></span> |
| 4478 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 4479 |
|
| 4480 |
|
| 4481 |
The offerer MUST be prepared to use the parameter sets |
| 4482 |
included in sprop-parameter-sets, when present, for |
| 4483 |
decoding the incoming NAL unit stream, and ignore sprop- |
| 4484 |
level-parameter-sets, when present. |
| 4485 |
|
| 4486 |
When sprop-parameter-sets is not present in the answer, |
| 4487 |
the answerer MUST transmit parameter sets in-band. |
| 4488 |
|
| 4489 |
o Otherwise (the level to use in the answerer-to-offerer |
| 4490 |
direction is not equal to the default level in the answer), |
| 4491 |
the following applies. |
| 4492 |
|
| 4493 |
The offerer MUST be prepared to use the parameter sets |
| 4494 |
that are included in sprop-level-parameter-sets for the |
| 4495 |
level to use in the answerer-to-offerer direction, when |
| 4496 |
present in the answer, for decoding the incoming NAL unit |
| 4497 |
stream, and ignore all other parameter sets included in |
| 4498 |
sprop-level-parameter-sets and sprop-parameter-sets, when |
| 4499 |
present in the answer. |
| 4500 |
|
| 4501 |
When no parameter sets for the level to use in the |
| 4502 |
answerer-to-offerer direction are present in sprop-level- |
| 4503 |
parameter-sets in the answer, the answerer MUST transmit |
| 4504 |
parameter sets in-band. |
| 4505 |
|
| 4506 |
When sprop-parameter-sets or sprop-level-parameter-sets is |
| 4507 |
conveyed using the "fmtp" source attribute as specified in <a href="./rfc5576#section-6.3">Section</a> |
| 4508 |
<a href="./rfc5576#section-6.3">6.3 of [RFC5576]</a>, the receiver of the parameters MUST store the |
| 4509 |
parameter sets included in the sprop-parameter-sets or sprop- |
| 4510 |
level-parameter-sets for the accepted level and associate them to |
| 4511 |
the source given as a part of the "fmtp" source attribute. |
| 4512 |
Parameter sets associated with one source MUST only be used to |
| 4513 |
decode NAL units conveyed in RTP packets from the same source. |
| 4514 |
When this mechanism is in use, SSRC collision detection and |
| 4515 |
resolution MUST be performed as specified in [<a href="./rfc5576" title=""Source-Specific Media Attributes in the Session Description Protocol (SDP)"">RFC5576</a>]. |
| 4516 |
|
| 4517 |
Informative note: Conveyance of sprop-parameter-sets and sprop- |
| 4518 |
level-parameter-sets using the "fmtp" source attribute may be |
| 4519 |
used in topologies like Topo-Video-switch-MCU [<a href="./rfc5117" title=""RTP Topologies"">RFC5117</a>] to |
| 4520 |
enable out-of-band transport of parameter sets. |
| 4521 |
|
| 4522 |
For streams being delivered over multicast, the following rules |
| 4523 |
apply: |
| 4524 |
|
| 4525 |
o The media format configuration is identified by profile-level- id, |
| 4526 |
including the level part, packetization-mode, and mst-mode. These |
| 4527 |
media format configuration parameters (including the level part of |
| 4528 |
profile-level-id) MUST be used symmetrically; i.e., the answerer |
| 4529 |
|
| 4530 |
|
| 4531 |
|
| 4532 |
<span class="grey">Wenger, et al. Standards Track [Page 81]</span></pre> |
| 4533 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-82" ></span> |
| 4534 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 4535 |
|
| 4536 |
|
| 4537 |
MUST either maintain all configuration parameters or remove the |
| 4538 |
media format (payload type) completely. Note that this implies |
| 4539 |
that the level part of profile-level-id for Offer/Answer in |
| 4540 |
multicast is not changeable. |
| 4541 |
|
| 4542 |
To simplify handling and matching of these configurations, the |
| 4543 |
same RTP payload type number used in the offer should also be used |
| 4544 |
in the answer, as specified in [<a href="./rfc3264" title=""An Offer/Answer Model with Session Description Protocol (SDP)"">RFC3264</a>]. An answer MUST NOT |
| 4545 |
contain a payload type number used in the offer unless the |
| 4546 |
configuration is the same as in the offer. |
| 4547 |
|
| 4548 |
o Parameter sets received MUST be associated with the originating |
| 4549 |
source, and MUST be only used in decoding the incoming NAL unit |
| 4550 |
stream from the same source. |
| 4551 |
|
| 4552 |
o The rules for other parameters are the same as above for unicast |
| 4553 |
as long as the above rules are obeyed. |
| 4554 |
|
| 4555 |
Table 14 lists the interpretation of all the parameters that MUST be |
| 4556 |
used for the various combinations of offer, answer, and direction |
| 4557 |
attributes. Note that the two columns wherein the scalable-layer-id |
| 4558 |
parameter is used only apply to answers, whereas the other columns |
| 4559 |
apply to both offers and answers. |
| 4560 |
|
| 4561 |
Table 14. Interpretation of parameters for various combinations of |
| 4562 |
offers, answers, direction attributes, with and without scalable- |
| 4563 |
layer-id. Columns that do not indicate offer or answer apply to |
| 4564 |
both. |
| 4565 |
|
| 4566 |
|
| 4567 |
|
| 4568 |
|
| 4569 |
|
| 4570 |
|
| 4571 |
|
| 4572 |
|
| 4573 |
|
| 4574 |
|
| 4575 |
|
| 4576 |
|
| 4577 |
|
| 4578 |
|
| 4579 |
|
| 4580 |
|
| 4581 |
|
| 4582 |
|
| 4583 |
|
| 4584 |
|
| 4585 |
|
| 4586 |
|
| 4587 |
|
| 4588 |
<span class="grey">Wenger, et al. Standards Track [Page 82]</span></pre> |
| 4589 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-83" ></span> |
| 4590 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 4591 |
|
| 4592 |
|
| 4593 |
sendonly --+ |
| 4594 |
answer: recvonly,scalable-layer-id --+ | |
| 4595 |
recvonly w/o scalable-layer-id --+ | | |
| 4596 |
answer: sendrecv, scalable-layer-id --+ | | | |
| 4597 |
sendrecv w/o scalable-layer-id --+ | | | | |
| 4598 |
| | | | | |
| 4599 |
profile-level-id C X C X P |
| 4600 |
max-recv-level R R R R - |
| 4601 |
max-recv-base-level R R R R - |
| 4602 |
packetization-mode C X C X P |
| 4603 |
mst-mode C X C X P |
| 4604 |
sprop-avc-ready P P - - P |
| 4605 |
sprop-deint-buf-req P P - - P |
| 4606 |
sprop-init-buf-time P P - - P |
| 4607 |
sprop-interleaving-depth P P - - P |
| 4608 |
sprop-max-don-diff P P - - P |
| 4609 |
sprop-mst-csdon-always-present P P - - P |
| 4610 |
sprop-mst-max-don-diff P P - - P |
| 4611 |
sprop-mst-remux-buf-size P P - - P |
| 4612 |
sprop-no-NAL-reordering-required P P - - P |
| 4613 |
sprop-operation-point-info P P - - P |
| 4614 |
sprop-remux-buf-req P P - - P |
| 4615 |
sprop-remux-init-buf-time P P - - P |
| 4616 |
sprop-scalability-info P P - - P |
| 4617 |
deint-buf-cap R R R R - |
| 4618 |
max-br R R R R - |
| 4619 |
max-cpb R R R R - |
| 4620 |
max-dpb R R R R - |
| 4621 |
max-fs R R R R - |
| 4622 |
max-mbps R R R R - |
| 4623 |
max-rcmd-nalu-size R R R R - |
| 4624 |
redundant-pic-cap R R R R - |
| 4625 |
remux-buf-cap R R R R - |
| 4626 |
in-band-parameter-sets R R R R - |
| 4627 |
sprop-parameter-sets S S - - S |
| 4628 |
sprop-level-parameter-sets S S - - S |
| 4629 |
scalable-layer-id X O X O - |
| 4630 |
|
| 4631 |
Legend: |
| 4632 |
|
| 4633 |
C: configuration for sending and receiving streams |
| 4634 |
P: properties of the stream to be sent |
| 4635 |
R: receiver capabilities |
| 4636 |
S: out-of-band parameter sets |
| 4637 |
O: operation point selection |
| 4638 |
X: MUST NOT be present |
| 4639 |
-: not usable, when present SHOULD be ignored |
| 4640 |
|
| 4641 |
|
| 4642 |
|
| 4643 |
|
| 4644 |
<span class="grey">Wenger, et al. Standards Track [Page 83]</span></pre> |
| 4645 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-84" ></span> |
| 4646 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 4647 |
|
| 4648 |
|
| 4649 |
Parameters used for declaring receiver capabilities are in general |
| 4650 |
downgradable; i.e., they express the upper limit for a sender's |
| 4651 |
possible behavior. Thus, a sender MAY select to set its encoder |
| 4652 |
using only lower/lesser or equal values of these parameters. |
| 4653 |
|
| 4654 |
Parameters declaring a configuration point are not changeable, with |
| 4655 |
the exception of the level part of the profile-level-id parameter for |
| 4656 |
unicast usage. This expresses values a receiver expects to be used |
| 4657 |
and must be used verbatim on the sender side. If level downgrading |
| 4658 |
(for profile-level-id) is used, an answerer MUST NOT include the |
| 4659 |
scalable-layer-id parameter. |
| 4660 |
|
| 4661 |
When a sender's capabilities are declared, and non-downgradable |
| 4662 |
parameters are used in this declaration, then these parameters |
| 4663 |
express a configuration that is acceptable for the sender to receive |
| 4664 |
streams. In order to achieve high interoperability levels, it is |
| 4665 |
often advisable to offer multiple alternative configurations, e.g., |
| 4666 |
for the packetization mode. It is impossible to offer multiple |
| 4667 |
configurations in a single payload type. Thus, when multiple |
| 4668 |
configuration offers are made, each offer requires its own RTP |
| 4669 |
payload type associated with the offer. |
| 4670 |
|
| 4671 |
A receiver SHOULD understand all media type parameters, even if it |
| 4672 |
only supports a subset of the payload format's functionality. This |
| 4673 |
ensures that a receiver is capable of understanding when an offer to |
| 4674 |
receive media can be downgraded to what is supported by the receiver |
| 4675 |
of the offer. |
| 4676 |
|
| 4677 |
An answerer MAY extend the offer with additional media format |
| 4678 |
configurations. However, to enable their usage, in most cases a |
| 4679 |
second offer is required from the offerer to provide the stream |
| 4680 |
property parameters that the media sender will use. This also has |
| 4681 |
the effect that the offerer has to be able to receive this media |
| 4682 |
format configuration, not only to send it. |
| 4683 |
|
| 4684 |
If an offerer wishes to have non-symmetric capabilities between |
| 4685 |
sending and receiving, the offerer can allow asymmetric levels via |
| 4686 |
level-asymmetry-allowed equal to 1. Alternatively, the offerer can |
| 4687 |
offer different RTP sessions, i.e., different media lines declared as |
| 4688 |
"recvonly" and "sendonly", respectively. This may have further |
| 4689 |
implications on the system, and may require additional external |
| 4690 |
semantics to associate the two media lines. |
| 4691 |
|
| 4692 |
<span class="h4"><a class="selflink" id="section-7.2.3" href="#section-7.2.3">7.2.3</a>. Dependency Signaling in Multi-Session Transmission</span> |
| 4693 |
|
| 4694 |
If MST is used, the rules on signaling media decoding dependency in |
| 4695 |
SDP as defined in [<a href="./rfc5583" title=""Signaling Media Decoding Dependency in the Session Description Protocol (SDP)"">RFC5583</a>] apply. The rules on "hierarchical or |
| 4696 |
layered encoding" with multicast in <a href="./rfc4566#section-5.7">Section 5.7 of [RFC4566]</a> do not |
| 4697 |
|
| 4698 |
|
| 4699 |
|
| 4700 |
<span class="grey">Wenger, et al. Standards Track [Page 84]</span></pre> |
| 4701 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-85" ></span> |
| 4702 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 4703 |
|
| 4704 |
|
| 4705 |
apply, i.e., the notation for Connection Data "c=" SHALL NOT be used |
| 4706 |
with more than one address. Additionally, the order of dependencies |
| 4707 |
of the RTP sessions indicated by the "a=depend" attribute as defined |
| 4708 |
in [<a href="./rfc5583" title=""Signaling Media Decoding Dependency in the Session Description Protocol (SDP)"">RFC5583</a>] MUST represent the decoding order of the VC) NAL units |
| 4709 |
in an access unit, i.e., the order of session dependency is given |
| 4710 |
from the base or the lowest enhancement RTP session (the most |
| 4711 |
important) to the highest enhancement RTP session (the least |
| 4712 |
important). |
| 4713 |
|
| 4714 |
<span class="h4"><a class="selflink" id="section-7.2.4" href="#section-7.2.4">7.2.4</a>. Usage in Declarative Session Descriptions</span> |
| 4715 |
|
| 4716 |
When SVC over RTP is offered with SDP in a declarative style, as in |
| 4717 |
Real Time Streaming Protocol (RTSP) [<a href="./rfc2326" title=""Real Time Streaming Protocol (RTSP)"">RFC2326</a>] or Session Announcement |
| 4718 |
Protocol (SAP) [<a href="./rfc2974" title=""Session Announcement Protocol"">RFC2974</a>], the following considerations are necessary. |
| 4719 |
|
| 4720 |
o All parameters capable of indicating both stream properties and |
| 4721 |
receiver capabilities are used to indicate only stream properties. |
| 4722 |
For example, in this case, the parameter profile-level-id declares |
| 4723 |
the values used by the stream, not the capabilities for receiving |
| 4724 |
streams. This results in that the following interpretation of the |
| 4725 |
parameters MUST be used: |
| 4726 |
|
| 4727 |
Declaring actual configuration or stream properties: |
| 4728 |
|
| 4729 |
- profile-level-id |
| 4730 |
- packetization-mode |
| 4731 |
- mst-mode |
| 4732 |
- sprop-deint-buf-req |
| 4733 |
- sprop-interleaving-depth |
| 4734 |
- sprop-max-don-diff |
| 4735 |
- sprop-init-buf-time |
| 4736 |
- sprop-mst-csdon-always-present |
| 4737 |
- sprop-mst-remux-buf-size |
| 4738 |
- sprop-remux-buf-req |
| 4739 |
- sprop-remux-init-buf-time |
| 4740 |
- sprop-mst-max-don-diff |
| 4741 |
- sprop-scalability-info |
| 4742 |
- sprop-operation-point-info |
| 4743 |
- sprop-no-NAL-reordering-required |
| 4744 |
- sprop-avc-ready |
| 4745 |
|
| 4746 |
Out-of-band transporting of parameter sets: |
| 4747 |
|
| 4748 |
- sprop-parameter-sets |
| 4749 |
- sprop-level-parameter-sets |
| 4750 |
|
| 4751 |
|
| 4752 |
|
| 4753 |
|
| 4754 |
|
| 4755 |
|
| 4756 |
<span class="grey">Wenger, et al. Standards Track [Page 85]</span></pre> |
| 4757 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-86" ></span> |
| 4758 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 4759 |
|
| 4760 |
|
| 4761 |
Not usable (when present, they SHOULD be ignored): |
| 4762 |
|
| 4763 |
- max-mbps |
| 4764 |
- max-fs |
| 4765 |
- max-cpb |
| 4766 |
- max-dpb |
| 4767 |
- max-br |
| 4768 |
- max-recv-level |
| 4769 |
- max-recv-base-level |
| 4770 |
- redundant-pic-cap |
| 4771 |
- max-rcmd-nalu-size |
| 4772 |
- deint-buf-cap |
| 4773 |
- remux-buf-cap |
| 4774 |
- scalable-layer-id |
| 4775 |
|
| 4776 |
o A receiver of the SDP is required to support all parameters and |
| 4777 |
values of the parameters provided; otherwise, the receiver MUST |
| 4778 |
reject (RTSP) or not participate in (SAP) the session. It falls |
| 4779 |
on the creator of the session to use values that are expected to |
| 4780 |
be supported by the receiving application. |
| 4781 |
|
| 4782 |
<span class="h3"><a class="selflink" id="section-7.3" href="#section-7.3">7.3</a>. Examples</span> |
| 4783 |
|
| 4784 |
In the following examples, "{data}" is used to indicate a data string |
| 4785 |
encoded as base64. |
| 4786 |
|
| 4787 |
<span class="h4"><a class="selflink" id="section-7.3.1" href="#section-7.3.1">7.3.1</a>. Example for Offering a Single SVC Session</span> |
| 4788 |
|
| 4789 |
Example 1: The offerer offers one video media description including |
| 4790 |
two RTP payload types. The first payload type offers H264, and the |
| 4791 |
second offers H264-SVC. Both payload types have different fmtp |
| 4792 |
parameters as profile-level-id, packetization-mode, and sprop- |
| 4793 |
parameter-sets. |
| 4794 |
|
| 4795 |
Offerer -> Answerer SDP message: |
| 4796 |
|
| 4797 |
m=video 20000 RTP/AVP 97 96 |
| 4798 |
a=rtpmap:96 H264/90000 |
| 4799 |
a=fmtp:96 profile-level-id=4de00a; packetization-mode=0; |
| 4800 |
sprop-parameter-sets={sps0},{pps0}; |
| 4801 |
a=rtpmap:97 H264-SVC/90000 |
| 4802 |
a=fmtp:97 profile-level-id=53000c; packetization-mode=1; |
| 4803 |
sprop-parameter-sets={sps0},{pps0},{sps1},{pps1}; |
| 4804 |
|
| 4805 |
If the answerer does not support media subtype H264-SVC, it can issue |
| 4806 |
an answer accepting only the base layer offer (payload type 96). In |
| 4807 |
the following example, the receiver supports H264-SVC, so it lists |
| 4808 |
payload type 97 first as the preferred option. |
| 4809 |
|
| 4810 |
|
| 4811 |
|
| 4812 |
<span class="grey">Wenger, et al. Standards Track [Page 86]</span></pre> |
| 4813 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-87" ></span> |
| 4814 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 4815 |
|
| 4816 |
|
| 4817 |
Answerer -> Offerer SDP message: |
| 4818 |
|
| 4819 |
m=video 40000 RTP/AVP 97 96 |
| 4820 |
a=rtpmap:96 H264/90000 |
| 4821 |
a=fmtp:96 profile-level-id=4de00a; packetization-mode=0; |
| 4822 |
sprop-parameter-sets={sps2},{pps2}; |
| 4823 |
a=rtpmap:97 H264-SVC/90000 |
| 4824 |
a=fmtp:97 profile-level-id=53000c; packetization-mode=1; |
| 4825 |
sprop-parameter-sets={sps2},{pps2},{sps3},{pps3}; |
| 4826 |
|
| 4827 |
<span class="h4"><a class="selflink" id="section-7.3.2" href="#section-7.3.2">7.3.2</a>. Example for Offering a Single SVC Session Using</span> |
| 4828 |
<span class="h4"> scalable-layer-id</span> |
| 4829 |
|
| 4830 |
Example 2: Offerer offers the same media configurations as shown in |
| 4831 |
the example above for receiving and sending the stream, but using a |
| 4832 |
single RTP payload type and including sprop-operation-point-info. |
| 4833 |
|
| 4834 |
Offerer -> Answerer SDP message: |
| 4835 |
|
| 4836 |
m=video 20000 RTP/AVP 97 |
| 4837 |
a=rtpmap:97 H264-SVC/90000 |
| 4838 |
a=fmtp:97 profile-level-id=53000c; packetization-mode=1; |
| 4839 |
sprop-parameter-sets={sps0},{sps1},{pps0},{pps1}; |
| 4840 |
sprop-operation-point-info=<1,0,0,0,4de00a,3200,176,144,128, |
| 4841 |
256>,<2,1,1,0,53000c,6400,352,288,256,512>; |
| 4842 |
|
| 4843 |
In this example, the receiver supports H264-SVC and chooses the lower |
| 4844 |
operation point offered in the RTP payload type for sending and |
| 4845 |
receiving the stream. |
| 4846 |
|
| 4847 |
Answerer -> Offerer SDP message: |
| 4848 |
|
| 4849 |
m=video 40000 RTP/AVP 97 |
| 4850 |
a=rtpmap:97 H264-SVC/90000 |
| 4851 |
a=fmtp:97 sprop-parameter-sets={sps2},{sps3},{pps2},{pps3}; |
| 4852 |
scalable-layer-id=1; |
| 4853 |
|
| 4854 |
In an equivalent example showing the use of sprop-scalability-info |
| 4855 |
instead using the sprop-operation-point-info, the sprop-operation- |
| 4856 |
point-info would be exchanged by the sprop-scalability-info followed |
| 4857 |
by the binary (base16) representation of the Scalability Information |
| 4858 |
SEI message. |
| 4859 |
|
| 4860 |
<span class="h4"><a class="selflink" id="section-7.3.3" href="#section-7.3.3">7.3.3</a>. Example for Offering Multiple Sessions in MST</span> |
| 4861 |
|
| 4862 |
Example 3: In this example, the offerer offers a multi-session |
| 4863 |
transmission with up to three sessions. The base session media |
| 4864 |
description includes payload types that are backward compatible with |
| 4865 |
|
| 4866 |
|
| 4867 |
|
| 4868 |
<span class="grey">Wenger, et al. Standards Track [Page 87]</span></pre> |
| 4869 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-88" ></span> |
| 4870 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 4871 |
|
| 4872 |
|
| 4873 |
[<a href="./rfc6184" title=""RTP Payload Format for H.264 Video"">RFC6184</a>], and three different payload types are offered. The other |
| 4874 |
two media are using payload types with media subtype H264-SVC. In |
| 4875 |
each media description, different values of profile-level-id, |
| 4876 |
packetization-mode, mst-mode, and sprop-parameter-sets are offered. |
| 4877 |
|
| 4878 |
Offerer -> Answerer SDP message: |
| 4879 |
|
| 4880 |
a=group:DDP L1 L2 L3 |
| 4881 |
m=video 20000 RTP/AVP 96 97 98 |
| 4882 |
a=rtpmap:96 H264/90000 |
| 4883 |
a=fmtp:96 profile-level-id=4de00a; packetization-mode=0; |
| 4884 |
mst-mode=NI-T; sprop-parameter-sets={sps0},{pps0}; |
| 4885 |
a=rtpmap:97 H264/90000 |
| 4886 |
a=fmtp:97 profile-level-id=4de00a; packetization-mode=1; |
| 4887 |
mst-mode=NI-TC; sprop-parameter-sets={sps0},{pps0}; |
| 4888 |
a=rtpmap:98 H264/90000 |
| 4889 |
a=fmtp:98 profile-level-id=4de00a; packetization-mode=2; |
| 4890 |
mst-mode=I-C; init-buf-time=156320; |
| 4891 |
sprop-parameter-sets={sps0},{pps0}; |
| 4892 |
a=mid:L1 |
| 4893 |
m=video 20002 RTP/AVP 99 100 |
| 4894 |
a=rtpmap:99 H264-SVC/90000 |
| 4895 |
a=fmtp:99 profile-level-id=53000c; packetization-mode=1; |
| 4896 |
mst-mode=NI-T; sprop-parameter-sets={sps1},{pps1}; |
| 4897 |
a=rtpmap:100 H264-SVC/90000 |
| 4898 |
a=fmtp:100 profile-level-id=53000c; packetization-mode=2; |
| 4899 |
mst-mode=I-C; sprop-parameter-sets={sps1},{pps1}; |
| 4900 |
a=mid:L2 |
| 4901 |
a=depend:99 lay L1:96,97; 100 lay L1:98 |
| 4902 |
m=video 20004 RTP/AVP 101 |
| 4903 |
a=rtpmap:101 H264-SVC/90000 |
| 4904 |
a=fmtp:101 profile-level-id=53001F; packetization-mode=1; |
| 4905 |
mst-mode=NI-T; sprop-parameter-sets={sps2},{pps2}; |
| 4906 |
a=mid:L3 |
| 4907 |
a=depend:101 lay L1:96,97 L2:99 |
| 4908 |
|
| 4909 |
It is assumed that in this example the answerer only supports the NI- |
| 4910 |
T mode for multi-session transmission. For this reason, it chooses |
| 4911 |
the corresponding payload type (96) for the base RTP session. For |
| 4912 |
the two enhancement RTP sessions, the answerer also chooses the |
| 4913 |
payload types that use the NI-T mode (99 and 101). |
| 4914 |
|
| 4915 |
|
| 4916 |
|
| 4917 |
|
| 4918 |
|
| 4919 |
|
| 4920 |
|
| 4921 |
|
| 4922 |
|
| 4923 |
|
| 4924 |
<span class="grey">Wenger, et al. Standards Track [Page 88]</span></pre> |
| 4925 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-89" ></span> |
| 4926 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 4927 |
|
| 4928 |
|
| 4929 |
Answerer -> Offerer SDP message: |
| 4930 |
|
| 4931 |
a=group:DDP L1 L2 L3 |
| 4932 |
m=video 40000 RTP/AVP 96 |
| 4933 |
a=rtpmap:96 H264/90000 |
| 4934 |
a=fmtp:96 profile-level-id=4de00a; packetization-mode=0; |
| 4935 |
mst-mode=NI-T; sprop-parameter-sets={sps3},{pps3}; |
| 4936 |
a=mid:L1 |
| 4937 |
m=video 40002 RTP/AVP 99 |
| 4938 |
a=rtpmap:99 H264-SVC/90000 |
| 4939 |
a=fmtp:99 profile-level-id=53000c; packetization-mode=1; |
| 4940 |
mst-mode=NI-T; sprop-parameter-sets={sps4},{pps4}; |
| 4941 |
a=mid:L2 |
| 4942 |
a=depend:99 lay L1:96 |
| 4943 |
m=video 40004 RTP/AVP 101 |
| 4944 |
a=rtpmap:101 H264-SVC/90000 |
| 4945 |
a=fmtp:101 profile-level-id=53001F; packetization-mode=1; |
| 4946 |
mst-mode=NI-T; sprop-parameter-sets={sps5},{pps5}; |
| 4947 |
a=mid:L3 |
| 4948 |
a=depend:101 lay L1:96 L2:99 |
| 4949 |
|
| 4950 |
<span class="h4"><a class="selflink" id="section-7.3.4" href="#section-7.3.4">7.3.4</a>. Example for Offering Multiple Sessions in MST Including</span> |
| 4951 |
<span class="h4"> Operation with Answerer Using scalable-layer-id</span> |
| 4952 |
|
| 4953 |
Example 4: In this example, the offerer offers a multi-session |
| 4954 |
transmission of three layers with up to two sessions. The base |
| 4955 |
session media description has a payload type that is backward |
| 4956 |
compatible with [<a href="./rfc6184" title=""RTP Payload Format for H.264 Video"">RFC6184</a>]. Note that no parameter sets are provided, |
| 4957 |
in which case in-band transport must be used. The other media |
| 4958 |
description contains two enhancement layers and uses the media |
| 4959 |
subtype H264-SVC. It includes two operation point definitions. |
| 4960 |
|
| 4961 |
Offerer -> Answerer SDP message: |
| 4962 |
|
| 4963 |
a=group:DDP L1 L2 |
| 4964 |
m=video 20000 RTP/AVP 96 |
| 4965 |
a=rtpmap:96 H264/90000 |
| 4966 |
a=fmtp:96 profile-level-id=4de00a; packetization-mode=0; |
| 4967 |
mst-mode=NI-T; |
| 4968 |
a=mid:L1 |
| 4969 |
m=video 20002 RTP/AVP 97 |
| 4970 |
a=rtpmap:97 H264-SVC/90000 |
| 4971 |
a=fmtp:97 profile-level-id=53001F; packetization-mode=1; |
| 4972 |
mst-mode=NI-TC; sprop-operation-point-info=<2,0,1,0,53000c, |
| 4973 |
3200,352,288,384,512>,<3,1,2,0,53001F,6400,704,576,768,1024>; |
| 4974 |
a=mid:L2 |
| 4975 |
a=depend:97 lay L1:96 |
| 4976 |
|
| 4977 |
|
| 4978 |
|
| 4979 |
|
| 4980 |
<span class="grey">Wenger, et al. Standards Track [Page 89]</span></pre> |
| 4981 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-90" ></span> |
| 4982 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 4983 |
|
| 4984 |
|
| 4985 |
It is assumed that the answerer wants to send and receive the base |
| 4986 |
layer (payload type 96), but it only wants to send and receive the |
| 4987 |
lower enhancement layer, i.e., the one with layer id equal to 2. For |
| 4988 |
this reason, the response will include the selection of the desired |
| 4989 |
layer by setting scalable-layer-id equal to 2. Note that the answer |
| 4990 |
only includes the scalable-layer-id information. The answer could |
| 4991 |
include sprop-parameter-sets in the response. |
| 4992 |
|
| 4993 |
Answerer -> Offerer SDP message: |
| 4994 |
|
| 4995 |
a=group:DDP L1 L2 |
| 4996 |
m=video 40000 RTP/AVP 96 |
| 4997 |
a=rtpmap:96 H264/90000 |
| 4998 |
a=fmtp:96 profile-level-id=4de00a; packetization-mode=0; |
| 4999 |
mst-mode=NI-T; |
| 5000 |
a=mid:L1 |
| 5001 |
m=video 40002 RTP/AVP 97 |
| 5002 |
a=rtpmap:97 H264-SVC/90000 |
| 5003 |
a=fmtp:97 scalable-layer-id=2; |
| 5004 |
a=mid:L2 |
| 5005 |
a=depend:97 lay L1:96 |
| 5006 |
|
| 5007 |
<span class="h4"><a class="selflink" id="section-7.3.5" href="#section-7.3.5">7.3.5</a>. Example for Negotiating an SVC Stream with a Constrained Base</span> |
| 5008 |
<span class="h4"> Layer in SST</span> |
| 5009 |
|
| 5010 |
Example 5: The offerer (Alice) offers one video description including |
| 5011 |
two RTP payload types with differing levels and packetization modes. |
| 5012 |
|
| 5013 |
Offerer -> Answerer SDP message: |
| 5014 |
|
| 5015 |
m=video 20000 RTP/AVP 97 96 |
| 5016 |
a=rtpmap:96 H264-SVC/90000 |
| 5017 |
a=fmtp:96 profile-level-id=53001e; packetization-mode=0; |
| 5018 |
a=rtpmap:97 H264-SVC/90000 |
| 5019 |
a=fmtp:97 profile-level-id=53001f; packetization-mode=1; |
| 5020 |
|
| 5021 |
The answerer (Bridge) chooses packetization mode 1, and indicates |
| 5022 |
that it would receive an SVC stream with the base layer being |
| 5023 |
constrained. |
| 5024 |
|
| 5025 |
Answerer -> Offerer SDP message: |
| 5026 |
|
| 5027 |
m=video 40000 RTP/AVP 97 |
| 5028 |
a=rtpmap:97 H264-SVC/90000 |
| 5029 |
a=fmtp:97 profile-level-id=53001f; packetization-mode=1; |
| 5030 |
max-recv-base-level=000d |
| 5031 |
|
| 5032 |
|
| 5033 |
|
| 5034 |
|
| 5035 |
|
| 5036 |
<span class="grey">Wenger, et al. Standards Track [Page 90]</span></pre> |
| 5037 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-91" ></span> |
| 5038 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 5039 |
|
| 5040 |
|
| 5041 |
The answering endpoint must send an SVC stream at Level 3.1. Since |
| 5042 |
the offering endpoint did not declare max-recv-base-level, the base |
| 5043 |
layer of the SVC stream the answering endpoint must send is not |
| 5044 |
specifically constrained. The offering endpoint (Alice) must send an |
| 5045 |
SVC stream at Level 3.1, for which the base layer must be of a level |
| 5046 |
not higher than Level 1.3. |
| 5047 |
|
| 5048 |
<span class="h3"><a class="selflink" id="section-7.4" href="#section-7.4">7.4</a>. Parameter Set Considerations</span> |
| 5049 |
|
| 5050 |
<a href="./rfc6184#section-8.4">Section 8.4 of [RFC6184]</a> applies in this memo, with the following |
| 5051 |
applies additionally for multi-session transmission (MST). |
| 5052 |
|
| 5053 |
In MST, regardless of out-of-band or in-band transport of parameter |
| 5054 |
sets are in use, parameter sets required for decoding NAL units |
| 5055 |
carried in one particular RTP session SHOULD be carried in the same |
| 5056 |
session, MAY be carried in a session that the particular RTP session |
| 5057 |
depends on, and MUST NOT be carried in a session that the particular |
| 5058 |
RTP session does not depend on. |
| 5059 |
|
| 5060 |
<span class="h2"><a class="selflink" id="section-8" href="#section-8">8</a>. Security Considerations</span> |
| 5061 |
|
| 5062 |
The security considerations of the RTP Payload Format for H.264 Video |
| 5063 |
specification [<a href="./rfc6184" title=""RTP Payload Format for H.264 Video"">RFC6184</a>] apply. Additionally, the following applies. |
| 5064 |
|
| 5065 |
Decoders MUST exercise caution with respect to the handling of |
| 5066 |
reserved NAL unit types and reserved SEI messages, particularly if |
| 5067 |
they contain active elements, and MUST restrict their domain of |
| 5068 |
applicability to the presentation containing the stream. The safest |
| 5069 |
way is to simply discard these NAL units and SEI messages. |
| 5070 |
|
| 5071 |
When integrity protection is applied to a stream, care MUST be taken |
| 5072 |
that the stream being transported may be scalable; hence a receiver |
| 5073 |
may be able to access only part of the entire stream. |
| 5074 |
|
| 5075 |
End-to-end security with either authentication, integrity, or |
| 5076 |
confidentiality protection will prevent a MANE from performing media- |
| 5077 |
aware operations other than discarding complete packets. And in the |
| 5078 |
case of confidentiality protection it will even be prevented from |
| 5079 |
performing discarding of packets in a media-aware way. To allow any |
| 5080 |
MANE to perform its operations, it will be required to be a trusted |
| 5081 |
entity that is included in the security context establishment. This |
| 5082 |
applies both for the media path and for the RTCP path, if RTCP |
| 5083 |
packets need to be rewritten. |
| 5084 |
|
| 5085 |
|
| 5086 |
|
| 5087 |
|
| 5088 |
|
| 5089 |
|
| 5090 |
|
| 5091 |
|
| 5092 |
<span class="grey">Wenger, et al. Standards Track [Page 91]</span></pre> |
| 5093 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-92" ></span> |
| 5094 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 5095 |
|
| 5096 |
|
| 5097 |
<span class="h2"><a class="selflink" id="section-9" href="#section-9">9</a>. Congestion Control</span> |
| 5098 |
|
| 5099 |
Within any given RTP session carrying payload according to this |
| 5100 |
specification, the provisions of <a href="./rfc6184#section-10">Section 10 of [RFC6184]</a> apply. |
| 5101 |
Reducing the session bitrate is possible by one or more of the |
| 5102 |
following means: |
| 5103 |
|
| 5104 |
a) Within the highest layer identified by the DID field remove any |
| 5105 |
NAL units with QID higher than a certain value. |
| 5106 |
|
| 5107 |
b) Remove all NAL units with TID higher than a certain value. |
| 5108 |
|
| 5109 |
c) Remove all NAL units associated with a DID higher than a certain |
| 5110 |
value. |
| 5111 |
|
| 5112 |
Informative note: Removal of all coded slice NAL units |
| 5113 |
associated with DIDs higher than a certain value in the entire |
| 5114 |
stream is required in order to preserve conformance of the |
| 5115 |
resulting SVC stream. |
| 5116 |
|
| 5117 |
d) Utilize the PRID field to indicate the relative importance of NAL |
| 5118 |
units, and remove all NAL units associated with a PRID higher than |
| 5119 |
a certain value. Note that the use of the PRID is application- |
| 5120 |
specific. |
| 5121 |
|
| 5122 |
e) Remove NAL units or entire packets according to application- |
| 5123 |
specific rules. The result will depend on the particular coding |
| 5124 |
structure used as well as any additional application-specific |
| 5125 |
functionality (e.g., concealment performed at the receiving |
| 5126 |
decoder). In general, this will result in the reception of a non- |
| 5127 |
conforming bitstream and hence the decoder behavior is not |
| 5128 |
specified by [<a href="#ref-H.264" title=""Advanced video coding for generic audiovisual services"">H.264</a>]. Significant artifacts may therefore appear |
| 5129 |
in the decoded output if the particular decoder implementation |
| 5130 |
does not take appropriate action in response to congestion |
| 5131 |
control. |
| 5132 |
|
| 5133 |
Informative note: The discussion above is centered on NAL units |
| 5134 |
rather than packets, primarily because that is the level where |
| 5135 |
senders can meaningfully manipulate the scalable bitstream. The |
| 5136 |
mapping of NAL units to RTP packets is fairly flexible when using |
| 5137 |
aggregation packets. Depending on the nature of the congestion |
| 5138 |
control algorithm, the "dimension" of congestion measurement |
| 5139 |
(packet count or bitrate) and reaction to it (reducing packet |
| 5140 |
count or bitrate or both) can be adjusted accordingly. |
| 5141 |
|
| 5142 |
All aforementioned means are available to the RTP sender, regardless |
| 5143 |
of whether that sender is located in the sending endpoint or in a |
| 5144 |
mixer-based MANE. |
| 5145 |
|
| 5146 |
|
| 5147 |
|
| 5148 |
<span class="grey">Wenger, et al. Standards Track [Page 92]</span></pre> |
| 5149 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-93" ></span> |
| 5150 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 5151 |
|
| 5152 |
|
| 5153 |
When a translator-based MANE is employed, then the MANE MAY |
| 5154 |
manipulate the session only on the MANE's outgoing path, so that the |
| 5155 |
sensed end-to-end congestion falls within the permissible envelope. |
| 5156 |
As with all translators, in this case, the MANE needs to rewrite RTCP |
| 5157 |
RRs to reflect the manipulations it has performed on the session. |
| 5158 |
|
| 5159 |
Informative note: Applications MAY also implement, in addition or |
| 5160 |
separately, other congestion control mechanisms, e.g., as |
| 5161 |
described in [<a href="./rfc5775" title=""Asynchronous Layered Coding (ALC) Protocol Instantiation"">RFC5775</a>] and [<a href="#ref-Yan" title=""Media- and TCP-friendly congestion control for scalable video streams"">Yan</a>]. |
| 5162 |
|
| 5163 |
<span class="h2"><a class="selflink" id="section-10" href="#section-10">10</a>. IANA Considerations</span> |
| 5164 |
|
| 5165 |
A new media type, as specified in <a href="#section-7.1">Section 7.1</a> of this memo, has been |
| 5166 |
registered with IANA. |
| 5167 |
|
| 5168 |
<span class="h2"><a class="selflink" id="section-11" href="#section-11">11</a>. Informative Appendix: Application Examples</span> |
| 5169 |
|
| 5170 |
<span class="h3"><a class="selflink" id="section-11.1" href="#section-11.1">11.1</a>. Introduction</span> |
| 5171 |
|
| 5172 |
Scalable video coding is a concept that has been around since at |
| 5173 |
least MPEG-2 [<a href="#ref-MPEG2">MPEG2</a>], which goes back as early as 1993. |
| 5174 |
Nevertheless, it has never gained wide acceptance, perhaps partly |
| 5175 |
because applications didn't materialize in the form envisioned during |
| 5176 |
standardization. |
| 5177 |
|
| 5178 |
ISO/IEC MPEG and ITU-T VCEG, respectively, performed a requirement |
| 5179 |
analysis for the SVC project. The MPEG and VCEG requirement |
| 5180 |
documents are available in [<a href="#ref-JVT-N026" title=""SVC requirements specified by MPEG (ISO/IEC JTC1 SC29 WG11)"">JVT-N026</a>] and [<a href="#ref-JVT-N027" title=""SVC requirements specified by VCEG (ITU-T SG16 Q.6)"">JVT-N027</a>], respectively. |
| 5181 |
|
| 5182 |
The following introduces four main application scenarios that the |
| 5183 |
authors consider relevant and that are implementable with this |
| 5184 |
specification. |
| 5185 |
|
| 5186 |
<span class="h3"><a class="selflink" id="section-11.2" href="#section-11.2">11.2</a>. Layered Multicast</span> |
| 5187 |
|
| 5188 |
This well-understood form of the use of layered coding [<a href="#ref-McCanne" title=""Receiver- driven layered multicast"">McCanne</a>] |
| 5189 |
implies that all layers are individually conveyed in their own RTP |
| 5190 |
packet streams, each carried in its own RTP session using the IP |
| 5191 |
(multicast) address and port number as the single demultiplexing |
| 5192 |
point. Receivers "tune" into the layers by subscribing to the IP |
| 5193 |
multicast, normally by using IGMP [<a href="#ref-IGMP" title=""Internet Group Management Protocol, Version 3"">IGMP</a>]. Depending on the |
| 5194 |
application scenario, it is also possible to convey a number of |
| 5195 |
layers in one RTP session, when finer operation points within the |
| 5196 |
subset of layers are not needed. |
| 5197 |
|
| 5198 |
Layered multicast has the great advantage of simplicity and easy |
| 5199 |
implementation. However, it has also the great disadvantage of |
| 5200 |
utilizing many different transport addresses. While the authors |
| 5201 |
|
| 5202 |
|
| 5203 |
|
| 5204 |
<span class="grey">Wenger, et al. Standards Track [Page 93]</span></pre> |
| 5205 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-94" ></span> |
| 5206 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 5207 |
|
| 5208 |
|
| 5209 |
consider this not to be a major problem for a professionally |
| 5210 |
maintained content server, receiving client endpoints need to open |
| 5211 |
many ports to IP multicast addresses in their firewalls. This is a |
| 5212 |
practical problem from a firewall and network address translation |
| 5213 |
(NAT) viewpoint. Furthermore, even today IP multicast is not as |
| 5214 |
widely deployed as many wish. |
| 5215 |
|
| 5216 |
The authors consider layered multicast an important application |
| 5217 |
scenario for the following reasons. First, it is well understood and |
| 5218 |
the implementation constraints are well known. Second, there may |
| 5219 |
well be large-scale IP networks outside the immediate Internet |
| 5220 |
context that may wish to employ layered multicast in the future. One |
| 5221 |
possible example could be a combination of content creation and core- |
| 5222 |
network distribution for the various mobile TV services, e.g., those |
| 5223 |
being developed by 3GPP (MBMS) [<a href="#ref-MBMS">MBMS</a>] and DVB (DVB-H) [<a href="#ref-DVB-H" title="ETSI TR 102 377">DVB-H</a>]. |
| 5224 |
|
| 5225 |
<span class="h3"><a class="selflink" id="section-11.3" href="#section-11.3">11.3</a>. Streaming</span> |
| 5226 |
|
| 5227 |
In this scenario, a streaming server has a repository of stored SVC |
| 5228 |
coded layers for a given content. At the time of streaming, and |
| 5229 |
according to the capabilities, connectivity, and congestion situation |
| 5230 |
of the client(s), the streaming server generates and serves a |
| 5231 |
scalable stream. Both unicast and multicast serving is possible. At |
| 5232 |
the same time, the streaming server may use the same repository of |
| 5233 |
stored layers to compose different streams (with a different set of |
| 5234 |
layers) intended for other audiences. |
| 5235 |
|
| 5236 |
As every endpoint receives only a single SVC RTP session, the number |
| 5237 |
of firewall pinholes can be optimized to one. |
| 5238 |
|
| 5239 |
The main difference between this scenario and straightforward |
| 5240 |
simulcasting lies in the architecture and the requirements of the |
| 5241 |
streaming server, and is therefore out of the scope of IETF |
| 5242 |
standardization. However, compelling arguments can be made why such |
| 5243 |
a streaming server design makes sense. One possible argument is |
| 5244 |
related to storage space and channel bandwidth. Another is bandwidth |
| 5245 |
adaptability without transcoding -- a considerable advantage in a |
| 5246 |
congestion controlled network. When the streaming server learns |
| 5247 |
about congestion, it can reduce the sending bitrate by choosing fewer |
| 5248 |
layers when composing the layered stream; see <a href="#section-9">Section 9</a>. SVC is |
| 5249 |
designed to gracefully support both bandwidth ramp-down and bandwidth |
| 5250 |
ramp-up with a considerable dynamic range. This payload format is |
| 5251 |
designed to allow for bandwidth flexibility in the mentioned sense. |
| 5252 |
While, in theory, a transcoding step could achieve a similar dynamic |
| 5253 |
range, the computational demands are impractically high and video |
| 5254 |
quality is typically lowered -- therefore, few (if any) streaming |
| 5255 |
servers implement full transcoding. |
| 5256 |
|
| 5257 |
|
| 5258 |
|
| 5259 |
|
| 5260 |
<span class="grey">Wenger, et al. Standards Track [Page 94]</span></pre> |
| 5261 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-95" ></span> |
| 5262 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 5263 |
|
| 5264 |
|
| 5265 |
<span class="h3"><a class="selflink" id="section-11.4" href="#section-11.4">11.4</a>. Videoconferencing (Unicast to MANE, Unicast to Endpoints)</span> |
| 5266 |
|
| 5267 |
Videoconferencing has traditionally relied on Multipoint Control |
| 5268 |
Units (MCUs). These units connect endpoints in a star configuration |
| 5269 |
and operate as follows. Coded video is transmitted from each |
| 5270 |
endpoint to the MCU, where it is decoded, scaled, and composited to |
| 5271 |
construct output frames, which are then re-encoded and transmitted to |
| 5272 |
the endpoint(s). In systems supporting personalized layout (each |
| 5273 |
user is allowed to select the layout of his/her screen), the |
| 5274 |
compositing and encoding process is performed for each of the |
| 5275 |
receiving endpoints. Even without personalized layout, rate matching |
| 5276 |
still requires that the encoding process at the MCU is performed |
| 5277 |
separately for each endpoint. As a result, MCUs have considerable |
| 5278 |
complexity and introduce significant delay. The cascaded encodings |
| 5279 |
also reduce the video quality. Particularly for multipoint |
| 5280 |
connections, interactive communication is cumbersome as the end-to- |
| 5281 |
end delay is very high [<a href="#ref-G.114" title=""One-way transmission time"">G.114</a>]. A simpler architecture is the |
| 5282 |
switching MCU, in which one of the incoming video streams is |
| 5283 |
redirected to the receiving endpoints. Obviously, only one user at a |
| 5284 |
time can be seen and rate matching cannot be performed, thus forcing |
| 5285 |
all transmitting endpoints to transmit at the lowest bit rate |
| 5286 |
available in the MCU-to-endpoint connections. |
| 5287 |
|
| 5288 |
With scalable video coding the MCU can be replaced with an |
| 5289 |
application-level router (ALR): this unit simply selects which |
| 5290 |
incoming packets should be transmitted to which of the receiving |
| 5291 |
endpoints [<a href="#ref-Eleft" title=""Multipoint Videoconferencing with Scalable Video Coding"">Eleft</a>]. In such a system, each endpoint performs its own |
| 5292 |
composition of the incoming video streams. Assuming, for example, a |
| 5293 |
system that uses spatial scalability with two layers, personalized |
| 5294 |
layout is equivalent to instructing the ALR to only send the required |
| 5295 |
packets for the corresponding resolution to the particular endpoint. |
| 5296 |
Similarly, rate matching at the ALR for a particular endpoint can be |
| 5297 |
performed by selecting an appropriate subset of the incoming video |
| 5298 |
packets to transmit to the particular endpoint. Personalized layout |
| 5299 |
and rate matching thus become routing decisions, and require no |
| 5300 |
signal processing. Note that scalability also allows participants to |
| 5301 |
enjoy the best video quality afforded by their links, i.e., users no |
| 5302 |
longer have to be forced to operate at the quality supported by the |
| 5303 |
weakest endpoint. Most importantly, the ALR has an insignificant |
| 5304 |
contribution to the end-to-end delay, typically an order of magnitude |
| 5305 |
less than an MCU. This makes it possible to have fully interactive |
| 5306 |
multipoint conferences with even a very large number of participants. |
| 5307 |
There are significant advantages as well in terms of error resilience |
| 5308 |
and, in fact, error tolerance can be increased by nearly an order of |
| 5309 |
magnitude here as well (e.g., using unequal error protection). |
| 5310 |
Finally, the very low delay of an ALR allows these systems to be |
| 5311 |
|
| 5312 |
|
| 5313 |
|
| 5314 |
|
| 5315 |
|
| 5316 |
<span class="grey">Wenger, et al. Standards Track [Page 95]</span></pre> |
| 5317 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-96" ></span> |
| 5318 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 5319 |
|
| 5320 |
|
| 5321 |
cascaded, with significant benefits in terms of system design and |
| 5322 |
deployment. Cascading of traditional MCUs is impossible due to the |
| 5323 |
very high delay that even a single MCU introduces. |
| 5324 |
|
| 5325 |
Scalable video coding enables a very significant paradigm shift in |
| 5326 |
videoconferencing systems, bringing the complexity of video |
| 5327 |
communication systems (particularly the servers residing within the |
| 5328 |
network) in line with other types of network applications. |
| 5329 |
|
| 5330 |
<span class="h3"><a class="selflink" id="section-11.5" href="#section-11.5">11.5</a>. Mobile TV (Multicast to MANE, Unicast to Endpoint)</span> |
| 5331 |
|
| 5332 |
This scenario is a bit more complex, and designed to optimize the |
| 5333 |
network traffic in a core network, while still requiring only a |
| 5334 |
single pinhole in the endpoint's firewall. One of its key |
| 5335 |
applications is the mobile TV market. |
| 5336 |
|
| 5337 |
Consider a large private IP network, e.g., the core network of the |
| 5338 |
Third Generation Partnership Project (3GPP). Streaming servers |
| 5339 |
within this core network can be assumed to be professionally |
| 5340 |
maintained. It is assumed that these servers can have many ports |
| 5341 |
open to the network and that layered multicast is a real option. |
| 5342 |
Therefore, the streaming server multicasts SVC scalable layers, |
| 5343 |
instead of simulcasting different representations of the same content |
| 5344 |
at different bitrates. |
| 5345 |
|
| 5346 |
Also consider many endpoints of different classes. Some of these |
| 5347 |
endpoints may lack the processing power or the display size to |
| 5348 |
meaningfully decode all layers; others may have these capabilities. |
| 5349 |
Users of some endpoints may wish not to pay for high quality and are |
| 5350 |
happy with a base service, which may be cheaper or even free. Other |
| 5351 |
users are willing to pay for high quality. Finally, some connected |
| 5352 |
users may have a bandwidth problem in that they can't receive the |
| 5353 |
bandwidth they would want to receive -- be it through congestion, |
| 5354 |
connectivity, change of service quality, or for whatever other |
| 5355 |
reasons. However, all these users have in common that they don't |
| 5356 |
want to be exposed too much, and therefore the number of firewall |
| 5357 |
pinholes needs to be small. |
| 5358 |
|
| 5359 |
This situation can be handled best by introducing middleboxes close |
| 5360 |
to the edge of the core network, which receive the layered multicast |
| 5361 |
streams and compose the single SVC scalable bitstream according to |
| 5362 |
the needs of the endpoint connected. These middleboxes are called |
| 5363 |
MANEs throughout this specification. In practice, the authors |
| 5364 |
envision the MANE to be part of (or at least physically and |
| 5365 |
topologically close to) the base station of a mobile network, where |
| 5366 |
all the signaling and media traffic necessarily are multiplexed on |
| 5367 |
the same physical link. |
| 5368 |
|
| 5369 |
|
| 5370 |
|
| 5371 |
|
| 5372 |
<span class="grey">Wenger, et al. Standards Track [Page 96]</span></pre> |
| 5373 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-97" ></span> |
| 5374 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 5375 |
|
| 5376 |
|
| 5377 |
MANEs necessarily need to be fairly complex devices. They certainly |
| 5378 |
need to understand the signaling, so, for example, to associate the |
| 5379 |
payload type octet in the RTP header with the SVC payload type. |
| 5380 |
|
| 5381 |
A MANE may aggregate multiple RTP streams, possibly from multiple RTP |
| 5382 |
sessions, thus to reduce the number of firewall pinholes required at |
| 5383 |
the endpoints, or may optimize the outgoing RTP stream to the MTU |
| 5384 |
size of the outgoing path by utilizing the aggregation and |
| 5385 |
fragmentation mechanisms of this memo. This type of MANE is |
| 5386 |
conceptually easy to implement and can offer powerful features, |
| 5387 |
primarily because it necessarily can "see" the payload (including the |
| 5388 |
RTP payload headers), utilize the wealth of layering information |
| 5389 |
available therein, and manipulate it. |
| 5390 |
|
| 5391 |
A MANE can also perform stream thinning, in order to adhere to |
| 5392 |
congestion control principles as discussed in <a href="#section-9">Section 9</a>. While the |
| 5393 |
implementation of the forward (media) channel of such a MANE appears |
| 5394 |
to be comparatively simple, the need to rewrite RTCP RRs makes even |
| 5395 |
such a MANE a complex device. |
| 5396 |
|
| 5397 |
While the implementation complexity of either case of a MANE, as |
| 5398 |
discussed above, is fairly high, the computational demands are |
| 5399 |
comparatively low. |
| 5400 |
|
| 5401 |
<span class="h2"><a class="selflink" id="section-12" href="#section-12">12</a>. Acknowledgements</span> |
| 5402 |
|
| 5403 |
Miska Hannuksela contributed significantly to the designs of the |
| 5404 |
PACSI NAL unit and the NI-C mode for decoding order recovery. Roni |
| 5405 |
Even organized and coordinated the design team for the development of |
| 5406 |
this memo, and provided valuable comments. Jonathan Lennox |
| 5407 |
contributed to the NAL unit reordering algorithm for MST and provided |
| 5408 |
input on several parts of this memo. Peter Amon, Sam Ganesan, Mike |
| 5409 |
Nilsson, Colin Perkins, and Thomas Wiegand were members of the design |
| 5410 |
team and provided valuable contributions. Magnus Westerlund has also |
| 5411 |
made valuable comments. Charles Eckel and Stuart Taylor provided |
| 5412 |
valuable comments after the first WGLC for this document. Xiaohui |
| 5413 |
(Joanne) Wei helped improving Table 13 and the SDP examples. |
| 5414 |
|
| 5415 |
The work of Thomas Schierl has been supported by the European |
| 5416 |
Commission under contract number FP7-ICT-248036, project COAST. |
| 5417 |
|
| 5418 |
<span class="h2"><a class="selflink" id="section-13" href="#section-13">13</a>. References</span> |
| 5419 |
|
| 5420 |
<span class="h3"><a class="selflink" id="section-13.1" href="#section-13.1">13.1</a>. Normative References</span> |
| 5421 |
|
| 5422 |
[<a id="ref-H.264">H.264</a>] ITU-T Recommendation H.264, "Advanced video coding for |
| 5423 |
generic audiovisual services", March 2010. |
| 5424 |
|
| 5425 |
|
| 5426 |
|
| 5427 |
|
| 5428 |
<span class="grey">Wenger, et al. Standards Track [Page 97]</span></pre> |
| 5429 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-98" ></span> |
| 5430 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 5431 |
|
| 5432 |
|
| 5433 |
[<a id="ref-RFC6184">RFC6184</a>] Wang, Y.-K., Even, R., Kristensen, T., and R. Jesup, "RTP |
| 5434 |
Payload Format for H.264 Video", <a href="./rfc6184">RFC 6184</a>, May 2011. |
| 5435 |
|
| 5436 |
[ISO/IEC14496-10] |
| 5437 |
ISO/IEC International Standard 14496-10:2005. |
| 5438 |
|
| 5439 |
[<a id="ref-RFC2119">RFC2119</a>] Bradner, S., "Key words for use in RFCs to Indicate |
| 5440 |
Requirement Levels", <a href="https://www.rfc-editor.org/bcp/bcp14">BCP 14</a>, <a href="./rfc2119">RFC 2119</a>, March 1997. |
| 5441 |
|
| 5442 |
[<a id="ref-RFC3264">RFC3264</a>] Rosenberg, J. and H. Schulzrinne, "An Offer/Answer Model |
| 5443 |
with Session Description Protocol (SDP)", <a href="./rfc3264">RFC 3264</a>, June |
| 5444 |
2002. |
| 5445 |
|
| 5446 |
[<a id="ref-RFC3550">RFC3550</a>] Schulzrinne, H., Casner, S., Frederick, R., and V. |
| 5447 |
Jacobson, "RTP: A Transport Protocol for Real-Time |
| 5448 |
Applications", STD 64, <a href="./rfc3550">RFC 3550</a>, July 2003. |
| 5449 |
|
| 5450 |
[<a id="ref-RFC4288">RFC4288</a>] Freed, N. and J. Klensin, "Media Type Specifications and |
| 5451 |
Registration Procedures", <a href="https://www.rfc-editor.org/bcp/bcp13">BCP 13</a>, <a href="./rfc4288">RFC 4288</a>, December 2005. |
| 5452 |
|
| 5453 |
[<a id="ref-RFC4566">RFC4566</a>] Handley, M., Jacobson, V., and C. Perkins, "SDP: Session |
| 5454 |
Description Protocol", <a href="./rfc4566">RFC 4566</a>, July 2006. |
| 5455 |
|
| 5456 |
[<a id="ref-RFC4648">RFC4648</a>] Josefsson, S., "The Base16, Base32, and Base64 Data |
| 5457 |
Encodings", <a href="./rfc4648">RFC 4648</a>, October 2006. |
| 5458 |
|
| 5459 |
[<a id="ref-RFC5576">RFC5576</a>] Lennox, J., Ott, J., and T. Schierl, "Source-Specific |
| 5460 |
Media Attributes in the Session Description Protocol |
| 5461 |
(SDP)", <a href="./rfc5576">RFC 5576</a>, June 2009. |
| 5462 |
|
| 5463 |
[<a id="ref-RFC5583">RFC5583</a>] Schierl, T. and S. Wenger, "Signaling Media Decoding |
| 5464 |
Dependency in the Session Description Protocol (SDP)", <a href="./rfc5583">RFC</a> |
| 5465 |
<a href="./rfc5583">5583</a>, July 2009. |
| 5466 |
|
| 5467 |
[<a id="ref-RFC6051">RFC6051</a>] Perkins, C. and T. Schierl, "Rapid Synchronisation of RTP |
| 5468 |
Flows", <a href="./rfc6051">RFC 6051</a>, November 2010. |
| 5469 |
|
| 5470 |
<span class="h3"><a class="selflink" id="section-13.2" href="#section-13.2">13.2</a>. Informative References</span> |
| 5471 |
|
| 5472 |
[<a id="ref-DVB-H">DVB-H</a>] DVB - Digital Video Broadcasting (DVB); DVB-H |
| 5473 |
Implementation Guidelines, ETSI TR 102 377, 2005. |
| 5474 |
|
| 5475 |
[<a id="ref-Eleft">Eleft</a>] Eleftheriadis, A., R. Civanlar, and O. Shapiro, |
| 5476 |
"Multipoint Videoconferencing with Scalable Video Coding", |
| 5477 |
Journal of Zhejiang University SCIENCE A, Vol. 7, Nr. 5, |
| 5478 |
April 2006, pp. 696-705. (Proceedings of the Packet Video |
| 5479 |
2006 Workshop.) |
| 5480 |
|
| 5481 |
|
| 5482 |
|
| 5483 |
|
| 5484 |
<span class="grey">Wenger, et al. Standards Track [Page 98]</span></pre> |
| 5485 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-99" ></span> |
| 5486 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 5487 |
|
| 5488 |
|
| 5489 |
[<a id="ref-G.114">G.114</a>] ITU-T Rec. G.114, "One-way transmission time", May 2003. |
| 5490 |
|
| 5491 |
[<a id="ref-H.241">H.241</a>] ITU-T Rec. H.241, "Extended video procedures and control |
| 5492 |
signals for H.300-series terminals", May 2006. |
| 5493 |
|
| 5494 |
[<a id="ref-IGMP">IGMP</a>] Cain, B., Deering, S., Kouvelas, I., Fenner, B., and A. |
| 5495 |
Thyagarajan, "Internet Group Management Protocol, Version |
| 5496 |
3", <a href="./rfc3376">RFC 3376</a>, October 2002. |
| 5497 |
|
| 5498 |
[<a id="ref-JVT-N026">JVT-N026</a>] Ohm J.-R., Koenen, R., and Chiariglione, L. (ed.), "SVC |
| 5499 |
requirements specified by MPEG (ISO/IEC JTC1 SC29 WG11)", |
| 5500 |
JVT-N026, available from <a href="http://ftp3.itu.ch/av-arch/jvt-site/2005_01_HongKong/JVT-N026.doc">http://ftp3.itu.ch/av-arch/</a> |
| 5501 |
<a href="http://ftp3.itu.ch/av-arch/jvt-site/2005_01_HongKong/JVT-N026.doc">jvt-site/2005_01_HongKong/JVT-N026.doc</a>, Hong Kong, China, |
| 5502 |
January 2005. |
| 5503 |
|
| 5504 |
[<a id="ref-JVT-N027">JVT-N027</a>] Sullivan, G. and Wiegand, T. (ed.), "SVC requirements |
| 5505 |
specified by VCEG (ITU-T SG16 Q.6)", JVT-N027, available |
| 5506 |
from <a href="http://ftp3.itu.int/av-arch/jvt-site/2005_01_HongKong/JVT-N027.doc">http://ftp3.itu.int/av-arch/</a> |
| 5507 |
<a href="http://ftp3.itu.int/av-arch/jvt-site/2005_01_HongKong/JVT-N027.doc">jvt-site/2005_01_HongKong/JVT-N027.doc</a>, Hong Kong, China, |
| 5508 |
January 2005. |
| 5509 |
|
| 5510 |
[<a id="ref-McCanne">McCanne</a>] McCanne, S., Jacobson, V., and Vetterli, M., "Receiver- |
| 5511 |
driven layered multicast", in Proc. of ACM SIGCOMM'96, |
| 5512 |
pages 117-130, Stanford, CA, August 1996. |
| 5513 |
|
| 5514 |
[<a id="ref-MBMS">MBMS</a>] 3GPP - Technical Specification Group Services and System |
| 5515 |
Aspects; Multimedia Broadcast/Multicast Service (MBMS); |
| 5516 |
Protocols and codecs (Release 6), December 2005. |
| 5517 |
|
| 5518 |
[<a id="ref-MPEG2">MPEG2</a>] ISO/IEC International Standard 13818-2:1993. |
| 5519 |
|
| 5520 |
[<a id="ref-RFC2326">RFC2326</a>] Schulzrinne, H., Rao, A., and R. Lanphier, "Real Time |
| 5521 |
Streaming Protocol (RTSP)", <a href="./rfc2326">RFC 2326</a>, April 1998. |
| 5522 |
|
| 5523 |
[<a id="ref-RFC2974">RFC2974</a>] Handley, M., Perkins, C., and E. Whelan, "Session |
| 5524 |
Announcement Protocol", <a href="./rfc2974">RFC 2974</a>, October 2000. |
| 5525 |
|
| 5526 |
[<a id="ref-RFC5117">RFC5117</a>] Westerlund, M. and S. Wenger, "RTP Topologies", <a href="./rfc5117">RFC 5117</a>, |
| 5527 |
January 2008. |
| 5528 |
|
| 5529 |
[<a id="ref-RFC5775">RFC5775</a>] Luby, M., Watson, M., and L. Vicisano, "Asynchronous |
| 5530 |
Layered Coding (ALC) Protocol Instantiation", <a href="./rfc5775">RFC 5775</a>, |
| 5531 |
April 2010. |
| 5532 |
|
| 5533 |
[<a id="ref-Yan">Yan</a>] Yan, J., Katrinis, K., May, M., and Plattner, R., "Media- |
| 5534 |
and TCP-friendly congestion control for scalable video |
| 5535 |
streams", in IEEE Trans. Multimedia, pages 196-206, April |
| 5536 |
2006. |
| 5537 |
|
| 5538 |
|
| 5539 |
|
| 5540 |
<span class="grey">Wenger, et al. Standards Track [Page 99]</span></pre> |
| 5541 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-100" ></span> |
| 5542 |
<span class="grey"><a href="./rfc6190">RFC 6190</a> RTP Payload Format for SVC May 2011</span> |
| 5543 |
|
| 5544 |
|
| 5545 |
Authors' Addresses |
| 5546 |
|
| 5547 |
Stephan Wenger |
| 5548 |
2400 Skyfarm Dr. |
| 5549 |
Hillsborough, CA 94010 |
| 5550 |
USA |
| 5551 |
|
| 5552 |
Phone: +1-415-713-5473 |
| 5553 |
EMail: [email protected] |
| 5554 |
|
| 5555 |
|
| 5556 |
Ye-Kui Wang |
| 5557 |
Huawei Technologies |
| 5558 |
400 Crossing Blvd, 2nd Floor |
| 5559 |
Bridgewater, NJ 08807 |
| 5560 |
USA |
| 5561 |
|
| 5562 |
Phone: +1-908-541-3518 |
| 5563 |
EMail: [email protected] |
| 5564 |
|
| 5565 |
|
| 5566 |
Thomas Schierl |
| 5567 |
Fraunhofer HHI |
| 5568 |
Einsteinufer 37 |
| 5569 |
D-10587 Berlin |
| 5570 |
Germany |
| 5571 |
|
| 5572 |
Phone: +49-30-31002-227 |
| 5573 |
EMail: [email protected] |
| 5574 |
|
| 5575 |
|
| 5576 |
Alex Eleftheriadis |
| 5577 |
Vidyo, Inc. |
| 5578 |
433 Hackensack Ave. |
| 5579 |
Hackensack, NJ 07601 |
| 5580 |
USA |
| 5581 |
|
| 5582 |
Phone: +1-201-467-5135 |
| 5583 |
EMail: [email protected] |
| 5584 |
|
| 5585 |
|
| 5586 |
|
| 5587 |
|
| 5588 |
|
| 5589 |
|
| 5590 |
|
| 5591 |
|
| 5592 |
|
| 5593 |
|
| 5594 |
|
| 5595 |
|
| 5596 |
Wenger, et al. Standards Track [Page 100] |
| 5597 |
</pre> |