| 1 |
<pre>Network Working Group S. Yasukawa |
| 2 |
Request for Comments: 5439 NTT |
| 3 |
Category: Informational A. Farrel |
| 4 |
Old Dog Consulting |
| 5 |
O. Komolafe |
| 6 |
Cisco Systems |
| 7 |
February 2009 |
| 8 |
|
| 9 |
|
| 10 |
<span class="h1">An Analysis of Scaling Issues in MPLS-TE Core Networks</span> |
| 11 |
|
| 12 |
Status of This Memo |
| 13 |
|
| 14 |
This memo provides information for the Internet community. It does |
| 15 |
not specify an Internet standard of any kind. Distribution of this |
| 16 |
memo is unlimited. |
| 17 |
|
| 18 |
Copyright Notice |
| 19 |
|
| 20 |
Copyright (c) 2009 IETF Trust and the persons identified as the |
| 21 |
document authors. All rights reserved. |
| 22 |
|
| 23 |
This document is subject to <a href="https://www.rfc-editor.org/bcp/bcp78">BCP 78</a> and the IETF Trust's Legal |
| 24 |
Provisions Relating to IETF Documents (<a href="http://trustee.ietf.org/license-info">http://trustee.ietf.org/</a> |
| 25 |
<a href="http://trustee.ietf.org/license-info">license-info</a>) in effect on the date of publication of this document. |
| 26 |
Please review these documents carefully, as they describe your rights |
| 27 |
and restrictions with respect to this document. |
| 28 |
|
| 29 |
Abstract |
| 30 |
|
| 31 |
Traffic engineered Multiprotocol Label Switching (MPLS-TE) is |
| 32 |
deployed in providers' core networks. As providers plan to grow |
| 33 |
these networks, they need to understand whether existing protocols |
| 34 |
and implementations can support the network sizes that they are |
| 35 |
planning. |
| 36 |
|
| 37 |
This document presents an analysis of some of the scaling concerns |
| 38 |
for the number of Label Switching Paths (LSPs) in MPLS-TE core |
| 39 |
networks, and examines the value of two techniques (LSP hierarchies |
| 40 |
and multipoint-to-point LSPs) for improving scaling. The intention |
| 41 |
is to motivate the development of appropriate deployment techniques |
| 42 |
and protocol extensions to enable the application of MPLS-TE in large |
| 43 |
networks. |
| 44 |
|
| 45 |
This document only considers the question of achieving scalability |
| 46 |
for the support of point-to-point MPLS-TE LSPs. Point-to-multipoint |
| 47 |
MPLS-TE LSPs are for future study. |
| 48 |
|
| 49 |
|
| 50 |
|
| 51 |
|
| 52 |
<span class="grey">Yasukawa, et al. Informational [Page 1]</span></pre> |
| 53 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-2" ></span> |
| 54 |
<span class="grey"><a href="./rfc5439">RFC 5439</a> Scaling in MPLS-TE February 2009</span> |
| 55 |
|
| 56 |
|
| 57 |
Table of Contents |
| 58 |
|
| 59 |
<a href="#section-1">1</a>. Introduction ....................................................<a href="#page-3">3</a> |
| 60 |
<a href="#section-1.1">1.1</a>. Overview ...................................................<a href="#page-3">3</a> |
| 61 |
<a href="#section-1.2">1.2</a>. Glossary of Notation .......................................<a href="#page-5">5</a> |
| 62 |
<a href="#section-2">2</a>. Issues of Concern for Scaling ...................................<a href="#page-5">5</a> |
| 63 |
<a href="#section-2.1">2.1</a>. LSP State ..................................................<a href="#page-5">5</a> |
| 64 |
<a href="#section-2.2">2.2</a>. Processing Overhead ........................................<a href="#page-6">6</a> |
| 65 |
<a href="#section-2.3">2.3</a>. RSVP-TE Implications .......................................<a href="#page-6">6</a> |
| 66 |
<a href="#section-2.4">2.4</a>. Management .................................................<a href="#page-7">7</a> |
| 67 |
<a href="#section-3">3</a>. Network Topologies ..............................................<a href="#page-8">8</a> |
| 68 |
<a href="#section-3.1">3.1</a>. The Snowflake Network Topology .............................<a href="#page-9">9</a> |
| 69 |
<a href="#section-3.2">3.2</a>. The Ladder Network Topology ...............................<a href="#page-11">11</a> |
| 70 |
<a href="#section-3.3">3.3</a>. Commercial Drivers for Selected Configurations ............<a href="#page-14">14</a> |
| 71 |
<a href="#section-3.4">3.4</a>. Other Network Topologies ..................................<a href="#page-15">15</a> |
| 72 |
<a href="#section-4">4</a>. Required Network Sizes .........................................<a href="#page-16">16</a> |
| 73 |
<a href="#section-4.1">4.1</a>. Practical Numbers .........................................<a href="#page-16">16</a> |
| 74 |
<a href="#section-5">5</a>. Scaling in Flat Networks .......................................<a href="#page-16">16</a> |
| 75 |
<a href="#section-5.1">5.1</a>. Snowflake Networks ........................................<a href="#page-17">17</a> |
| 76 |
<a href="#section-5.2">5.2</a>. Ladder Networks ...........................................<a href="#page-18">18</a> |
| 77 |
<a href="#section-6">6</a>. Scaling Snowflake Networks with Forwarding Adjacencies .........<a href="#page-22">22</a> |
| 78 |
<a href="#section-6.1">6.1</a>. Two-Layer Hierarchy .......................................<a href="#page-22">22</a> |
| 79 |
6.1.1. Tuning the Network Topology to Suit the |
| 80 |
Two-Layer Hierarchy ................................<a href="#page-23">23</a> |
| 81 |
<a href="#section-6.2">6.2</a>. Alternative Two-Layer Hierarchy ...........................<a href="#page-24">24</a> |
| 82 |
<a href="#section-6.3">6.3</a>. Three-Layer Hierarchy .....................................<a href="#page-25">25</a> |
| 83 |
<a href="#section-6.4">6.4</a>. Issues with Hierarchical LSPs .............................<a href="#page-26">26</a> |
| 84 |
<a href="#section-7">7</a>. Scaling Ladder Networks with Forwarding Adjacencies ............<a href="#page-27">27</a> |
| 85 |
<a href="#section-7.1">7.1</a>. Two-Layer Hierarchy .......................................<a href="#page-27">27</a> |
| 86 |
<a href="#section-7.2">7.2</a>. Three-Layer Hierarchy .....................................<a href="#page-28">28</a> |
| 87 |
<a href="#section-7.3">7.3</a>. Issues with Hierarchical LSPs .............................<a href="#page-29">29</a> |
| 88 |
<a href="#section-8">8</a>. Scaling Improvements through Multipoint-to-Point LSPs ..........<a href="#page-30">30</a> |
| 89 |
<a href="#section-8.1">8.1</a>. Overview of MP2P LSPs .....................................<a href="#page-30">30</a> |
| 90 |
<a href="#section-8.2">8.2</a>. LSP State: A Better Measure of Scalability ................<a href="#page-31">31</a> |
| 91 |
<a href="#section-8.3">8.3</a>. Scaling Improvements for Snowflake Networks ...............<a href="#page-32">32</a> |
| 92 |
<a href="#section-8.3.1">8.3.1</a>. Comparison with Other Scenarios ....................<a href="#page-33">33</a> |
| 93 |
<a href="#section-8.4">8.4</a>. Scaling Improvements for Ladder Networks ..................<a href="#page-34">34</a> |
| 94 |
<a href="#section-8.4.1">8.4.1</a>. Comparison with Other Scenarios ....................<a href="#page-36">36</a> |
| 95 |
<a href="#section-8.4.2">8.4.2</a>. LSP State Compared with LSP Numbers ................<a href="#page-37">37</a> |
| 96 |
<a href="#section-8.5">8.5</a>. Issues with MP2P LSPs .....................................<a href="#page-37">37</a> |
| 97 |
<a href="#section-9">9</a>. Combined Models ................................................<a href="#page-39">39</a> |
| 98 |
<a href="#section-10">10</a>. An Alternate Solution .........................................<a href="#page-39">39</a> |
| 99 |
<a href="#section-10.1">10.1</a>. Pros and Cons of the Alternate Solution ..................<a href="#page-40">40</a> |
| 100 |
<a href="#section-11">11</a>. Management Considerations .....................................<a href="#page-42">42</a> |
| 101 |
<a href="#section-12">12</a>. Security Considerations .......................................<a href="#page-42">42</a> |
| 102 |
<a href="#section-13">13</a>. Recommendations ...............................................<a href="#page-42">42</a> |
| 103 |
|
| 104 |
|
| 105 |
|
| 106 |
|
| 107 |
|
| 108 |
<span class="grey">Yasukawa, et al. Informational [Page 2]</span></pre> |
| 109 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-3" ></span> |
| 110 |
<span class="grey"><a href="./rfc5439">RFC 5439</a> Scaling in MPLS-TE February 2009</span> |
| 111 |
|
| 112 |
|
| 113 |
<a href="#section-14">14</a>. Acknowledgements ..............................................<a href="#page-43">43</a> |
| 114 |
<a href="#section-15">15</a>. Normative References ..........................................<a href="#page-43">43</a> |
| 115 |
<a href="#section-16">16</a>. Informative References ........................................<a href="#page-43">43</a> |
| 116 |
|
| 117 |
<span class="h2"><a class="selflink" id="section-1" href="#section-1">1</a>. Introduction</span> |
| 118 |
|
| 119 |
Network operators and service providers are examining scaling issues |
| 120 |
as they look to deploy ever-larger traffic engineered Multiprotocol |
| 121 |
Label Switching (MPLS-TE) networks. Concerns have been raised about |
| 122 |
the number of Label Switched Paths (LSPs) that need to be supported |
| 123 |
at the edge and at the core of the network. The impact on control |
| 124 |
plane and management plane resources threatens to outweigh the |
| 125 |
benefits and popularity of MPLS-TE, while the physical limitations of |
| 126 |
the routers may constrain the deployment options. |
| 127 |
|
| 128 |
Historically, it has been assumed that all MPLS-TE scaling issues can |
| 129 |
be addressed using hierarchical LSP [<a href="./rfc4206" title=""Label Switched Paths (LSP) Hierarchy with Generalized Multi-Protocol Label Switching (GMPLS) Traffic Engineering (TE)"">RFC4206</a>]. However, analysis |
| 130 |
shows that the improvement gained by LSP hierarchies is not as |
| 131 |
significant in all topologies and at all points in the network as |
| 132 |
might have been presumed. Further, additional management issues are |
| 133 |
introduced to determine the end-points of the hierarchical LSPs and |
| 134 |
to operate them. Although this does not invalidate the benefits of |
| 135 |
LSP hierarchies, it does indicate that additional techniques may be |
| 136 |
desirable in order to fully scale MPLS-TE networks. |
| 137 |
|
| 138 |
This document examines the scaling properties of two generic MPLS-TE |
| 139 |
network topologies and investigates the benefits of two scaling |
| 140 |
techniques. |
| 141 |
|
| 142 |
<span class="h3"><a class="selflink" id="section-1.1" href="#section-1.1">1.1</a>. Overview</span> |
| 143 |
|
| 144 |
Physical topology scaling concerns are addressed by building networks |
| 145 |
that are not fully meshed. Network topologies tend to be meshed in |
| 146 |
the core but tree-shaped at the edges, giving rise to a snowflake |
| 147 |
design. Alternatively, the core may be more of a ladder shape with |
| 148 |
tree-shaped edges. |
| 149 |
|
| 150 |
MPLS-TE, however, establishes a logical full mesh between all edge |
| 151 |
points in the network, and this is where the scaling problems arise |
| 152 |
since the structure of the network tends to focus a large number of |
| 153 |
LSPs within the core of the network. |
| 154 |
|
| 155 |
This document presents two generic network topologies (the snowflake |
| 156 |
and the ladder) and attempts to parameterize the networks by making |
| 157 |
some generalities. It introduces terminology for the different |
| 158 |
scaling parameters and examines how many LSPs might be required to be |
| 159 |
carried within the core of a network. |
| 160 |
|
| 161 |
|
| 162 |
|
| 163 |
|
| 164 |
<span class="grey">Yasukawa, et al. Informational [Page 3]</span></pre> |
| 165 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-4" ></span> |
| 166 |
<span class="grey"><a href="./rfc5439">RFC 5439</a> Scaling in MPLS-TE February 2009</span> |
| 167 |
|
| 168 |
|
| 169 |
Two techniques (hierarchical LSPs and multipoint-to-point LSPs) are |
| 170 |
introduced and an examination is made of the scaling benefits that |
| 171 |
they offer as well as of some of the concerns with using these |
| 172 |
techniques. |
| 173 |
|
| 174 |
Of necessity, this document makes many generalizations. Not least |
| 175 |
among these is a set of assumptions about the symmetry and |
| 176 |
connectivity of the physical network. It is hoped that these |
| 177 |
generalizations will not impinge on the usefulness of the overview of |
| 178 |
the scaling properties that this document attempts to give. Indeed, |
| 179 |
the symmetry of the example topologies tends to highlight the scaling |
| 180 |
issues of the different solution models, and this may be useful in |
| 181 |
exposing the worst case scenarios. |
| 182 |
|
| 183 |
Although protection mechanisms like Fast Reroute (FRR) [<a href="./rfc4090" title=""Fast Reroute Extensions to RSVP-TE for LSP Tunnels"">RFC4090</a>] are |
| 184 |
briefly discussed, the main body of this document considers stable |
| 185 |
network cases. It should be noted that make-before-break |
| 186 |
re-optimisation after link failure may result in a significant number |
| 187 |
of 'duplicate' LSPs. This issue is not addressed in this document. |
| 188 |
|
| 189 |
It should also be understood that certain deployment models where |
| 190 |
separate traffic engineered LSPs are used to provide different |
| 191 |
services (such as layer 3 Virtual Private Networks (VPNs) [<a href="./rfc4110" title=""A Framework for Layer 3 Provider-Provisioned Virtual Private Networks (PPVPNs)"">RFC4110</a>] |
| 192 |
or pseudowires [<a href="./rfc3985" title=""Pseudo Wire Emulation Edge-to-Edge (PWE3) Architecture"">RFC3985</a>]) or different classes of service [<a href="./rfc3270" title=""Multi- Protocol Label Switching (MPLS) Support of Differentiated Services"">RFC3270</a>] |
| 193 |
may result in 'duplicate' or 'parallel' LSPs running between any pair |
| 194 |
of provider edge nodes (PEs). This scaling factor is also not |
| 195 |
considered in this document, but may be easily applied as a linear |
| 196 |
factor by the reader. |
| 197 |
|
| 198 |
The operation of security mechanisms in MPLS-TE networks [<a href="#ref-MPLS-SEC" title=""Security Framework for MPLS and GMPLS Networks"">MPLS-SEC</a>] |
| 199 |
may have an impact on the ability of the network to scale. For |
| 200 |
example, they may increase both the size and number of control plane |
| 201 |
messages. Additionally, they may increase the processing overhead as |
| 202 |
control plane messages are subject to processing algorithms (such as |
| 203 |
encryption), and security keys need to be managed. Deployers will |
| 204 |
need to consider the trade-offs between scaling objectives and |
| 205 |
security objectives in their networks, and should resist the |
| 206 |
temptation to respond to a degradation of scaling performance by |
| 207 |
turning off security techniques that have previously been deemed as |
| 208 |
necessary. Further analysis of the effects of security measures on |
| 209 |
scalability are not considered further in this document. |
| 210 |
|
| 211 |
This document is designed to help service providers discover whether |
| 212 |
existing protocols and implementations can support the network sizes |
| 213 |
that they are planning. To do this, it presents an analysis of some |
| 214 |
of the scaling concerns for MPLS-TE core networks and examines the |
| 215 |
|
| 216 |
|
| 217 |
|
| 218 |
|
| 219 |
|
| 220 |
<span class="grey">Yasukawa, et al. Informational [Page 4]</span></pre> |
| 221 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-5" ></span> |
| 222 |
<span class="grey"><a href="./rfc5439">RFC 5439</a> Scaling in MPLS-TE February 2009</span> |
| 223 |
|
| 224 |
|
| 225 |
value of two techniques for improving scaling. This should motivate |
| 226 |
the development of appropriate deployment techniques and protocol |
| 227 |
extensions to enable the application of MPLS-TE in large networks. |
| 228 |
|
| 229 |
This document only considers the question of achieving scalability |
| 230 |
for the support of point-to-point MPLS-TE LSPs. Point-to-multipoint |
| 231 |
MPLS-TE LSPs are for future study. |
| 232 |
|
| 233 |
<span class="h3"><a class="selflink" id="section-1.2" href="#section-1.2">1.2</a>. Glossary of Notation</span> |
| 234 |
|
| 235 |
This document applies consistent notation to define various |
| 236 |
parameters of the networks that are analyzed. These terms are |
| 237 |
defined as they are introduced throughout the document, but are |
| 238 |
grouped together here for quick reference. Refer to the full |
| 239 |
definitions in the text for detailed explanations. |
| 240 |
|
| 241 |
n A network level. n = 1 is the core of the network. |
| 242 |
See <a href="#section-3">Section 3</a> for more details on the definition of a level. |
| 243 |
P(n) A node at level n in the network. |
| 244 |
S(n) The number of nodes at level n. That is, the number of P(n) |
| 245 |
nodes. |
| 246 |
L(n) The number of LSPs seen by a P(n) node. |
| 247 |
X(n) The number of LSP segment states held by a P(n) node. |
| 248 |
M(n) The number of P(n+1) nodes subtended to a P(n) node. |
| 249 |
R The number of rungs in a ladder network. |
| 250 |
E The number of edge nodes (PEs) subtended below (directly or |
| 251 |
indirectly) a spar-node in a ladder network. |
| 252 |
K The cost-effectiveness of the network expressed in terms of |
| 253 |
the ratio of the number of PEs to the number of network nodes. |
| 254 |
|
| 255 |
<span class="h2"><a class="selflink" id="section-2" href="#section-2">2</a>. Issues of Concern for Scaling</span> |
| 256 |
|
| 257 |
This section presents some of the issues associated with the support |
| 258 |
of LSPs at a Label Switching Router (LSR) or within the network. |
| 259 |
These issues may mean that there is a limit to the number of LSPs |
| 260 |
that can be supported. |
| 261 |
|
| 262 |
<span class="h3"><a class="selflink" id="section-2.1" href="#section-2.1">2.1</a>. LSP State</span> |
| 263 |
|
| 264 |
LSP state is the data (information) that must be stored at an LSR in |
| 265 |
order to maintain an LSP. Here, we refer to the information that is |
| 266 |
necessary to maintain forwarding plane state and the additional |
| 267 |
information required when LSPs are established through control plane |
| 268 |
protocols. While the size of the LSP state is implementation- |
| 269 |
dependent, it is clear that any implementation will require some data |
| 270 |
in order to maintain LSP state. |
| 271 |
|
| 272 |
|
| 273 |
|
| 274 |
|
| 275 |
|
| 276 |
<span class="grey">Yasukawa, et al. Informational [Page 5]</span></pre> |
| 277 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-6" ></span> |
| 278 |
<span class="grey"><a href="./rfc5439">RFC 5439</a> Scaling in MPLS-TE February 2009</span> |
| 279 |
|
| 280 |
|
| 281 |
Thus, LSP state becomes a scaling concern because as the number of |
| 282 |
LSPs at an LSR increases, so the amount of memory required to |
| 283 |
maintain the LSPs increases in direct proportion. Since the memory |
| 284 |
capacity of an LSR is limited, there is a related limit placed on the |
| 285 |
number LSPs that can be supported. |
| 286 |
|
| 287 |
Note that techniques to reduce the memory requirements (such as data |
| 288 |
compression) may serve to increase the number of LSPs that can be |
| 289 |
supported, but this will only achieve a moderate multiplier and may |
| 290 |
significantly decrease the ability to process the state rapidly. |
| 291 |
|
| 292 |
In this document, we define X(n) as "the number of LSP segment states |
| 293 |
held by a P(n) node." This definition observes that an LSR at the |
| 294 |
end of an LSP only has to maintain state in one direction (i.e., into |
| 295 |
the network), while a transit LSR must maintain state in both |
| 296 |
directions (i.e., toward both ends of the LSP). Furthermore, in |
| 297 |
multipoint-to-point (MP2P) LSPs (see <a href="#section-8">Section 8</a>), a transit LSR may |
| 298 |
need to maintain LSP state for one downstream segment (toward the |
| 299 |
destination) and multiple upstream segments (from multiple sources). |
| 300 |
That is, we define LSP segment state as the state necessary to |
| 301 |
maintain an LSP in one direction to one adjacent node. |
| 302 |
|
| 303 |
<span class="h3"><a class="selflink" id="section-2.2" href="#section-2.2">2.2</a>. Processing Overhead</span> |
| 304 |
|
| 305 |
Depending largely on implementation issues, the number of LSPs |
| 306 |
passing through an LSR may impact the processing speed for each LSP. |
| 307 |
For example, control block search times can increase with the number |
| 308 |
of control blocks to be searched, and even excellent implementations |
| 309 |
cannot completely mitigate this fact. Thus, since CPU power is |
| 310 |
constrained in any LSR, there may be a practical limit to the number |
| 311 |
of LSPs that can be supported. |
| 312 |
|
| 313 |
Further processing overhead considerations depend on issues specific |
| 314 |
to the control plane protocols, and are discussed in the next |
| 315 |
section. |
| 316 |
|
| 317 |
<span class="h3"><a class="selflink" id="section-2.3" href="#section-2.3">2.3</a>. RSVP-TE Implications</span> |
| 318 |
|
| 319 |
Like many connection-oriented signaling protocols, RSVP-TE (Resource |
| 320 |
Reservation Protocol - Traffic Engineering) requires that state is |
| 321 |
held within the network in order to maintain LSPs. The impact of |
| 322 |
this is described in <a href="#section-2.1">Section 2.1</a>. Note that RSVP-TE requires that |
| 323 |
separate information is maintained for upstream and downstream |
| 324 |
relationships, but does not require any specific implementation of |
| 325 |
that state. |
| 326 |
|
| 327 |
|
| 328 |
|
| 329 |
|
| 330 |
|
| 331 |
|
| 332 |
<span class="grey">Yasukawa, et al. Informational [Page 6]</span></pre> |
| 333 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-7" ></span> |
| 334 |
<span class="grey"><a href="./rfc5439">RFC 5439</a> Scaling in MPLS-TE February 2009</span> |
| 335 |
|
| 336 |
|
| 337 |
RSVP-TE is a soft-state protocol, which means that protocol messages |
| 338 |
(refresh messages) must be regularly exchanged between signaling |
| 339 |
neighbors in order to maintain the state for each LSP that runs |
| 340 |
between the neighbors. A common period for the transmission (and |
| 341 |
receipt) of refresh messages is 30 seconds, meaning that each LSR |
| 342 |
must send and receive one message in each direction (upstream and |
| 343 |
downstream) every 30 seconds for every LSP it supports. This has the |
| 344 |
potential to be a significant constraint on the scaling of the |
| 345 |
network, but various improvements [<a href="./rfc2961" title=""RSVP Refresh Overhead Reduction Extensions"">RFC2961</a>] mean that this refresh |
| 346 |
processing can be significantly reduced, allowing an implementation |
| 347 |
to be optimized to remove nearly all concerns about soft-state |
| 348 |
scaling in a stable network. |
| 349 |
|
| 350 |
Observations of existing implementations indicate that there may be a |
| 351 |
threshold of around 50,000 LSPs above which an LSR struggles to |
| 352 |
achieve sufficient processing to maintain LSP state. Although |
| 353 |
refresh reduction [<a href="./rfc2961" title=""RSVP Refresh Overhead Reduction Extensions"">RFC2961</a>] may substantially improve this situation, |
| 354 |
it has also been observed that under these circumstances the size of |
| 355 |
the Srefresh may become very large, and the processing required may |
| 356 |
still cause significant disruption to an LSR. |
| 357 |
|
| 358 |
Another approach is to increase the refresh time. There is a |
| 359 |
correlation between the percentage increase in refresh time and the |
| 360 |
improvement in performance for the LSR. However, it should be noted |
| 361 |
that RSVP-TE's soft-state nature depends on regular refresh messages; |
| 362 |
thus, a degree of functionality is lost by increasing the refresh |
| 363 |
time. This loss may be partially mitigated by the use of the RSVP-TE |
| 364 |
Hello message, and can also be reduced by the use of various GMPLS |
| 365 |
extensions [<a href="./rfc3473" title=""Generalized Multi-Protocol Label Switching (GMPLS) Signaling Resource ReserVation Protocol-Traffic Engineering (RSVP-TE) Extensions"">RFC3473</a>], such as the use of [<a href="./rfc2961" title=""RSVP Refresh Overhead Reduction Extensions"">RFC2961</a>] message |
| 366 |
acknowledgements on all messages. |
| 367 |
|
| 368 |
RSVP-TE also requires that signaling adjacencies be maintained |
| 369 |
through the use of Hello message exchanges. Although [<a href="./rfc3209" title=""RSVP-TE: Extensions to RSVP for LSP Tunnels"">RFC3209</a>] |
| 370 |
suggests that Hello messages should be retransmitted every 5 ms, in |
| 371 |
practice, values of around 3 seconds are more common. Nevertheless, |
| 372 |
the support of Hello messages can represent a scaling limitation on |
| 373 |
an RSVP-TE implementation since one message must be sent and received |
| 374 |
to/from each signaling adjacency every time period. This can impose |
| 375 |
limits on the number of neighbors (physical or logical) that an LSR |
| 376 |
supports, but does not impact the number of LSPs that the LSR can |
| 377 |
handle. |
| 378 |
|
| 379 |
<span class="h3"><a class="selflink" id="section-2.4" href="#section-2.4">2.4</a>. Management</span> |
| 380 |
|
| 381 |
Another practical concern for the scalability of large MPLS-TE |
| 382 |
networks is the ability to manage the network. This may be |
| 383 |
constrained by the available tools, the practicality of managing |
| 384 |
large numbers of LSPs, and the management protocols in use. |
| 385 |
|
| 386 |
|
| 387 |
|
| 388 |
<span class="grey">Yasukawa, et al. Informational [Page 7]</span></pre> |
| 389 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-8" ></span> |
| 390 |
<span class="grey"><a href="./rfc5439">RFC 5439</a> Scaling in MPLS-TE February 2009</span> |
| 391 |
|
| 392 |
|
| 393 |
Management tools are software implementations. Although such |
| 394 |
implementations should not constrain the control plane protocols, it |
| 395 |
is realistic to appreciate that network deployments will be limited |
| 396 |
by the scalability of the available tools. In practice, most |
| 397 |
existing tools have a limit to the number of LSPs that they can |
| 398 |
support. While a Network Management System (NMS) may be able to |
| 399 |
support a large number of LSPs, the number that can be supported by |
| 400 |
an Element Management System (EMS) (or the number supported by an NMS |
| 401 |
per-LSR) is more likely to be limited. |
| 402 |
|
| 403 |
Similarly, practical constraints may be imposed by the operation of |
| 404 |
management protocols. For example, an LSR may be swamped by |
| 405 |
management protocol requests to read information about the LSPs that |
| 406 |
it supports, and this might impact its ability to sustain those LSPs |
| 407 |
in the control plane. OAM (Operations, Administration, and |
| 408 |
Management), alarms, and notifications can further add to the burden |
| 409 |
placed on an LSR and limit the number of LSPs it can support. |
| 410 |
|
| 411 |
All of these considerations encourage a reduction in the number of |
| 412 |
LSPs supported within the network and at any particular LSR. |
| 413 |
|
| 414 |
<span class="h2"><a class="selflink" id="section-3" href="#section-3">3</a>. Network Topologies</span> |
| 415 |
|
| 416 |
In order to provide some generic analysis of the potential scaling |
| 417 |
issues for MPLS-TE networks, this document explores two network |
| 418 |
topology models. These topologies are selected partly because of |
| 419 |
their symmetry, which makes them more tractable to a formulaic |
| 420 |
approach, and partly because they represent generalizations of real |
| 421 |
deployment models. <a href="#section-3.3">Section 3.3</a> provides a discussion of the |
| 422 |
commercial drivers for deployed topologies and gives more analysis of |
| 423 |
why it is reasonable to consider these two topologies. |
| 424 |
|
| 425 |
The first topology is the snowflake model. In this type of network, |
| 426 |
only the very core of the network is meshed. The edges of the |
| 427 |
network are formed as trees rooted in the core. |
| 428 |
|
| 429 |
The second network topology considered is the ladder model. In this |
| 430 |
type of network, the core of the network is shaped and meshed in the |
| 431 |
form of a ladder and trees are attached rooted to the edge of the |
| 432 |
ladder. |
| 433 |
|
| 434 |
The sections that follow examine these topologies in detail in order |
| 435 |
to parameterize them. |
| 436 |
|
| 437 |
|
| 438 |
|
| 439 |
|
| 440 |
|
| 441 |
|
| 442 |
|
| 443 |
|
| 444 |
<span class="grey">Yasukawa, et al. Informational [Page 8]</span></pre> |
| 445 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-9" ></span> |
| 446 |
<span class="grey"><a href="./rfc5439">RFC 5439</a> Scaling in MPLS-TE February 2009</span> |
| 447 |
|
| 448 |
|
| 449 |
<span class="h3"><a class="selflink" id="section-3.1" href="#section-3.1">3.1</a>. The Snowflake Network Topology</span> |
| 450 |
|
| 451 |
The snowflake topologies considered in this document are based on a |
| 452 |
hierarchy of connectivity within the core network. PE nodes have |
| 453 |
connectivity to P-nodes as shown in Figure 1. There is no direct |
| 454 |
connectivity between the PEs. Dual homing of PEs to multiple P-nodes |
| 455 |
is not considered in this document, although it may be a valuable |
| 456 |
addition to a network configuration. |
| 457 |
|
| 458 |
P |
| 459 |
/|\ |
| 460 |
/ | \ |
| 461 |
/ | \ |
| 462 |
/ | \ |
| 463 |
PE PE PE |
| 464 |
|
| 465 |
Figure 1 : PE to P-Node Connectivity |
| 466 |
|
| 467 |
The relationship between P-nodes is also structured in a hierarchical |
| 468 |
way. Thus, as shown in Figure 2, multiple P-nodes at one level are |
| 469 |
connected to a P-node at a higher level. We number the levels such |
| 470 |
that level 1 is the top level (top in our figure, and nearest to the |
| 471 |
core of the network) and level (n) is immediately above level (n+1); |
| 472 |
we denote a P-node at level n as a P(n). |
| 473 |
|
| 474 |
As with PEs, there is no direct connectivity between P(n+1) nodes. |
| 475 |
Again, dual homing of P(n+1) nodes to multiple P(n) nodes is not |
| 476 |
considered in this document, although it may be a valuable addition |
| 477 |
to a network configuration. |
| 478 |
|
| 479 |
P(n) |
| 480 |
/|\ |
| 481 |
/ | \ |
| 482 |
/ | \ |
| 483 |
/ | \ |
| 484 |
P(n+1) P(n+1) P(n+1) |
| 485 |
|
| 486 |
Figure 2 : Relationship between P-Nodes |
| 487 |
|
| 488 |
At the top level, P(1) nodes are connected in a full mesh. In |
| 489 |
reality, the level 1 part of the network may be slightly less well- |
| 490 |
connected than this, but assuming a full mesh provides for |
| 491 |
generality. Thus, the snowflake topology comprises a clique with |
| 492 |
topologically equivalent trees subtended from each node in the |
| 493 |
clique. |
| 494 |
|
| 495 |
|
| 496 |
|
| 497 |
|
| 498 |
|
| 499 |
|
| 500 |
<span class="grey">Yasukawa, et al. Informational [Page 9]</span></pre> |
| 501 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-10" ></span> |
| 502 |
<span class="grey"><a href="./rfc5439">RFC 5439</a> Scaling in MPLS-TE February 2009</span> |
| 503 |
|
| 504 |
|
| 505 |
The key multipliers for scalability are the number of P(1) nodes and |
| 506 |
the multiplier relationship between P(n) and P(n+1) at each level, |
| 507 |
down to and including PEs. |
| 508 |
|
| 509 |
We define the multiplier M(n) as the number of P(n+1) nodes at level |
| 510 |
(n+1) attached to any one P(n). Assume that M(n) is constant for all |
| 511 |
nodes at level n. Since nodes at the same level are not |
| 512 |
interconnected (except at the top level), and since each P(n+1) node |
| 513 |
is connected to precisely one P(n) node, M(n) is one less than the |
| 514 |
degree of the node at level n (that is, the P(n) node is attached to |
| 515 |
M(n) nodes at level (n+1) and to 1 node at level (n-1)). |
| 516 |
|
| 517 |
We define S(n) as the number of nodes at level (n). |
| 518 |
|
| 519 |
Thus: |
| 520 |
|
| 521 |
S(n) = S(1)*M(1)*M(2)*...*M(n-1) |
| 522 |
|
| 523 |
So the number of PEs can be expressed as: |
| 524 |
|
| 525 |
S(PE) = S(1)*M(1)*M(2)*...*M(n) |
| 526 |
|
| 527 |
where the network has (n) layers of P-nodes. |
| 528 |
|
| 529 |
Thus, we may depict an example snowflake network as shown in Figure |
| 530 |
3. In this case: |
| 531 |
|
| 532 |
S(1) = 3 |
| 533 |
M(1) = 3 |
| 534 |
S(2) = S(1)*M(1) = 9 |
| 535 |
M(2) = 2 |
| 536 |
S(PE) = S(1)*M(1)*M(2) = 18 |
| 537 |
|
| 538 |
|
| 539 |
|
| 540 |
|
| 541 |
|
| 542 |
|
| 543 |
|
| 544 |
|
| 545 |
|
| 546 |
|
| 547 |
|
| 548 |
|
| 549 |
|
| 550 |
|
| 551 |
|
| 552 |
|
| 553 |
|
| 554 |
|
| 555 |
|
| 556 |
<span class="grey">Yasukawa, et al. Informational [Page 10]</span></pre> |
| 557 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-11" ></span> |
| 558 |
<span class="grey"><a href="./rfc5439">RFC 5439</a> Scaling in MPLS-TE February 2009</span> |
| 559 |
|
| 560 |
|
| 561 |
PE PE PE PE PE PE |
| 562 |
\ \/ \/ / |
| 563 |
PE--P(2) P(2) P(2) P(2)--PE |
| 564 |
\ | | / |
| 565 |
\| |/ |
| 566 |
PE--P(2)---P(1)------P(1)---P(2)--PE |
| 567 |
/ \ / \ |
| 568 |
PE \ / PE |
| 569 |
\/ |
| 570 |
P(1) |
| 571 |
/|\ |
| 572 |
/ | \ |
| 573 |
/ | \ |
| 574 |
PE--P(2) P(2) P(2)--PE |
| 575 |
/ /\ \ |
| 576 |
PE PE PE PE |
| 577 |
|
| 578 |
Figure 3 : An Example Snowflake Network |
| 579 |
|
| 580 |
<span class="h3"><a class="selflink" id="section-3.2" href="#section-3.2">3.2</a>. The Ladder Network Topology</span> |
| 581 |
|
| 582 |
The ladder networks considered in this section are based on an |
| 583 |
arrangement of routers in the core network that resembles a ladder. |
| 584 |
|
| 585 |
Ladder networks typically have long and thin cores that are arranged |
| 586 |
as conventional ladders. That is, they have one or more spars |
| 587 |
connected by rungs. Each node on a spar may have: |
| 588 |
|
| 589 |
- connection to one or more other spars, |
| 590 |
- connection to a tree of other core nodes, |
| 591 |
- connection to customer nodes. |
| 592 |
|
| 593 |
Figure 4 shows a simplified example of a ladder network. A core of |
| 594 |
twelve nodes makes up two spars connected by six rungs. |
| 595 |
|
| 596 |
|
| 597 |
|
| 598 |
|
| 599 |
|
| 600 |
|
| 601 |
|
| 602 |
|
| 603 |
|
| 604 |
|
| 605 |
|
| 606 |
|
| 607 |
|
| 608 |
|
| 609 |
|
| 610 |
|
| 611 |
|
| 612 |
<span class="grey">Yasukawa, et al. Informational [Page 11]</span></pre> |
| 613 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-12" ></span> |
| 614 |
<span class="grey"><a href="./rfc5439">RFC 5439</a> Scaling in MPLS-TE February 2009</span> |
| 615 |
|
| 616 |
|
| 617 |
PE PE PE PE |
| 618 |
PE PE PE | PE | PE PE PE | PE | PE |
| 619 |
\| \|/ |/ | \| \|/ |
| 620 |
PE-P-----P-----P-----P------P-----P--PE |
| 621 |
| | | | | |\ |
| 622 |
| | | | | | PE |
| 623 |
| | | | | | |
| 624 |
PE-P-----P-----P-----P------P-----P |
| 625 |
/| /|\ |\ |\ |\ \ |
| 626 |
PE PE PE | PE | PE | PE | PE PE |
| 627 |
PE PE PE PE |
| 628 |
|
| 629 |
Figure 4 : A Simplified Ladder Network |
| 630 |
|
| 631 |
In practice, not all nodes on a spar (call them spar-nodes) need to |
| 632 |
have subtended PEs. That is, they can exist simply to give |
| 633 |
connectivity along the spar to other spar-nodes, or across a rung to |
| 634 |
another spar. Similarly, the connectivity between spars can be more |
| 635 |
complex with multiple connections from one spar-node to another spar. |
| 636 |
Lastly, the network may be complicated by the inclusion of more than |
| 637 |
two spars (or simplified by reduction to a single spar). |
| 638 |
|
| 639 |
These variables make the ladder network non-trivial to model. For |
| 640 |
the sake of simplicity, we will make the following restrictions: |
| 641 |
|
| 642 |
- There are precisely two spars in the core network. |
| 643 |
|
| 644 |
- Every spar-node connects to precisely one spar-node on the other |
| 645 |
spar. That is, each spar-node is attached to precisely one rung. |
| 646 |
|
| 647 |
- Each spar-node connects to either one (end-spar) or two (core-spar) |
| 648 |
other spar-nodes on the same spar. |
| 649 |
|
| 650 |
- Every spar-node has the same number of PEs subtended. This does |
| 651 |
not mean that there are no P-nodes subtended to the spar-nodes, but |
| 652 |
does mean that the edge tree subtended to each spar-node is |
| 653 |
identical. |
| 654 |
|
| 655 |
From these restrictions, we are able to quantify a ladder network as |
| 656 |
follows: |
| 657 |
|
| 658 |
R - The number of rungs. That is, the number of spar-nodes on |
| 659 |
each spar. |
| 660 |
S(1) - The number of spar-nodes in the network. S(1)=2*R. |
| 661 |
E - The number of subtended edge nodes (PEs) to each spar-node. |
| 662 |
|
| 663 |
|
| 664 |
|
| 665 |
|
| 666 |
|
| 667 |
|
| 668 |
<span class="grey">Yasukawa, et al. Informational [Page 12]</span></pre> |
| 669 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-13" ></span> |
| 670 |
<span class="grey"><a href="./rfc5439">RFC 5439</a> Scaling in MPLS-TE February 2009</span> |
| 671 |
|
| 672 |
|
| 673 |
The number of rungs may vary considerably. A number less than 3 is |
| 674 |
unlikely (since that would not be a significantly connected network), |
| 675 |
and a number greater than 100 seems improbable (because that would |
| 676 |
represent a very long, thin network). |
| 677 |
|
| 678 |
E can be treated as for the snowflake network. That is, we can |
| 679 |
consider a number of levels of attachment from P(1) nodes, which are |
| 680 |
the spar-nodes, through P(i) down to P(n), which are the PEs. |
| 681 |
Practically, we need to only consider n=2 (PEs attached direct to the |
| 682 |
spar-nodes) and n=3 (one level of P-nodes between the PEs and the |
| 683 |
spar-nodes). |
| 684 |
|
| 685 |
Let M(i) be the ratio of P(i) nodes to P(i-1) nodes, i.e., the |
| 686 |
connectivity between levels of P-node as defined for the snowflake |
| 687 |
topology. Hence, the number of nodes at any level (n) is: |
| 688 |
|
| 689 |
S(n) = S(1)*M(1)*M(2)*...*M(n-1) |
| 690 |
|
| 691 |
So the number of PEs subtended to a spar-node is: |
| 692 |
|
| 693 |
E = M(1)*M(2)*...*M(n) |
| 694 |
|
| 695 |
And the number of PEs can be expressed as: |
| 696 |
|
| 697 |
S(PE) = S(1)*M(1)*M(2)*...*M(n) |
| 698 |
= S(1)*E |
| 699 |
|
| 700 |
Thus, we may depict an example ladder network as shown in Figure 5. |
| 701 |
In this case: |
| 702 |
|
| 703 |
R = 5 |
| 704 |
S(1) = 10 |
| 705 |
M(1) = 2 |
| 706 |
S(2) = S(1)*M(1) = 20 |
| 707 |
M(2) = 2 |
| 708 |
E = M(1)*M(2) = 4 |
| 709 |
S(PE) = S(1)*E = 40 |
| 710 |
|
| 711 |
|
| 712 |
|
| 713 |
|
| 714 |
|
| 715 |
|
| 716 |
|
| 717 |
|
| 718 |
|
| 719 |
|
| 720 |
|
| 721 |
|
| 722 |
|
| 723 |
|
| 724 |
<span class="grey">Yasukawa, et al. Informational [Page 13]</span></pre> |
| 725 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-14" ></span> |
| 726 |
<span class="grey"><a href="./rfc5439">RFC 5439</a> Scaling in MPLS-TE February 2009</span> |
| 727 |
|
| 728 |
|
| 729 |
PE PE PE PE PE PE PE PE PE PE PE PE PE PE PE PE |
| 730 |
\| \| \| \| |/ |/ |/ |/ |
| 731 |
P(2) P(2) P(2) P(2) P(2) P(2) P(2) P(2) |
| 732 |
\ \ | \ / | / / |
| 733 |
PE \ \ | \ / | / / PE |
| 734 |
\ \ \| \/ |/ / / |
| 735 |
PE-P(2)---P(1)---P(1)---P(1)---P(1)---P(1)---P(2)-PE |
| 736 |
| | | | | |
| 737 |
| | | | | |
| 738 |
| | | | | |
| 739 |
PE-P(2)---P(1)---P(1)---P(1)---P(1)---P(1)---P(2)-PE |
| 740 |
/ / / | /\ |\ \ \ |
| 741 |
PE / / | / \ | \ \ PE |
| 742 |
/ / | / \ | \ \ |
| 743 |
P(2) P(2) P(2) P(2) P(2) P(2) P(2) P(2) |
| 744 |
/| /| /| /| |\ |\ |\ |\ |
| 745 |
PE PE PE PE PE PE PE PE PE PE PE PE PE PE PE PE |
| 746 |
|
| 747 |
Figure 5 : An Example Ladder Network |
| 748 |
|
| 749 |
<span class="h3"><a class="selflink" id="section-3.3" href="#section-3.3">3.3</a>. Commercial Drivers for Selected Configurations</span> |
| 750 |
|
| 751 |
It is reasonable to ask why these two particular network topologies |
| 752 |
have been chosen. |
| 753 |
|
| 754 |
The most important consideration is physical scalability. Each node |
| 755 |
(Label Switching Router - LSR) is only able to support a limited |
| 756 |
number of physical interfaces. This necessarily reduces the ability |
| 757 |
to fully mesh a network and leads to the tree-like structure of the |
| 758 |
network toward the PEs. |
| 759 |
|
| 760 |
A realistic commercial consideration for an operator is the fact that |
| 761 |
the only revenue-generating nodes in the network are the PEs. Other |
| 762 |
nodes are needed only to support connectivity and scalability. |
| 763 |
Therefore, there is a desire to maximize S(PE) while minimizing the |
| 764 |
sum of S(n) for all values of (n). This could be achieved by |
| 765 |
minimizing the number of levels and maximizing the connectivity at |
| 766 |
each layer, M(n). Ultimately, however, this would produce a network |
| 767 |
of just interconnected PEs, which is clearly in conflict with the |
| 768 |
physical scaling situation. |
| 769 |
|
| 770 |
Therefore, the solution calls for a "few" levels with "relatively |
| 771 |
large" connectivity at each level. We might say that the cost- |
| 772 |
effectiveness of the network can be stated as: |
| 773 |
|
| 774 |
K = S(PE)/(S(1)+S(2) + ... + S(n)) where n is the level above the PEs |
| 775 |
|
| 776 |
|
| 777 |
|
| 778 |
|
| 779 |
|
| 780 |
<span class="grey">Yasukawa, et al. Informational [Page 14]</span></pre> |
| 781 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-15" ></span> |
| 782 |
<span class="grey"><a href="./rfc5439">RFC 5439</a> Scaling in MPLS-TE February 2009</span> |
| 783 |
|
| 784 |
|
| 785 |
We should observe, however, that this equation may be naive in that |
| 786 |
the cost of a network is not actually a function of the number of |
| 787 |
routers (since a router chassis is often free or low cost), but is |
| 788 |
really a function of the cost of the line cards, which is, itself, a |
| 789 |
product of the capacity of the line cards. Thus, the relatively high |
| 790 |
connectivity decreases the cost-effectiveness, while a topology that |
| 791 |
tends to channel data through a network core tends to demand higher |
| 792 |
capacity (and so, more expensive) line cards. |
| 793 |
|
| 794 |
A further consideration is the availability of connectivity (usually |
| 795 |
fibers) between LSR sites. Although it is always possible to lay new |
| 796 |
fiber, this may not be cost-effective or timely. The physical shape |
| 797 |
and topography of the country in which the network is laid is likely |
| 798 |
to be as much of a problem. If the country is 'long and thin', then |
| 799 |
a ladder network is likely to be used. |
| 800 |
|
| 801 |
This document examines the implications for control plane and data |
| 802 |
plane scalability of this type of network when MPLS-TE LSPs are used |
| 803 |
to provide full connectivity between all PEs. |
| 804 |
|
| 805 |
<span class="h3"><a class="selflink" id="section-3.4" href="#section-3.4">3.4</a>. Other Network Topologies</span> |
| 806 |
|
| 807 |
As explained in <a href="#section-1">Section 1</a>, this document is using two symmetrical and |
| 808 |
generalized network topologies for simplicity of modelling. In |
| 809 |
practice, there are two other topological considerations. |
| 810 |
|
| 811 |
a. Multihoming |
| 812 |
It is relatively common for a node at level (n) to be attached to |
| 813 |
more than one node at level (n-1). This is particularly common at |
| 814 |
PEs that may be connected to more than one P(n). |
| 815 |
|
| 816 |
b. Meshing within a level |
| 817 |
A level in the network will often include links between P-nodes at |
| 818 |
the same level, including the possibility of links between PEs. |
| 819 |
This may result in a network that looks like a series of |
| 820 |
concentric circles with spokes. |
| 821 |
|
| 822 |
Both of these features are likely to have some impact on the scaling |
| 823 |
of the networks. However, for the purposes of establishing the |
| 824 |
ground rules for scaling, this document restricts itself to the |
| 825 |
consideration of the symmetrical networks described in Sections <a href="#section-2.1">2.1</a> |
| 826 |
and 2.2. Discussion of other network formats is for future study. |
| 827 |
|
| 828 |
|
| 829 |
|
| 830 |
|
| 831 |
|
| 832 |
|
| 833 |
|
| 834 |
|
| 835 |
|
| 836 |
<span class="grey">Yasukawa, et al. Informational [Page 15]</span></pre> |
| 837 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-16" ></span> |
| 838 |
<span class="grey"><a href="./rfc5439">RFC 5439</a> Scaling in MPLS-TE February 2009</span> |
| 839 |
|
| 840 |
|
| 841 |
<span class="h2"><a class="selflink" id="section-4" href="#section-4">4</a>. Required Network Sizes</span> |
| 842 |
|
| 843 |
An important question for this evaluation and analysis is the size of |
| 844 |
the network that operators require. How many PEs are required? What |
| 845 |
ratio of P to PE is acceptable? How many ports do devices have for |
| 846 |
physical connectivity? What type of MPLS-TE connectivity between PEs |
| 847 |
is required? |
| 848 |
|
| 849 |
Although presentation of figures for desired network sizes must be |
| 850 |
treated with caution because history shows that networks grow beyond |
| 851 |
all projections, it is useful to set some acceptable lower bounds. |
| 852 |
That is, we can state that we are interested in networks of at least |
| 853 |
a certain size. |
| 854 |
|
| 855 |
The most important features are: |
| 856 |
|
| 857 |
- The network should have at least 1000 PEs. |
| 858 |
- Each pair of PEs should be connected by at least one LSP in each |
| 859 |
direction. |
| 860 |
|
| 861 |
<span class="h3"><a class="selflink" id="section-4.1" href="#section-4.1">4.1</a>. Practical Numbers</span> |
| 862 |
|
| 863 |
In practice, reasonable target numbers are as follows. |
| 864 |
|
| 865 |
S(PE) >= 1000 |
| 866 |
Number of levels is 3. That is: 1, 2, and PE. |
| 867 |
M(2) <= 20 |
| 868 |
M(1) <= 20 |
| 869 |
S(1) <= 100 |
| 870 |
|
| 871 |
<span class="h2"><a class="selflink" id="section-5" href="#section-5">5</a>. Scaling in Flat Networks</span> |
| 872 |
|
| 873 |
Before proceeding to examine potential scaling improvements, we need |
| 874 |
to examine how well the flat networks described in the previous |
| 875 |
sections scale. |
| 876 |
|
| 877 |
Consider the requirement for a full mesh of LSPs linking all PEs. |
| 878 |
That is, each PE has an LSP to and from every other LSP. Thus, if |
| 879 |
there are S(PE) PEs in the network, there are S(PE)*(S(PE) - 1) LSPs. |
| 880 |
|
| 881 |
Define L(n) as the number of LSPs handled by a level (n) LSR. |
| 882 |
|
| 883 |
L(PE) = 2*(S(PE) - 1) |
| 884 |
|
| 885 |
|
| 886 |
|
| 887 |
|
| 888 |
|
| 889 |
|
| 890 |
|
| 891 |
|
| 892 |
<span class="grey">Yasukawa, et al. Informational [Page 16]</span></pre> |
| 893 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-17" ></span> |
| 894 |
<span class="grey"><a href="./rfc5439">RFC 5439</a> Scaling in MPLS-TE February 2009</span> |
| 895 |
|
| 896 |
|
| 897 |
<span class="h3"><a class="selflink" id="section-5.1" href="#section-5.1">5.1</a>. Snowflake Networks</span> |
| 898 |
|
| 899 |
There are a total of S(PE) PEs in the network and, since each PE |
| 900 |
establishes an LSP with every other PE, it would be expected that |
| 901 |
there are S(PE) - 1 LSPs incoming to each PE and the same number of |
| 902 |
LSPs outgoing from the same PE, giving a total of 2(S(PE) - 1) on the |
| 903 |
incident link. Hence, in a snowflake topology (see Figure 3), since |
| 904 |
there are M(2) PEs attached to each P(2) node, it may tempting to |
| 905 |
think that L(2) (the number of LSPs traversing each P(2) node) is |
| 906 |
simply 2*(S(PE) - 1)*M(2). However, it should be noted that of the |
| 907 |
S(PE) - 1 LSPs incoming to each PE, M(2) - 1 originated from nodes |
| 908 |
attached to the same P(2) node, and so this value would count the |
| 909 |
LSPs between the M(2) PEs attached to each P(2) node twice: once when |
| 910 |
outgoing from the M(2) - 1 other nodes and once when incoming into a |
| 911 |
particular PE. |
| 912 |
|
| 913 |
There are a total of M(2)*(M(2) - 1) LSPs between these M(2) PEs and, |
| 914 |
since this value is erroneously included twice in 2*(S(PE) - 1)*M(2), |
| 915 |
the correct value is: |
| 916 |
|
| 917 |
L(2) = 2*M(2)*(S(PE) - 1) - M(2)*(M(2) - 1) |
| 918 |
= M(2)*(2*S(PE) - M(2) - 1) |
| 919 |
|
| 920 |
An alternative way of looking at this, that proves extensible for the |
| 921 |
calculation of L(1), is to observe that each PE subtended to a P(2) |
| 922 |
node has an LSP in each direction to all S(PE) - M(2) PEs in the rest |
| 923 |
of the system, and there are M(2) such locally subtended PEs; thus, |
| 924 |
2*M(2)*(S(PE) - M(2)). Additionally, there are M(2)*(M(2) - 1) LSPs |
| 925 |
between the locally subtended PEs. So: |
| 926 |
|
| 927 |
L(2) = 2*M(2)*(S(PE) - M(2)) + M(2)*(M(2) - 1) |
| 928 |
= M(2)*(2*S(PE) - M(2) - 1) |
| 929 |
|
| 930 |
L(1) can be computed in the same way as this second evaluation of |
| 931 |
L(2). Each PE subtended below a P(1) node has an LSP in each |
| 932 |
direction to all PEs not below the P(1) node. There are M(1)*M(2) |
| 933 |
PEs below the P(1) node, so this accounts for 2*M(1)*M(2)*(S(PE) - |
| 934 |
M(1)*M(2)) LSPs. To this, we need to add the number of LSPs that |
| 935 |
pass through the P(1) node and that run between the PEs subtended |
| 936 |
below the P(1). Consider each P(2): it has M(2) PEs, each of which |
| 937 |
has an LSP going to all of the PEs subtended to the other P(2) nodes |
| 938 |
subtended to the P(1). There are M(1) - 1 such other P(2) nodes, and |
| 939 |
so M(2)*(M(1) - 1) other such PEs. So the number of LSPs from the |
| 940 |
PEs below a P(2) node is M(2)*M(2)*(M(1) - 1). And there are M(1) |
| 941 |
P(2) nodes below the P(1), giving rise to a total of |
| 942 |
M(2)*M(2)*M(1)*(M(1) - 1) LSPs. Thus: |
| 943 |
|
| 944 |
|
| 945 |
|
| 946 |
|
| 947 |
|
| 948 |
<span class="grey">Yasukawa, et al. Informational [Page 17]</span></pre> |
| 949 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-18" ></span> |
| 950 |
<span class="grey"><a href="./rfc5439">RFC 5439</a> Scaling in MPLS-TE February 2009</span> |
| 951 |
|
| 952 |
|
| 953 |
L(1) = 2*M(1)*M(2)*(S(PE) - M(1)*M(2)) + M(2)*M(2)*M(1)*(M(1) - 1) |
| 954 |
= M(1)*M(2)*(2*S(PE) - M(2)*(M(1) + 1)) |
| 955 |
|
| 956 |
So, for example, with S(1) = 5, M(1) = 10, and M(2) = 20, we see: |
| 957 |
|
| 958 |
S(PE) = 1000 |
| 959 |
L(PE) = 1998 |
| 960 |
L(2) = 39580 |
| 961 |
L(1) = 356000 |
| 962 |
|
| 963 |
Alternatively, with S(1) = 10, M(1) = 10, and M(2) = 20, we see: |
| 964 |
|
| 965 |
S(PE) = 2000 |
| 966 |
L(PE) = 3998 |
| 967 |
L(2) = 79580 |
| 968 |
L(1) = 756000 |
| 969 |
|
| 970 |
In both examples, the number of LSPs at the core (P(1)) nodes is |
| 971 |
probably unacceptably large, even though there are only a relatively |
| 972 |
modest number of PEs. In fact, L(2) may even be too large in the |
| 973 |
second example. |
| 974 |
|
| 975 |
<span class="h3"><a class="selflink" id="section-5.2" href="#section-5.2">5.2</a>. Ladder Networks</span> |
| 976 |
|
| 977 |
In ladder networks, L(PE) remains the same at 2*(S(PE) - 1). |
| 978 |
|
| 979 |
L(2) can be computed using the same mechanism as for the snowflake |
| 980 |
topology because the subtended tree is the same format. Hence, |
| 981 |
|
| 982 |
L(2) = 2*M(2)*(S(PE) - 1) - M(2)*(M(2) - 1) |
| 983 |
|
| 984 |
But L(1) requires a different computation because each P(1) not only |
| 985 |
sees LSPs for the subtended PEs, but is also a transit node for some |
| 986 |
of the LSPs that cross the core (the core is not fully meshed). |
| 987 |
|
| 988 |
Each P(1) sees: |
| 989 |
|
| 990 |
o all of the LSPs between locally attached PEs, |
| 991 |
o less those LSPs between locally attached PEs that can be served |
| 992 |
exclusively by the attached P(2) nodes, |
| 993 |
o all LSPs between locally attached PEs and remote PEs, and |
| 994 |
o LSPs in transit that pass through the P(1). |
| 995 |
|
| 996 |
The first three numbers are easily determined and match what we have |
| 997 |
seen from the snowflake network. They are: |
| 998 |
|
| 999 |
|
| 1000 |
|
| 1001 |
|
| 1002 |
|
| 1003 |
|
| 1004 |
<span class="grey">Yasukawa, et al. Informational [Page 18]</span></pre> |
| 1005 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-19" ></span> |
| 1006 |
<span class="grey"><a href="./rfc5439">RFC 5439</a> Scaling in MPLS-TE February 2009</span> |
| 1007 |
|
| 1008 |
|
| 1009 |
o E*(E-1) |
| 1010 |
o M(1)*M(2)*(M(2)-1) = E*(M(2) - 1) |
| 1011 |
o 2*E*E*(S(1) - 1) |
| 1012 |
|
| 1013 |
The number of LSPs in transit is more complicated to compute. It is |
| 1014 |
simplified by not considering the ends of the ladders but by |
| 1015 |
examining an arbitrary segment of the middle of the ladder, such as |
| 1016 |
shown in Figure 6. We look to compute and generalize the number of |
| 1017 |
LSPs traversing each core link (labeled a and b in Figure 6) and so |
| 1018 |
determine the number of transit LSPs seen by each P(1). |
| 1019 |
|
| 1020 |
: : : : : : |
| 1021 |
: : : : : : |
| 1022 |
P(2) P(2) P(2) P(2) P(2) P(2) |
| 1023 |
\ | \ / | / |
| 1024 |
\ | \ / | / |
| 1025 |
\| \/ |/ |
| 1026 |
......P(1)---P(1)---P(1)...... |
| 1027 |
| a | | |
| 1028 |
| |b | |
| 1029 |
| | | |
| 1030 |
......P(1)---P(1)---P(1)...... |
| 1031 |
/| /\ |\ |
| 1032 |
/ | / \ | \ |
| 1033 |
/ | / \ | \ |
| 1034 |
P(2) P(2) P(2) P(2) P(2) P(2) |
| 1035 |
: : : : : : |
| 1036 |
: : : : : : |
| 1037 |
|
| 1038 |
Figure 6 : An Arbitrary Section of a Ladder Network |
| 1039 |
|
| 1040 |
Of course, the number of LSPs carried on links a and b in Figure 6 |
| 1041 |
depends on how LSPs are routed through the core network. But if we |
| 1042 |
assume a symmetrical routing policy and an even distribution of LSPs |
| 1043 |
across all shortest paths, the result is the same. |
| 1044 |
|
| 1045 |
Now we can see that each P(1) sees half of 2a+b LSPs (since each LSP |
| 1046 |
would otherwise be counted twice as it passed through the P(1)), |
| 1047 |
except that some of the LSPs are locally terminated and so are only |
| 1048 |
included once in the sum 2a+b. |
| 1049 |
|
| 1050 |
So L(1) = a + b/2 - |
| 1051 |
(locally terminated transit LSPs)/2 + |
| 1052 |
(locally contained LSPs) |
| 1053 |
|
| 1054 |
|
| 1055 |
|
| 1056 |
|
| 1057 |
|
| 1058 |
|
| 1059 |
|
| 1060 |
<span class="grey">Yasukawa, et al. Informational [Page 19]</span></pre> |
| 1061 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-20" ></span> |
| 1062 |
<span class="grey"><a href="./rfc5439">RFC 5439</a> Scaling in MPLS-TE February 2009</span> |
| 1063 |
|
| 1064 |
|
| 1065 |
Thus: |
| 1066 |
|
| 1067 |
L(1) = a + b/2 - |
| 1068 |
2*E*E*(S(1) - 1)/2 + |
| 1069 |
E*(E-1) - E*(M(2) - 1) |
| 1070 |
= a + b/2 + |
| 1071 |
E*E*(2 - S(1)) - E*M(2) |
| 1072 |
|
| 1073 |
So all we have to do is work out a and b. |
| 1074 |
|
| 1075 |
Recall that the ladder length R = S(1)/2, and define X = E*E. |
| 1076 |
|
| 1077 |
Consider the contribution made by all of the LSPs that make n hops on |
| 1078 |
the ladder to the totals of each of a and b. If the ladder was |
| 1079 |
unbounded, then we could say that in the case of a, there are n*2X |
| 1080 |
LSPs along the spar only, and n(n-1)*2X/n = 2X(n-1) LSPs use a rung |
| 1081 |
and the spar. Thus, the LSPs that make n hops on the ladder |
| 1082 |
contribute (4n-2)X LSPs to a. Note that the edge cases are special |
| 1083 |
because LSPs that make only one hop on the ladder cannot transit a |
| 1084 |
P(1) but only start or end there. |
| 1085 |
|
| 1086 |
So with a ladder of length R = S(1)/2, we could say: |
| 1087 |
|
| 1088 |
R |
| 1089 |
a = SUM[(4i-2)*X] + 2RX |
| 1090 |
i=2 |
| 1091 |
|
| 1092 |
= 2*X*R*(R+1) |
| 1093 |
|
| 1094 |
And similarly, considering b in an unbounded ladder, the LSPs that |
| 1095 |
only travel one hop on the LSP are a special case, contributing 2X |
| 1096 |
LSPs, and every other LSP that traverses n hops on the ladder |
| 1097 |
contributes 2n*2X/n = 4X LSPs. So: |
| 1098 |
|
| 1099 |
R+1 |
| 1100 |
b = 2X + SUM[4X] |
| 1101 |
i=2 |
| 1102 |
|
| 1103 |
= 2*X + 4*X*R |
| 1104 |
|
| 1105 |
In fact, the ladders are bounded, and so the number of LSPs is |
| 1106 |
reduced because of the effect of the ends of the ladders. The links |
| 1107 |
that see the most LSPs are in the middle of the ladder. Consider a |
| 1108 |
ladder of length R; a node in the middle of the ladder is R/2 hops |
| 1109 |
away from the end of the ladder. So we see that the formula for the |
| 1110 |
contribution to the count of spar-only LSPs for a is only valid up to |
| 1111 |
n=R/2, and for spar-and-rung LSPs, up to n=1+R/2. Above these |
| 1112 |
limits, the contribution made by spar-only LSPs decays as (n-R/2)*2X. |
| 1113 |
|
| 1114 |
|
| 1115 |
|
| 1116 |
<span class="grey">Yasukawa, et al. Informational [Page 20]</span></pre> |
| 1117 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-21" ></span> |
| 1118 |
<span class="grey"><a href="./rfc5439">RFC 5439</a> Scaling in MPLS-TE February 2009</span> |
| 1119 |
|
| 1120 |
|
| 1121 |
However, for a first-order approximation, we will use the values of a |
| 1122 |
and b as computed above. This gives us an upper bound of the number |
| 1123 |
of LSPs without using a more complex formula for the reduction made |
| 1124 |
by the effect of the ends of the ladder. |
| 1125 |
|
| 1126 |
From this: |
| 1127 |
|
| 1128 |
L(1) = a + b/2 + |
| 1129 |
E*E*(2 - S(1)) - E*M(2) |
| 1130 |
= 2*X*R*(R+1) + |
| 1131 |
X + 2*X*R + |
| 1132 |
E*E*(2 - S(1)) - E*M(2) |
| 1133 |
= E*E*S(1)*(1 + S(1)/2) + |
| 1134 |
E*E + E*E*S(1) + |
| 1135 |
2*E+E - E*E*S(1) - E*M(2) |
| 1136 |
= E*E*S(1)*(1 + S(1)/2) + 3*E+E - E*M(2) |
| 1137 |
= E*E*S(1)*S(1)/2 + E*E*S(1) + 3*E*E - E*M(2) |
| 1138 |
|
| 1139 |
So, for example, with S(1) = 6, M(1) = 10, and M(2) = 17, we see: |
| 1140 |
|
| 1141 |
E = 170 |
| 1142 |
S(PE) = 1020 |
| 1143 |
L(PE) = 2038 |
| 1144 |
L(2) = 34374 |
| 1145 |
L(1) = 777410 |
| 1146 |
|
| 1147 |
Alternatively, with S(1) = 10, M(1) = 10, and M(2) = 20, we see: |
| 1148 |
|
| 1149 |
E = 200 |
| 1150 |
S(PE) = 2000 |
| 1151 |
L(PE) = 3998 |
| 1152 |
L(2) = 79580 |
| 1153 |
L(1) = 2516000 |
| 1154 |
|
| 1155 |
In both examples, the number of LSPs at the core (P(1)) nodes is |
| 1156 |
probably unacceptably large, even though there are only a relatively |
| 1157 |
modest number of PEs. In fact, L(2) may even be too large in the |
| 1158 |
second example. |
| 1159 |
|
| 1160 |
Compare the L(1) values with the total number of LSPs in the system |
| 1161 |
S(PE)*(S(PE) - 1), which is 1039380 and 3998000, respectively. |
| 1162 |
|
| 1163 |
|
| 1164 |
|
| 1165 |
|
| 1166 |
|
| 1167 |
|
| 1168 |
|
| 1169 |
|
| 1170 |
|
| 1171 |
|
| 1172 |
<span class="grey">Yasukawa, et al. Informational [Page 21]</span></pre> |
| 1173 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-22" ></span> |
| 1174 |
<span class="grey"><a href="./rfc5439">RFC 5439</a> Scaling in MPLS-TE February 2009</span> |
| 1175 |
|
| 1176 |
|
| 1177 |
<span class="h2"><a class="selflink" id="section-6" href="#section-6">6</a>. Scaling Snowflake Networks with Forwarding Adjacencies</span> |
| 1178 |
|
| 1179 |
One of the purposes of LSP hierarchies [<a href="./rfc4206" title=""Label Switched Paths (LSP) Hierarchy with Generalized Multi-Protocol Label Switching (GMPLS) Traffic Engineering (TE)"">RFC4206</a>] is to improve the |
| 1180 |
scaling properties of MPLS-TE networks. LSP tunnels (sometimes known |
| 1181 |
as Forwarding Adjacencies (FAs)) may be established to provide |
| 1182 |
connectivity over the core of the network, and multiple edge-to-edge |
| 1183 |
LSPs may be tunneled down a single FA LSP. |
| 1184 |
|
| 1185 |
In our network we consider a mesh of FA LSPs between all core nodes |
| 1186 |
at the same level. We consider two possibilities here. In the |
| 1187 |
first, all P(2) nodes are connected to all other P(2) nodes by LSP |
| 1188 |
tunnels, and the PE-to-PE LSPs are tunneled across the core of the |
| 1189 |
network. In the second, an extra layer of LSP hierarchy is |
| 1190 |
introduced by connecting all P(1) nodes in an LSP mesh and tunneling |
| 1191 |
the P(2)-to-P(2) tunnels through these. |
| 1192 |
|
| 1193 |
<span class="h3"><a class="selflink" id="section-6.1" href="#section-6.1">6.1</a>. Two-Layer Hierarchy</span> |
| 1194 |
|
| 1195 |
In this hierarchy model, the P(2) nodes are connected by a mesh of |
| 1196 |
tunnels. This means that the P(1) nodes do not see the PE-to-PE |
| 1197 |
LSPs. |
| 1198 |
|
| 1199 |
It remains the case that: |
| 1200 |
|
| 1201 |
L(PE) = 2*(S(PE) - 1) |
| 1202 |
|
| 1203 |
L(2) is slightly increased. It can be computed as the sum of all |
| 1204 |
LSPs for all attached PEs, including the LSPs between the attached PE |
| 1205 |
(this figure is unchanged from <a href="#section-5.1">Section 5.1</a>, i.e., M(2)*(2*S(PE) - |
| 1206 |
M(2) - 1)), plus the number of FA LSPs providing a mesh to the other |
| 1207 |
P(2) nodes. Since the number of P(2) nodes is S(2), each P(2) node |
| 1208 |
sees 2*(S(2) - 1) FA LSPs. Thus: |
| 1209 |
|
| 1210 |
L(2) = M(2)*(2*S(PE) - M(2) - 1) + 2*(S(2) - 1) |
| 1211 |
|
| 1212 |
L(1), however, is significantly reduced and can be computed as the |
| 1213 |
sum of the number of FA LSPs to and from each attached P(2) to each |
| 1214 |
other P(2) in the network, including (but counting only once) the FA |
| 1215 |
LSPs between attached P(2) nodes. In fact, the problem is identical |
| 1216 |
to the L(2) computation in <a href="#section-5.1">Section 5.1</a>. So: |
| 1217 |
|
| 1218 |
L(1) = M(1)*(2*S(2) - M(1) - 1) |
| 1219 |
|
| 1220 |
|
| 1221 |
|
| 1222 |
|
| 1223 |
|
| 1224 |
|
| 1225 |
|
| 1226 |
|
| 1227 |
|
| 1228 |
<span class="grey">Yasukawa, et al. Informational [Page 22]</span></pre> |
| 1229 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-23" ></span> |
| 1230 |
<span class="grey"><a href="./rfc5439">RFC 5439</a> Scaling in MPLS-TE February 2009</span> |
| 1231 |
|
| 1232 |
|
| 1233 |
So, for example, with S(1) = 5, M(1) = 10, and M(2) = 20, we see: |
| 1234 |
|
| 1235 |
S(PE) = 1000 |
| 1236 |
S(2) = 50 |
| 1237 |
L(PE) = 1998 |
| 1238 |
L(2) = 39678 |
| 1239 |
L(1) = 890 |
| 1240 |
|
| 1241 |
Alternatively, with S(1) = 10, M(1) = 10, and M(2) = 20, we see: |
| 1242 |
|
| 1243 |
S(PE) = 2000 |
| 1244 |
S(2) = 100 |
| 1245 |
L(PE) = 3998 |
| 1246 |
L(2) = 79778 |
| 1247 |
L(1) = 1890 |
| 1248 |
|
| 1249 |
So, in both examples, potential problems at the core (P(1)) nodes |
| 1250 |
caused by an excessive number of LSPs can be avoided, but any problem |
| 1251 |
with L(2) is made slightly worse, as can be seen from the table |
| 1252 |
below. |
| 1253 |
|
| 1254 |
Example| Count | Unmodified | 2-Layer |
| 1255 |
| | (<a href="#section-5.1">Section 5.1</a>) | Hierarchy |
| 1256 |
-------+-------+---------------+---------- |
| 1257 |
A | L(2) | 39580 | 39678 |
| 1258 |
| L(1) | 356000 | 890 |
| 1259 |
-------+-------+---------------+---------- |
| 1260 |
B | L(2) | 79580 | 79778 |
| 1261 |
| L(1) | 756000 | 1890 |
| 1262 |
|
| 1263 |
<span class="h4"><a class="selflink" id="section-6.1.1" href="#section-6.1.1">6.1.1</a>. Tuning the Network Topology to Suit the Two-Layer Hierarchy</span> |
| 1264 |
|
| 1265 |
Clearly, we can reduce L(2) by selecting appropriate values of S(1), |
| 1266 |
M(1), and M(2). We can do this without negative consequences, since |
| 1267 |
no change will affect L(PE) and since a large percentage increase in |
| 1268 |
L(1) is sustainable now that L(1) is so small. |
| 1269 |
|
| 1270 |
Observe that: |
| 1271 |
|
| 1272 |
L(2) = M(2)*(2*S(PE) - M(2) - 1) + 2*(S(2) - 1) |
| 1273 |
|
| 1274 |
where S(PE) = S(1)*M(1)*M(2) and S(2) = S(1)*M(1). So L(2) scales |
| 1275 |
with M(2)^2 and we can have the most impact by reducing M(2) while |
| 1276 |
keeping S(PE) constant. |
| 1277 |
|
| 1278 |
|
| 1279 |
|
| 1280 |
|
| 1281 |
|
| 1282 |
|
| 1283 |
|
| 1284 |
<span class="grey">Yasukawa, et al. Informational [Page 23]</span></pre> |
| 1285 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-24" ></span> |
| 1286 |
<span class="grey"><a href="./rfc5439">RFC 5439</a> Scaling in MPLS-TE February 2009</span> |
| 1287 |
|
| 1288 |
|
| 1289 |
For example, with S(1) = 10, M(1) = 10, and M(2) = 10, we see: |
| 1290 |
|
| 1291 |
S(PE) = 1000 |
| 1292 |
S(2) = 100 |
| 1293 |
L(PE) = 1998 |
| 1294 |
L(2) = 20088 |
| 1295 |
L(1) = 1890 |
| 1296 |
|
| 1297 |
And similarly, with S(1) = 20, M(1) = 20, and M(2) = 5, we see: |
| 1298 |
|
| 1299 |
S(PE) = 2000 |
| 1300 |
S(2) = 400 |
| 1301 |
L(PE) = 3998 |
| 1302 |
L(2) = 20768 |
| 1303 |
L(1) = 15580 |
| 1304 |
|
| 1305 |
These considerable scaling benefits must be offset against the cost- |
| 1306 |
effectiveness of the network. Recall from <a href="#section-3.3">Section 3.3</a> that: |
| 1307 |
|
| 1308 |
K = S(PE)/(S(1)+S(2) ... + S(n)) |
| 1309 |
|
| 1310 |
where n is the level above the PEs, so that for our network: |
| 1311 |
|
| 1312 |
K = S(PE) / (S(1) + S(2)) |
| 1313 |
|
| 1314 |
Thus, in the first example the cost-effectiveness has been halved |
| 1315 |
from 1000/55 to 1000/110. In the second example, it has been reduced |
| 1316 |
to roughly one quarter, changing from 2000/110 to 2000/420. |
| 1317 |
|
| 1318 |
So, although the tuning changes may be necessary to reach the desired |
| 1319 |
network size, they come at a considerable cost to the operator. |
| 1320 |
|
| 1321 |
<span class="h3"><a class="selflink" id="section-6.2" href="#section-6.2">6.2</a>. Alternative Two-Layer Hierarchy</span> |
| 1322 |
|
| 1323 |
An alternative to the two-layer hierarchy presented in <a href="#section-6.1">Section 6.1</a> is |
| 1324 |
to provide a full mesh of FA LSPs between P(1) nodes. This technique |
| 1325 |
is only of benefit to any nodes in the core of the level 1 network. |
| 1326 |
It makes no difference to the PE and P(2) nodes since they continue |
| 1327 |
to see only the PE-to-PE LSPs. Furthermore, this approach increases |
| 1328 |
the burden at the P(1) nodes since they have to support all of the |
| 1329 |
PE-to-PE LSPs as in the flat model plus the additional 2*(S(1) - 1) |
| 1330 |
P(1)-to-P(1) FA LSPs. Thus, this approach should only be considered |
| 1331 |
where there is a mesh of P-nodes within the ring of P(1) nodes, and |
| 1332 |
is not considered further in this document. |
| 1333 |
|
| 1334 |
|
| 1335 |
|
| 1336 |
|
| 1337 |
|
| 1338 |
|
| 1339 |
|
| 1340 |
<span class="grey">Yasukawa, et al. Informational [Page 24]</span></pre> |
| 1341 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-25" ></span> |
| 1342 |
<span class="grey"><a href="./rfc5439">RFC 5439</a> Scaling in MPLS-TE February 2009</span> |
| 1343 |
|
| 1344 |
|
| 1345 |
<span class="h3"><a class="selflink" id="section-6.3" href="#section-6.3">6.3</a>. Three-Layer Hierarchy</span> |
| 1346 |
|
| 1347 |
As demonstrated by <a href="#section-6.2">Section 6.2</a>, introducing a mesh of FA LSPs at the |
| 1348 |
top level (P(1)) has no benefit, but if we introduce an additional |
| 1349 |
level in the network (P(3) between P(2) and PE) to make a four-level |
| 1350 |
snowflake, we can introduce a new layer of FA LSPs so that we have a |
| 1351 |
full mesh of FA LSPs between all P(3) nodes to carry the PE-to-PE |
| 1352 |
LSPs, and a full mesh of FA LSPs between all P(2) nodes to carry the |
| 1353 |
P(3)-to-P(3) LSPs. |
| 1354 |
|
| 1355 |
The number of PEs is S(PE) = S(1)*M(1)*M(2)*M(3), and the number of |
| 1356 |
PE-to-PE LSPs at a PE remains L(PE) = 2*(S(PE) - 1). |
| 1357 |
|
| 1358 |
The number of LSPs at a P(3) can be deduced from <a href="#section-6.1">Section 6.1</a>. It is |
| 1359 |
the sum of all LSPs for all attached PEs, including the LSPs between |
| 1360 |
the attached PE, plus the number of FA LSPs providing a mesh to the |
| 1361 |
other P(3) nodes. |
| 1362 |
|
| 1363 |
L(3) = M(3)*(2*S(PE) - M(3) - 1) + 2*(S(3) - 1) |
| 1364 |
|
| 1365 |
The number of LSPs at P(2) can also be deduced from <a href="#section-6.1">Section 6.1</a> since |
| 1366 |
it is the sum of all LSPs for all attached P(3) nodes, including the |
| 1367 |
LSPs between the attached PE plus the number of FA LSPs providing a |
| 1368 |
mesh to the other P(2) nodes. |
| 1369 |
|
| 1370 |
L(2) = M(2)*(2*S(3) - M(2) - 1) + 2*(S(2) - 1) |
| 1371 |
|
| 1372 |
Finally, L(1) can be copied straight from 6.1. |
| 1373 |
|
| 1374 |
L(1) = M(1)*(2*S(2) - M(1) - 1) |
| 1375 |
|
| 1376 |
For example, with S(1) = 5, M(1) = 5, M(2) = 5, and M(3) = 8, we see: |
| 1377 |
|
| 1378 |
S(PE) = 1000 |
| 1379 |
S(3) = 125 |
| 1380 |
S(2) = 25 |
| 1381 |
L(PE) = 1998 |
| 1382 |
L(3) = 16176 |
| 1383 |
L(2) = 1268 |
| 1384 |
L(1) = 220 |
| 1385 |
|
| 1386 |
|
| 1387 |
|
| 1388 |
|
| 1389 |
|
| 1390 |
|
| 1391 |
|
| 1392 |
|
| 1393 |
|
| 1394 |
|
| 1395 |
|
| 1396 |
<span class="grey">Yasukawa, et al. Informational [Page 25]</span></pre> |
| 1397 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-26" ></span> |
| 1398 |
<span class="grey"><a href="./rfc5439">RFC 5439</a> Scaling in MPLS-TE February 2009</span> |
| 1399 |
|
| 1400 |
|
| 1401 |
Similarly, with S(1) = 5, M(1) = 5, M(2) = 8, and M(3) = 10, we see: |
| 1402 |
|
| 1403 |
S(PE) = 2000 |
| 1404 |
S(3) = 200 |
| 1405 |
S(2) = 25 |
| 1406 |
L(PE) = 3998 |
| 1407 |
L(3) = 40038 |
| 1408 |
L(2) = 3184 |
| 1409 |
L(1) = 220 |
| 1410 |
|
| 1411 |
Clearly, there are considerable scaling improvements with this three- |
| 1412 |
layer hierarchy, and all of the numbers (even L(3) in the second |
| 1413 |
example) are manageable. |
| 1414 |
|
| 1415 |
Of course, the extra level in the network tends to reduce the cost- |
| 1416 |
effectiveness of the networks with values of K = 1000/155 and K = |
| 1417 |
2000/230 (from 1000/55 and 2000/110) for the examples above. That is |
| 1418 |
a reduction by a factor of 3 in the first case and 2 in the second |
| 1419 |
case. Such a change in cost-effectiveness has to be weighed against |
| 1420 |
the desire to deploy such a large network. If LSP hierarchies are |
| 1421 |
the only scaling tool available, and networks this size are required, |
| 1422 |
the cost-effectiveness may need to be sacrificed. |
| 1423 |
|
| 1424 |
<span class="h3"><a class="selflink" id="section-6.4" href="#section-6.4">6.4</a>. Issues with Hierarchical LSPs</span> |
| 1425 |
|
| 1426 |
A basic observation for hierarchical scaling techniques is that it is |
| 1427 |
hard to have any impact on the number of LSPs that must be supported |
| 1428 |
by the level of P(n) nodes adjacent to the PEs (for example, it is |
| 1429 |
hard to reduce L(3) in <a href="#section-6.3">Section 6.3</a>). In fact, the only way we can |
| 1430 |
change the number of LSPs supported by these nodes is to change the |
| 1431 |
scaling ratio M(n) in the network -- in other words, to change the |
| 1432 |
number of PEs subtended to any P(n). But such a change has a direct |
| 1433 |
effect on the number of PEs in the network and so the cost- |
| 1434 |
effectiveness is impacted. |
| 1435 |
|
| 1436 |
Another concern with the hierarchical approach is that it must be |
| 1437 |
configured and managed. This may not seem like a large burden, but |
| 1438 |
it must be recalled that the P(n) nodes are not at the edge of the |
| 1439 |
network -- they are a set of nodes that must be identified so that |
| 1440 |
the FA LSPs can be configured and provisioned. Effectively, the |
| 1441 |
operator must plan and construct a layered network with a ring of |
| 1442 |
P(n) nodes giving access to the level (n) network. This design |
| 1443 |
activity is open to considerable risk as failing to close the ring |
| 1444 |
(i.e., allowing a node to be at both level (n+1) and at level (n)) |
| 1445 |
may cause operational confusion. |
| 1446 |
|
| 1447 |
|
| 1448 |
|
| 1449 |
|
| 1450 |
|
| 1451 |
|
| 1452 |
<span class="grey">Yasukawa, et al. Informational [Page 26]</span></pre> |
| 1453 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-27" ></span> |
| 1454 |
<span class="grey"><a href="./rfc5439">RFC 5439</a> Scaling in MPLS-TE February 2009</span> |
| 1455 |
|
| 1456 |
|
| 1457 |
Protocol techniques (such as IGP automesh [<a href="./rfc4972" title=""Routing Extensions for Discovery of Multiprotocol (MPLS) Label Switch Router (LSR) Traffic Engineering (TE) Mesh Membership"">RFC4972</a>]) have been |
| 1458 |
developed to reduce the configuration necessary to build this type of |
| 1459 |
multi-level network. In the case of automesh, the routing protocol |
| 1460 |
is used to advertise the membership of a 'mesh group', and all |
| 1461 |
members of the mesh group can discover each other and connect with |
| 1462 |
LSP tunnels. Thus, the P(n) nodes giving access to level (n) can |
| 1463 |
advertise their existence to each other, and it is not necessary to |
| 1464 |
configure each with information about all of the others. Although |
| 1465 |
this process can help to reduce the configuration overhead, it does |
| 1466 |
not eliminate it, as each member of the mesh group must still be |
| 1467 |
planned and configured for membership. |
| 1468 |
|
| 1469 |
An important consideration for the use of hierarchical LSPs is how |
| 1470 |
they can be protected using MPLS Fast Reroute (FRR) [<a href="./rfc4090" title=""Fast Reroute Extensions to RSVP-TE for LSP Tunnels"">RFC4090</a>]. FRR |
| 1471 |
may provide link protection either by protecting the tunnels as they |
| 1472 |
traverse a broken link or by treating each level (n) tunnel LSP as a |
| 1473 |
link in level (n+1) and providing protection for the level (n+1) LSPs |
| 1474 |
(although in this model, fault detection and propagation time may be |
| 1475 |
an issue). Node protection may be performed in a similar way, but |
| 1476 |
protection of the first and last nodes of a hierarchical LSP is |
| 1477 |
particularly difficult. Additionally, the whole notion of scaling |
| 1478 |
with regard to FRR gives rise to separate concerns that are outside |
| 1479 |
the scope of this document as currently formulated. |
| 1480 |
|
| 1481 |
Finally, observe that we have been explaining these techniques using |
| 1482 |
conveniently symmetrical networks. Consider how we would arrange the |
| 1483 |
hierarchical LSPs in a network where some PEs are connected closer to |
| 1484 |
the center of the network than others. |
| 1485 |
|
| 1486 |
<span class="h2"><a class="selflink" id="section-7" href="#section-7">7</a>. Scaling Ladder Networks with Forwarding Adjacencies</span> |
| 1487 |
|
| 1488 |
<span class="h3"><a class="selflink" id="section-7.1" href="#section-7.1">7.1</a>. Two-Layer Hierarchy</span> |
| 1489 |
|
| 1490 |
In <a href="#section-6.2">Section 6.2</a>, we observed that there is no value to placing FA LSPs |
| 1491 |
between the P(1) nodes of our example snowflake topologies. This is |
| 1492 |
because those LSPs would be just one hop long and would, in fact, |
| 1493 |
only serve to increase the burden at the P(1) nodes. However, in the |
| 1494 |
ladder model, there is value to this approach. The P(1) nodes are |
| 1495 |
the spar-nodes of the ladder, and they are not all mutually adjacent. |
| 1496 |
That is, the P(1)-to-P(1) hierarchical LSPs can create a full mesh of |
| 1497 |
P(1) nodes where one does not exist in the physical topology. |
| 1498 |
|
| 1499 |
The number of LSPs seen by a P(1) node is then: |
| 1500 |
|
| 1501 |
o all of the tunnels terminating at the P(1) node, |
| 1502 |
o any transit tunnels, and |
| 1503 |
o all of the LSPs due to subtended PEs. |
| 1504 |
|
| 1505 |
|
| 1506 |
|
| 1507 |
|
| 1508 |
<span class="grey">Yasukawa, et al. Informational [Page 27]</span></pre> |
| 1509 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-28" ></span> |
| 1510 |
<span class="grey"><a href="./rfc5439">RFC 5439</a> Scaling in MPLS-TE February 2009</span> |
| 1511 |
|
| 1512 |
|
| 1513 |
This is a substantial reduction; all of the transit LSPs are reduced |
| 1514 |
to just one per remote P(1) that causes any transit LSP. So: |
| 1515 |
|
| 1516 |
L(1) = 2*(S(1) - 1) + |
| 1517 |
O(S(1)*S(1)/2) + |
| 1518 |
2*E*E*(S(1) - 1) + E*(E-1) - E*(M(2) - 1) |
| 1519 |
|
| 1520 |
where O(S(1)*S(1)/2) gives an upper bound order of magnitude. So: |
| 1521 |
|
| 1522 |
L(1) = S(1)*S(1)/2 + 2*S(1) + 2*E*E*(S(1) - 1) - E*M(2) - 2 |
| 1523 |
|
| 1524 |
So, in our two examples: |
| 1525 |
|
| 1526 |
With S(1) = 6, M(1) = 10, and M(2) = 17, we see: |
| 1527 |
|
| 1528 |
E = 170 |
| 1529 |
S(PE) = 1020 |
| 1530 |
L(PE) = 2038 |
| 1531 |
L(2) = 34374 |
| 1532 |
L(1) = 286138 |
| 1533 |
|
| 1534 |
Alternatively, with S(1) = 10, M(1) = 10, and M(2) = 20, we see: |
| 1535 |
|
| 1536 |
E = 200 |
| 1537 |
S(PE) = 2000 |
| 1538 |
L(PE) = 3998 |
| 1539 |
L(2) = 79580 |
| 1540 |
L(1) = 716060 |
| 1541 |
|
| 1542 |
Both of these show significant improvements over the previous L(1) |
| 1543 |
figures of 777410 and 2516000. But the numbers are still too large |
| 1544 |
to manage, and there is no improvement in the L(2) figures. |
| 1545 |
|
| 1546 |
<span class="h3"><a class="selflink" id="section-7.2" href="#section-7.2">7.2</a>. Three-Layer Hierarchy</span> |
| 1547 |
|
| 1548 |
We can also apply the three-layer hierarchy to the ladder model. In |
| 1549 |
this case, the number of LSPs between P(1) nodes is not reduced, but |
| 1550 |
tunnels are also set up between all P(2) nodes. Thus, the number of |
| 1551 |
LSPs seen by a P(1) node is: |
| 1552 |
|
| 1553 |
o all of the tunnels terminating at the P(1) node, |
| 1554 |
o any transit tunnels between P(1) nodes, and |
| 1555 |
o all of the LSPs due to subtended P(2) nodes. |
| 1556 |
|
| 1557 |
No PE-to-PE LSPs are seen at the P(1) nodes. |
| 1558 |
|
| 1559 |
|
| 1560 |
|
| 1561 |
|
| 1562 |
|
| 1563 |
|
| 1564 |
<span class="grey">Yasukawa, et al. Informational [Page 28]</span></pre> |
| 1565 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-29" ></span> |
| 1566 |
<span class="grey"><a href="./rfc5439">RFC 5439</a> Scaling in MPLS-TE February 2009</span> |
| 1567 |
|
| 1568 |
|
| 1569 |
L(1) = 2*(S(1) - 1) + |
| 1570 |
O(S(1)*S(1)/2) + |
| 1571 |
2*(S(1) - 1)*M(1)*M(1) + M(1)*(M(1) - 1) |
| 1572 |
|
| 1573 |
where O(S(1)*S(1)/2) gives an upper bound order of magnitude. So: |
| 1574 |
|
| 1575 |
L(1) = S(1)*S(1)/2 + 2*S(1) + 2*M(1)*M(1)*S(1) - M(1)(M(1) + 1) - 2 |
| 1576 |
|
| 1577 |
Unfortunately, there is a small increase in the number of LSPs seen |
| 1578 |
by the P(2) nodes. Each P(2) now sees all of the PE-to-PE LSPs that |
| 1579 |
it saw before and is also an end-point for a set of P(2)-to-P(2) |
| 1580 |
tunnels. Thus, L(2) increases to: |
| 1581 |
|
| 1582 |
L(2) = 2*M(2)*(S(PE) - 1) - M(2)*(M(2) - 1) + 2*(S(1)*M(1) - 1) |
| 1583 |
|
| 1584 |
So, in our two examples: |
| 1585 |
|
| 1586 |
With S(1) = 6, M(1) = 10, and M(2) = 17, we see: |
| 1587 |
|
| 1588 |
E = 170 |
| 1589 |
S(PE) = 1020 |
| 1590 |
L(PE) = 2038 |
| 1591 |
L(2) = 34492 |
| 1592 |
L(1) = 1118 |
| 1593 |
|
| 1594 |
Alternatively, with S(1) = 10, M(1) = 10, and M(2) = 20, we see: |
| 1595 |
|
| 1596 |
E = 200 |
| 1597 |
S(PE) = 2000 |
| 1598 |
L(PE) = 3998 |
| 1599 |
L(2) = 79778 |
| 1600 |
L(1) = 1958 |
| 1601 |
|
| 1602 |
This represents a very dramatic decrease in LSPs across the core. |
| 1603 |
|
| 1604 |
<span class="h3"><a class="selflink" id="section-7.3" href="#section-7.3">7.3</a>. Issues with Hierarchical LSPs</span> |
| 1605 |
|
| 1606 |
The same issues exist for hierarchical LSPs as described in <a href="#section-6.4">Section</a> |
| 1607 |
<a href="#section-6.4">6.4</a>. Although dramatic improvements can be made to the scaling |
| 1608 |
numbers for the number of LSPs at core nodes, this can only be done |
| 1609 |
at the cost of configuring P(2) to P(2) tunnels. The mesh of P(1) |
| 1610 |
tunnels is not enough. |
| 1611 |
|
| 1612 |
But the sheer number of P(2) to P(2) tunnels that must be configured |
| 1613 |
is a significant management burden that can only be eased by using a |
| 1614 |
technique like automesh [<a href="./rfc4972" title=""Routing Extensions for Discovery of Multiprotocol (MPLS) Label Switch Router (LSR) Traffic Engineering (TE) Mesh Membership"">RFC4972</a>]. |
| 1615 |
|
| 1616 |
|
| 1617 |
|
| 1618 |
|
| 1619 |
|
| 1620 |
<span class="grey">Yasukawa, et al. Informational [Page 29]</span></pre> |
| 1621 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-30" ></span> |
| 1622 |
<span class="grey"><a href="./rfc5439">RFC 5439</a> Scaling in MPLS-TE February 2009</span> |
| 1623 |
|
| 1624 |
|
| 1625 |
It is significant, however, that the scaling problem at the P(2) |
| 1626 |
nodes cannot be improved by using tunnels and that the only solution |
| 1627 |
to ease this in the hierarchical approach would be to institute |
| 1628 |
another layer of hierarchy (that is, P(3) nodes) between the P(2) |
| 1629 |
nodes and the PEs. This is, of course, a significant expense. |
| 1630 |
|
| 1631 |
<span class="h2"><a class="selflink" id="section-8" href="#section-8">8</a>. Scaling Improvements through Multipoint-to-Point LSPs</span> |
| 1632 |
|
| 1633 |
An alternative (or complementary) scaling technique has been proposed |
| 1634 |
using multipoint-to-point (MP2P) LSPs. The fundamental improvement |
| 1635 |
in this case is achieved by reducing the number of LSPs toward the |
| 1636 |
destination as LSPs toward the same destination are merged. |
| 1637 |
|
| 1638 |
This section presents an overview of MP2P LSPs and describes their |
| 1639 |
applicability and scaling benefits. |
| 1640 |
|
| 1641 |
<span class="h3"><a class="selflink" id="section-8.1" href="#section-8.1">8.1</a>. Overview of MP2P LSPs</span> |
| 1642 |
|
| 1643 |
Note that the MP2P LSPs discussed here are for MPLS-TE and are not |
| 1644 |
the same concept familiar in the Label Distribution Protocol (LDP) |
| 1645 |
described in [<a href="./rfc5036" title=""LDP Specification"">RFC5036</a>]. |
| 1646 |
|
| 1647 |
Traffic flows generally converge toward their destination and this |
| 1648 |
can be utilized by MPLS in constructing an MP2P LSP. With such an |
| 1649 |
LSP, the Label Forwarding Information Base (LFIB) mappings at each |
| 1650 |
LSR are many-to-one so that multiple pairs {incoming interface, |
| 1651 |
incoming label} are mapped to a single pair {outgoing interface, |
| 1652 |
outgoing label}. Obviously, if per-platform labels are used, this |
| 1653 |
mapping may be optimized within an implementation. |
| 1654 |
|
| 1655 |
It is important to note that with MP2P MPLS-TE LSPs, the traffic |
| 1656 |
flows are merged. That is, some additional form of identifier is |
| 1657 |
required if de-merging is required. For example, if the payload is |
| 1658 |
IP traffic belonging to the same client network, no additional de- |
| 1659 |
merging information is required since the IP packet contains |
| 1660 |
sufficient data. On the other hand, if the data comes, for example, |
| 1661 |
from a variety of VPN client networks, then the flows will need to be |
| 1662 |
labeled in their own right as point-to-point (P2P) flows, so that |
| 1663 |
traffic can be disambiguated at the egress of the MP2P LSPs. |
| 1664 |
|
| 1665 |
Techniques for establishing MP2P MPLS-TE LSPs and for assigning the |
| 1666 |
correct bandwidth downstream of LSP merge points are out of the scope |
| 1667 |
of this document. |
| 1668 |
|
| 1669 |
|
| 1670 |
|
| 1671 |
|
| 1672 |
|
| 1673 |
|
| 1674 |
|
| 1675 |
|
| 1676 |
<span class="grey">Yasukawa, et al. Informational [Page 30]</span></pre> |
| 1677 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-31" ></span> |
| 1678 |
<span class="grey"><a href="./rfc5439">RFC 5439</a> Scaling in MPLS-TE February 2009</span> |
| 1679 |
|
| 1680 |
|
| 1681 |
<span class="h3"><a class="selflink" id="section-8.2" href="#section-8.2">8.2</a>. LSP State: A Better Measure of Scalability</span> |
| 1682 |
|
| 1683 |
Consider the network topology shown in Figure 3. Suppose that we |
| 1684 |
establish MP2P LSP tunnels such that there is one tunnel terminating |
| 1685 |
at each PE, and that that tunnel has every other PE as an ingress. |
| 1686 |
Thus, a PE-to-PE MP2P LSP tunnel would have S(PE)-1 ingresses and one |
| 1687 |
egress, and there would be S(PE) such tunnels. |
| 1688 |
|
| 1689 |
Note that there still remain 2*(S(PE) - 1) PE-to-PE P2P LSPs that are |
| 1690 |
carried through these tunnels. |
| 1691 |
|
| 1692 |
Let's consider the number of LSPs handled at each node in the |
| 1693 |
network. |
| 1694 |
|
| 1695 |
The PEs continue to handle the same number of PE-to-PE P2P LSPs, and |
| 1696 |
must also handle the MP2P LSPs. So: |
| 1697 |
|
| 1698 |
L(PE) = 2*(S(PE) - 1) + S(PE) |
| 1699 |
|
| 1700 |
But all P(n) nodes in the network only handle the MP2P LSP tunnels. |
| 1701 |
Nominally, this means that L(n) = S(PE) for all values of n. This |
| 1702 |
would appear to be a great success with the number of LSPs cut to |
| 1703 |
completely manageable levels. |
| 1704 |
|
| 1705 |
However, the number of LSPs is not the only issue (although it may |
| 1706 |
have some impact for some of the scaling concerns listed in <a href="#section-4">Section</a> |
| 1707 |
<a href="#section-4">4</a>). We are more interested in the amount of LSP state that is |
| 1708 |
maintained by an LSR. This reflects the amount of storage required |
| 1709 |
at the LSR, the amount of protocol processing, and the amount of |
| 1710 |
information that needs to be managed. |
| 1711 |
|
| 1712 |
In fact, we were also interested in this measure of scalability in |
| 1713 |
the earlier sections of this document, but in those cases we could |
| 1714 |
see a direct correlation between the number of LSPs and the amount of |
| 1715 |
LSP state since transit LSPs had two pieces of state information (one |
| 1716 |
on the incoming and one on the outgoing interface), and ingress or |
| 1717 |
egress LSPs had just one piece of state. |
| 1718 |
|
| 1719 |
We can quantify the amount of LSP state according to the number of |
| 1720 |
LSP segments managed by an LSR. So (as above), in the case of a P2P |
| 1721 |
LSP, an ingress or egress has one segment to maintain, while a |
| 1722 |
transit has two segments. Similarly, for an MP2P LSP, an LSR must |
| 1723 |
maintain one set of state information for each upstream segment |
| 1724 |
(which, we can assume, is in a one-to-one relationship with the |
| 1725 |
number of upstream neighbors) and exactly one downstream segment -- |
| 1726 |
ingresses obviously have no upstream neighbors, and egresses have no |
| 1727 |
downstream segments. |
| 1728 |
|
| 1729 |
|
| 1730 |
|
| 1731 |
|
| 1732 |
<span class="grey">Yasukawa, et al. Informational [Page 31]</span></pre> |
| 1733 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-32" ></span> |
| 1734 |
<span class="grey"><a href="./rfc5439">RFC 5439</a> Scaling in MPLS-TE February 2009</span> |
| 1735 |
|
| 1736 |
|
| 1737 |
So we can start again on our examination of the scaling properties of |
| 1738 |
MP2P LSPs using X(n) to represent the amount of LSP state held at |
| 1739 |
each P(n) node. |
| 1740 |
|
| 1741 |
<span class="h3"><a class="selflink" id="section-8.3" href="#section-8.3">8.3</a>. Scaling Improvements for Snowflake Networks</span> |
| 1742 |
|
| 1743 |
At the PEs, there is only connectivity to one other network node: the |
| 1744 |
P(2) node. But note that if P2P LSPs need to be used to allow |
| 1745 |
disambiguation of data at the MP2P LSP egresses, then these P2P LSPs |
| 1746 |
are tunneled within the MP2P LSPs. So X(PE) is: |
| 1747 |
|
| 1748 |
X(PE) = 2*(S(PE) - 1) if no disambiguation is required, |
| 1749 |
|
| 1750 |
and |
| 1751 |
|
| 1752 |
X(PE) = 4*(S(PE) - 1) if disambiguation is required. |
| 1753 |
|
| 1754 |
Each P(2) node has M(2) downstream PEs. The P(2) sees a single MP2P |
| 1755 |
LSP targeted at each downstream PE with one downstream segment (to |
| 1756 |
that PE) and M(2) - 1 upstream segments from the other subtended PEs. |
| 1757 |
Additionally, each of these LSPs has an upstream segment from the one |
| 1758 |
upstream P(1). This gives a total of M(2)*(1 + M(2)) LSP segments. |
| 1759 |
|
| 1760 |
There are also LSPs running from the subtended PEs to every other PE |
| 1761 |
in the network. There are S(PE) - M(2) such PEs, and the P(2) sees |
| 1762 |
one upstream segment for each of these from each subtended PE. It |
| 1763 |
also has one downstream segment for each of these LSPs. This gives |
| 1764 |
(M(2) + 1)*(S(PE) - M(2)) LSP segments. |
| 1765 |
|
| 1766 |
Thus: |
| 1767 |
|
| 1768 |
X(2) = M(2)*(1 + M(2)) + (M(2) + 1)*(S(PE) - M(2)) |
| 1769 |
= S(PE)*(M(2) + 1) |
| 1770 |
|
| 1771 |
Similarly, at each P(1) node there are M(1) downstream P(2) nodes and |
| 1772 |
so a total of M(1)*M(2) downstream PEs. Each P(1) is connected in a |
| 1773 |
full mesh with the other P(1) nodes and so has (S(1) - 1) neighbors. |
| 1774 |
|
| 1775 |
The P(1) sees a single MP2P LSP targeted at each downstream PE. This |
| 1776 |
has one downstream segment (to the P(2) to which the PE is connected) |
| 1777 |
and M(1) - 1 upstream segments from the other subtended P(2) nodes. |
| 1778 |
Additionally, each of these LSPs has an upstream segment from each of |
| 1779 |
the P(1) neighbors. This gives a total number of LSP segments of |
| 1780 |
M(1)*M(2)*(M(1) + S(1) - 1). |
| 1781 |
|
| 1782 |
There are also LSPs running from each of the subtended PEs to every |
| 1783 |
other PE in the network. There are S(PE) - M(1)M(2) such PEs, and |
| 1784 |
the P(1) sees one upstream segment for each of these from each |
| 1785 |
|
| 1786 |
|
| 1787 |
|
| 1788 |
<span class="grey">Yasukawa, et al. Informational [Page 32]</span></pre> |
| 1789 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-33" ></span> |
| 1790 |
<span class="grey"><a href="./rfc5439">RFC 5439</a> Scaling in MPLS-TE February 2009</span> |
| 1791 |
|
| 1792 |
|
| 1793 |
subtended P(2) (since the aggregation from the subtended PEs has |
| 1794 |
already happened at the P(2) nodes). It also has one downstream |
| 1795 |
segment to the appropriate next hop P(1) neighbor for each of these |
| 1796 |
LSPs. This gives (M(1) + 1)*(S(PE) - M(1)*M(2)) LSP segments. |
| 1797 |
|
| 1798 |
Thus: |
| 1799 |
|
| 1800 |
X(1) = M(1)*M(2)*(M(1) + S(1) - 1) + |
| 1801 |
(M(1) + 1)*(S(PE) - M(1)*M(2)) |
| 1802 |
= M(1)*M(2)*(S(1) - 2) + S(PE)*(M(1) + 1) |
| 1803 |
|
| 1804 |
So, for example, with S(1) = 10, M(1) = 10, and M(2) = 10, we see: |
| 1805 |
|
| 1806 |
S(PE) = 1000 |
| 1807 |
S(2) = 100 |
| 1808 |
X(PE) = 3996 (or 1998) |
| 1809 |
X(2) = 11000 |
| 1810 |
X(1) = 11800 |
| 1811 |
|
| 1812 |
And similarly, with S(1) = 20, M(1) = 20, and M(2) = 5, we see: |
| 1813 |
|
| 1814 |
S(PE) = 2000 |
| 1815 |
S(2) = 400 |
| 1816 |
X(PE) = 5996 (or 2998) |
| 1817 |
X(2) = 12000 |
| 1818 |
X(1) = 39800 |
| 1819 |
|
| 1820 |
<span class="h4"><a class="selflink" id="section-8.3.1" href="#section-8.3.1">8.3.1</a>. Comparison with Other Scenarios</span> |
| 1821 |
|
| 1822 |
For comparison with the examples in Sections <a href="#section-5">5</a> and <a href="#section-6">6</a>, we need to |
| 1823 |
convert those LSP-based figures to our new measure of LSP state. |
| 1824 |
|
| 1825 |
Observe that each LSP in Sections <a href="#section-5">5</a> and <a href="#section-6">6</a> generates two state units |
| 1826 |
at a transit LSR and one at an ingress or egress. So we can provide |
| 1827 |
conversions as follows: |
| 1828 |
|
| 1829 |
<a href="#section-5">Section 5</a> (flat snowflake network) |
| 1830 |
|
| 1831 |
L(PE) = 2*(S(PE) - 1) |
| 1832 |
L(2) = M(2)*(2*S(PE) - M(2) - 1) |
| 1833 |
L(1) = M(1)*M(2)*(2*S(PE) - M(2)*(M(1) + 1)) |
| 1834 |
X(PE) = 2*(S(PE) - 1) |
| 1835 |
X(2) = 2*M(2)*(2*S(PE) - M(2) - 1) |
| 1836 |
X(1) = 2*M(1)*M(2)*(2*S(PE) - M(2)*(M(1) + 1)) |
| 1837 |
|
| 1838 |
For the example with S(1) = 10, M(1) = 10, and M(2) = 10, this |
| 1839 |
gives a comparison table as follows: |
| 1840 |
|
| 1841 |
|
| 1842 |
|
| 1843 |
|
| 1844 |
<span class="grey">Yasukawa, et al. Informational [Page 33]</span></pre> |
| 1845 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-34" ></span> |
| 1846 |
<span class="grey"><a href="./rfc5439">RFC 5439</a> Scaling in MPLS-TE February 2009</span> |
| 1847 |
|
| 1848 |
|
| 1849 |
Count | Unmodified | MP2P |
| 1850 |
------+-------------+---------- |
| 1851 |
X(PE) | 1998 | 3996 |
| 1852 |
X(2) | 39780 | 11000 |
| 1853 |
X(1) | 378000 | 11800 |
| 1854 |
|
| 1855 |
Clearly, this technique is a significant improvement over the flat |
| 1856 |
network within the core of the network, although the PEs are more |
| 1857 |
heavily stressed if disambiguation is required. |
| 1858 |
|
| 1859 |
<a href="#section-6.1">Section 6.1</a> (two-layer hierarchy snowflake network) |
| 1860 |
|
| 1861 |
L(PE) = 2*(S(PE) - 1) |
| 1862 |
L(2) = M(2)*(2*S(PE) - M(2) - 1) + 2*(S(2) - 1) |
| 1863 |
L(1) = M(1)*(2*S(2) - M(1) - 1) |
| 1864 |
X(PE) = 2*(S(PE) - 1) |
| 1865 |
X(2) = 2*M(2)*(2*S(PE) - M(2) - 1) + 2*(S(2) - 1) |
| 1866 |
X(1) = 2*M(1)*(2*S(2) - M(1) - 1) |
| 1867 |
|
| 1868 |
Note that in the computation of X(2) the hierarchical LSPs only add |
| 1869 |
one state at each P(2) node. |
| 1870 |
|
| 1871 |
For the same example with S(1) = 10, M(1) = 10, and M(2) = 10, this |
| 1872 |
gives a comparison table as follows: |
| 1873 |
|
| 1874 |
Count | 2-Layer | MP2P |
| 1875 |
| Hierarchy | |
| 1876 |
------+-------------+---------- |
| 1877 |
X(PE) | 1998 | 3996 |
| 1878 |
X(2) | 39978 | 11000 |
| 1879 |
X(1) | 3780 | 11800 |
| 1880 |
|
| 1881 |
We can observe that the MP2P model is better at P(2), but the |
| 1882 |
hierarchical model is better at P(1). |
| 1883 |
|
| 1884 |
In fact, this comparison can be generalized to observe that the MP2P |
| 1885 |
model produces its best effects toward the edge of the network, while |
| 1886 |
the hierarchical model makes most impression at the core. However, |
| 1887 |
the requirement for disambiguation of P2P LSPs tunneled within the |
| 1888 |
MP2P LSPs does cause a double burden at the PEs. |
| 1889 |
|
| 1890 |
<span class="h3"><a class="selflink" id="section-8.4" href="#section-8.4">8.4</a>. Scaling Improvements for Ladder Networks</span> |
| 1891 |
|
| 1892 |
MP2P LSPs applied just within the ladder will not make a significant |
| 1893 |
difference, but applying MP2P for all LSPs and at all nodes makes a |
| 1894 |
very big difference without requiring any further configuration. |
| 1895 |
|
| 1896 |
|
| 1897 |
|
| 1898 |
|
| 1899 |
|
| 1900 |
<span class="grey">Yasukawa, et al. Informational [Page 34]</span></pre> |
| 1901 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-35" ></span> |
| 1902 |
<span class="grey"><a href="./rfc5439">RFC 5439</a> Scaling in MPLS-TE February 2009</span> |
| 1903 |
|
| 1904 |
|
| 1905 |
LSP state at a spar-node may be divided into those LSPs' segments |
| 1906 |
that enter or leave the spar-node due to subtended PEs (local LSP |
| 1907 |
segments), and those that enter or leave the spar-node due to remote |
| 1908 |
PEs (remote segments). |
| 1909 |
|
| 1910 |
The local segments may be counted as: |
| 1911 |
|
| 1912 |
o E LSPs targeting local PEs |
| 1913 |
o (S(1)-1)*E*M(1) LSPs targeting remote PEs |
| 1914 |
|
| 1915 |
The remote segments may be counted as: |
| 1916 |
|
| 1917 |
o (S(1)-1)*E outgoing LSPs targeting remote PEs |
| 1918 |
o <= 3*S(1)*E incoming LSPs targeting any PE (there are precisely |
| 1919 |
P(1) nodes attached to any other P(1) node) |
| 1920 |
|
| 1921 |
Hence, using X(1) as a measure of LSP state rather than a count of |
| 1922 |
LSPs, we get: |
| 1923 |
|
| 1924 |
X(1) <= E + (S(1)-1)*E*M(1) + (S(1)-1)*E + 3*S(1)*E |
| 1925 |
<= (4 + M(1))*S(1)*E - M(1)*E |
| 1926 |
|
| 1927 |
The number of LSPs at the P(2) nodes is also improved. We may also |
| 1928 |
count the LSP state in the same way so that there are: |
| 1929 |
|
| 1930 |
o M(2) LSPs targeting local PEs, |
| 1931 |
o M(2)*(S(1)*E) LSPs from local PEs to all other PEs, and |
| 1932 |
o S(1)*E - M(2) LSPs to remote PEs. |
| 1933 |
|
| 1934 |
So using X(2) as a measure of LSP state and not a count of LSPs, we |
| 1935 |
have: |
| 1936 |
|
| 1937 |
X(2) = M(2) + M(2)*(S(1)*E) + S(1)*E - M(2) |
| 1938 |
= (M(2) + 1)*S(1)*E |
| 1939 |
|
| 1940 |
Our examples from <a href="#section-5.2">Section 5.2</a> give us the following numbers: |
| 1941 |
|
| 1942 |
With S(1) = 6, M(1) = 10, and M(2) = 17, we see: |
| 1943 |
|
| 1944 |
E = 170 |
| 1945 |
S(PE) = 1020 |
| 1946 |
X(PE) = 2038 |
| 1947 |
X(2) = 18360 |
| 1948 |
X(1) <= 12580 |
| 1949 |
|
| 1950 |
|
| 1951 |
|
| 1952 |
|
| 1953 |
|
| 1954 |
|
| 1955 |
|
| 1956 |
<span class="grey">Yasukawa, et al. Informational [Page 35]</span></pre> |
| 1957 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-36" ></span> |
| 1958 |
<span class="grey"><a href="./rfc5439">RFC 5439</a> Scaling in MPLS-TE February 2009</span> |
| 1959 |
|
| 1960 |
|
| 1961 |
Alternatively, with S(1) = 10, M(1) = 10, and M(2) = 20, we see: |
| 1962 |
|
| 1963 |
E = 200 |
| 1964 |
S(PE) = 2000 |
| 1965 |
X(PE) = 3998 |
| 1966 |
X(2) = 42000 |
| 1967 |
X(1) <= 26000 |
| 1968 |
|
| 1969 |
<span class="h4"><a class="selflink" id="section-8.4.1" href="#section-8.4.1">8.4.1</a>. Comparison with Other Scenarios</span> |
| 1970 |
|
| 1971 |
The use of MP2P compares very favorably with all scaling scenarios. |
| 1972 |
It is the only technique able to reduce the value of X(2), and it |
| 1973 |
does this by a factor of almost two. The impact on X(1) is better |
| 1974 |
than everything except the three-level hierarchy. |
| 1975 |
|
| 1976 |
The following table provides a quick cross-reference for the figures |
| 1977 |
for the example ladder networks. Note that the previous figures are |
| 1978 |
modified to provide counts of LSP state rather than LSP numbers. |
| 1979 |
Again, each LSP contributes one state at its end points and two |
| 1980 |
states at transit nodes. |
| 1981 |
|
| 1982 |
Thus, for the all cases we have: |
| 1983 |
|
| 1984 |
X(PE) = 2*(S(PE) - 1) or |
| 1985 |
X(PE) = 4*(S(PE) - 1) if disambiguation is required. |
| 1986 |
|
| 1987 |
In the unmodified (flat) case, we have: |
| 1988 |
|
| 1989 |
X(2) = 2*(M(2)*(2*S(PE) - M(2) - 1)) |
| 1990 |
X(1) = 2*(M(1)*M(2)*(2*S(PE) - M(2)*(M(1) + 1))) |
| 1991 |
|
| 1992 |
In the two-level hierarchy, we have: |
| 1993 |
|
| 1994 |
X(2) = 2*(2*M(2)*(S(PE) - 1) - M(2)*(M(2) - 1)) |
| 1995 |
X(1) = S(1)*S(1) + 2*S(1) + 4*E*E*(S(1) - 1) - 2*E*M(2) - 2 |
| 1996 |
|
| 1997 |
In the three-level hierarchy, we have: |
| 1998 |
|
| 1999 |
X(2) = 2*(2*M(2)*(S(PE) - 1) - M(2)*(M(2) - 1)) + 2*(S(1)*M(1) - 1) |
| 2000 |
X(1) = S(1)*S(1) + 2*S(1) + 4*M(1)*M(1)*S(1) - 2*M(1)(M(1) + 1) - 2 |
| 2001 |
|
| 2002 |
Example A: S(1) = 6, M(1) = 10, and M(2) = 17 |
| 2003 |
Example B: S(1) = 10, M(1) = 10, and M(2) = 20 |
| 2004 |
|
| 2005 |
|
| 2006 |
|
| 2007 |
|
| 2008 |
|
| 2009 |
|
| 2010 |
|
| 2011 |
|
| 2012 |
<span class="grey">Yasukawa, et al. Informational [Page 36]</span></pre> |
| 2013 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-37" ></span> |
| 2014 |
<span class="grey"><a href="./rfc5439">RFC 5439</a> Scaling in MPLS-TE February 2009</span> |
| 2015 |
|
| 2016 |
|
| 2017 |
Example| Count | Unmodified | 2-Level | 3-Level | MP2P |
| 2018 |
| | | Hierarchy | Hierarchy | |
| 2019 |
-------+-------+------------+------------+-------------+------- |
| 2020 |
A | X(2) | 68748 | 68748 | 68866 | 18360 |
| 2021 |
| X(1) | 1554820 | 572266 | 2226 | 12580 |
| 2022 |
-------+-------+------------+------------+-------------+------- |
| 2023 |
B | X(2) | 159160 | 159160 | 159358 | 42000 |
| 2024 |
| X(1) | 5032000 | 1433998 | 3898 | 26000 |
| 2025 |
|
| 2026 |
<span class="h4"><a class="selflink" id="section-8.4.2" href="#section-8.4.2">8.4.2</a>. LSP State Compared with LSP Numbers</span> |
| 2027 |
|
| 2028 |
Recall that in <a href="#section-8.3">Section 8.3</a>, the true benefit of MP2P was analyzed |
| 2029 |
with respect to the LSP segment state required, rather than the |
| 2030 |
actual number of LSPs. This proved to be a more accurate comparison |
| 2031 |
of the techniques because the MP2P LSPs require state on each branch |
| 2032 |
of the LSP, so the saving is not linear with the reduced number of |
| 2033 |
LSPs. |
| 2034 |
|
| 2035 |
A similar analysis could be performed here for the ladder network. |
| 2036 |
The net effect is that it increases the state by an order of two for |
| 2037 |
all transit LSPs in the P2P models, and by a multiplier equal to the |
| 2038 |
degree of a node in the MP2P model. |
| 2039 |
|
| 2040 |
A rough estimate shows that, as with snowflake networks, MP2P |
| 2041 |
provides better scaling than the one-level hierarchical model and is |
| 2042 |
considerably better at the core. But MP2P compares less will with |
| 2043 |
the two-level hierarchy especially in the core. |
| 2044 |
|
| 2045 |
<span class="h3"><a class="selflink" id="section-8.5" href="#section-8.5">8.5</a>. Issues with MP2P LSPs</span> |
| 2046 |
|
| 2047 |
The biggest challenges for MP2P LSPs are the provision of support in |
| 2048 |
the control and data planes. To some extent, support must also be |
| 2049 |
provided in the management plane. |
| 2050 |
|
| 2051 |
Control plane support is just a matter of defining the protocols and |
| 2052 |
procedures [<a href="#ref-MP2P-RSVP" title=""Supporting Multipoint-to-Point Label Switched Paths in Multiprotocol Label Switching Traffic Engineering"">MP2P-RSVP</a>], although it must be clearly understood that |
| 2053 |
this will introduce some complexity to the control plane. |
| 2054 |
|
| 2055 |
Hardware issues may be a little more tricky. For example, the |
| 2056 |
capacity of the upstream segments must never (allowing for |
| 2057 |
statistical over-subscription) exceed the capacity of the downstream |
| 2058 |
segment. Similarly, data planes must be equipped with sufficient |
| 2059 |
buffers to handle incoming packet collisions. |
| 2060 |
|
| 2061 |
The management plane will be impacted in several ways. Firstly, the |
| 2062 |
management applications will need to handle LSPs with multiple |
| 2063 |
senders. This means that, although the applications need to process |
| 2064 |
fewer LSPs, they will be more complicated and will, in fact, need to |
| 2065 |
|
| 2066 |
|
| 2067 |
|
| 2068 |
<span class="grey">Yasukawa, et al. Informational [Page 37]</span></pre> |
| 2069 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-38" ></span> |
| 2070 |
<span class="grey"><a href="./rfc5439">RFC 5439</a> Scaling in MPLS-TE February 2009</span> |
| 2071 |
|
| 2072 |
|
| 2073 |
process the same number of ingresses and egresses. Other issues like |
| 2074 |
diagnostics and OAM would also need to be enhanced to support MP2P, |
| 2075 |
but might be borrowed heavily from LDP networks. |
| 2076 |
|
| 2077 |
Lastly, note that when the MP2P solution is used, the receiver (the |
| 2078 |
single egress PE of an MP2P tunnel) cannot use the incoming label as |
| 2079 |
an indicator of the source of the data. Contrast this with P2P LSPs. |
| 2080 |
Depending on deployment, this might not be an issue since the PE-PE |
| 2081 |
connectivity may in any case be a tunnel with inner labels to |
| 2082 |
discriminate the data flows. |
| 2083 |
|
| 2084 |
In other deployments, it may be considered necessary to include |
| 2085 |
additional PE-PE P2P LSPs and tunnel these through the MP2P LSPs. |
| 2086 |
This would require the PEs to support twice as many LSPs. Since PEs |
| 2087 |
are not usually as fully specified as P-routers, this may cause some |
| 2088 |
concern; however, the use of penultimate hop popping on the MP2P LSPs |
| 2089 |
might help to reduce this issue. |
| 2090 |
|
| 2091 |
In all cases, care must be taken not to confuse the reduction in the |
| 2092 |
number of LSPs with a reduction in the LSP state that is required. |
| 2093 |
In fact, the discussion in <a href="#section-8.3">Section 8.3</a> is slightly optimistic since |
| 2094 |
LSP state toward the destination will probably need to include sender |
| 2095 |
information and so will increase depending on the number of senders |
| 2096 |
for the MP2P LSP. <a href="#section-8.4">Section 8.4</a>, on the other hand, counts LSP state |
| 2097 |
rather than LSPs. This issue is clearly dependent on the protocol |
| 2098 |
solution for MP2P RSVP-TE, which is out of scope for this document. |
| 2099 |
|
| 2100 |
MPLS Fast Reroute (FRR) [<a href="./rfc4090" title=""Fast Reroute Extensions to RSVP-TE for LSP Tunnels"">RFC4090</a>] is an attractive scheme for |
| 2101 |
providing rapid local protection from node or link failures. Such a |
| 2102 |
scheme has, however, not been designed for MP2P at the time of |
| 2103 |
writing, so it remains to be seen how practical it could be, |
| 2104 |
especially in the case of the failure of a merge node. Initial |
| 2105 |
examination of this case suggests that FRR would not be a problem for |
| 2106 |
MP2P, given that each flow can be handled separately. |
| 2107 |
|
| 2108 |
As a final note, observe that the MP2P scenario presented in this |
| 2109 |
document may be optimistic. MP2P LSP merging may be hard to achieve |
| 2110 |
between LSPs with significantly different traffic and Quality of |
| 2111 |
Service (QoS) parameters. Therefore, it may be necessary to increase |
| 2112 |
the number of MP2P LSPs arriving at an egress. |
| 2113 |
|
| 2114 |
|
| 2115 |
|
| 2116 |
|
| 2117 |
|
| 2118 |
|
| 2119 |
|
| 2120 |
|
| 2121 |
|
| 2122 |
|
| 2123 |
|
| 2124 |
<span class="grey">Yasukawa, et al. Informational [Page 38]</span></pre> |
| 2125 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-39" ></span> |
| 2126 |
<span class="grey"><a href="./rfc5439">RFC 5439</a> Scaling in MPLS-TE February 2009</span> |
| 2127 |
|
| 2128 |
|
| 2129 |
<span class="h2"><a class="selflink" id="section-9" href="#section-9">9</a>. Combined Models</span> |
| 2130 |
|
| 2131 |
There is nothing to prevent the combination of hierarchical and MP2P |
| 2132 |
solutions within a network. |
| 2133 |
|
| 2134 |
Note that if MP2P LSPs are tunneled through P2P FA LSPs across the |
| 2135 |
core, none of the benefit of LSP merging is seen for the hops during |
| 2136 |
which the MP2P LSPs are tunneled. |
| 2137 |
|
| 2138 |
On the other hand, it is possible to construct solutions where MP2P |
| 2139 |
FA LSPs are constructed within the network, resulting in savings from |
| 2140 |
both modes of operation. |
| 2141 |
|
| 2142 |
<span class="h2"><a class="selflink" id="section-10" href="#section-10">10</a>. An Alternate Solution</span> |
| 2143 |
|
| 2144 |
A simple solution to reducing the number of LSP tunnels handled by |
| 2145 |
any node in the network has been proposed. In this solution it is |
| 2146 |
observed that part of the problem is caused purely by the total |
| 2147 |
number of LSP in the network, and that this is a function of the |
| 2148 |
number of PEs since a full mesh of PE-PE LSPs is required. The |
| 2149 |
conclusion of this observation is to move the tunnel end-points |
| 2150 |
further into the network so that, instead of having a full mesh of |
| 2151 |
PE-PE tunnels, we have only a full mesh of P(n)-P(n) tunnels. |
| 2152 |
|
| 2153 |
Obviously, there is no change in the physical network topology, so |
| 2154 |
the PEs remain subtended to the P(n) nodes, and the consequence is |
| 2155 |
that there is no TE on the links between PEs and P(n) nodes. |
| 2156 |
|
| 2157 |
In this case, we have already done the hard work for computing the |
| 2158 |
number of LSPs in the previous sections. The power of the analysis |
| 2159 |
in the earlier sections is demonstrated by its applicability to this |
| 2160 |
new model -- all we need to do is make minor changes to the formulae. |
| 2161 |
This is most simply done by removing a layer from the network. We |
| 2162 |
introduce the term "tunnel end-point" (TEP) and replace the P(n) |
| 2163 |
nodes with TEPs. Thus, the example of a flat snowflake network in |
| 2164 |
Figure 3 becomes as shown in Figure 7. Corresponding changes can be |
| 2165 |
made to all of the sample topologies. |
| 2166 |
|
| 2167 |
|
| 2168 |
|
| 2169 |
|
| 2170 |
|
| 2171 |
|
| 2172 |
|
| 2173 |
|
| 2174 |
|
| 2175 |
|
| 2176 |
|
| 2177 |
|
| 2178 |
|
| 2179 |
|
| 2180 |
<span class="grey">Yasukawa, et al. Informational [Page 39]</span></pre> |
| 2181 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-40" ></span> |
| 2182 |
<span class="grey"><a href="./rfc5439">RFC 5439</a> Scaling in MPLS-TE February 2009</span> |
| 2183 |
|
| 2184 |
|
| 2185 |
PE PE PE PE PE PE |
| 2186 |
\ \/ \/ / |
| 2187 |
PE--TEP TEP TEP TEP--PE |
| 2188 |
\ | | / |
| 2189 |
\| |/ |
| 2190 |
PE--TEP---P(1)------P(1)---TEP--PE |
| 2191 |
/ \ / \ |
| 2192 |
PE \ / PE |
| 2193 |
\/ |
| 2194 |
P(1) |
| 2195 |
/|\ |
| 2196 |
/ | \ |
| 2197 |
/ | \ |
| 2198 |
PE--TEP TEP TEP--PE |
| 2199 |
/ /\ \ |
| 2200 |
PE PE PE PE |
| 2201 |
|
| 2202 |
Figure 7 : An Example Snowflake Network with Tunnel End-Points |
| 2203 |
|
| 2204 |
To perform the scaling calculations we need only replace the PE |
| 2205 |
counts in the formulae with TEP counts, and observe that there is one |
| 2206 |
fewer layer in the network. For example, in the flat snowflake |
| 2207 |
network shown in Figure 7, we can see that the number of LSPs seen at |
| 2208 |
a TEP is: |
| 2209 |
|
| 2210 |
L(TEP) = 2*(S(TPE) - 1) |
| 2211 |
|
| 2212 |
In our sample networks, S(TPE) is typically of the order of 50 or 100 |
| 2213 |
(the original values of S(2)), so L(TEP) is less than 200, which is |
| 2214 |
quite manageable. |
| 2215 |
|
| 2216 |
Similarly, the number of LSPs handled by a P(1) node can be derived |
| 2217 |
from the original formula for the number of LSPs seen at a P(2) node, |
| 2218 |
since all we have done is reduce n in P(n) from 2 to 1. So our new |
| 2219 |
formula is: |
| 2220 |
|
| 2221 |
L(1) = M(1)*(2*S(TEP) - M(1) - 1) |
| 2222 |
|
| 2223 |
With figures for M(1) = 10 and S(TEP) = 100, this gives us L(1) = |
| 2224 |
1890. This is also very manageable. |
| 2225 |
|
| 2226 |
<span class="h3"><a class="selflink" id="section-10.1" href="#section-10.1">10.1</a>. Pros and Cons of the Alternate Solution</span> |
| 2227 |
|
| 2228 |
On the face of it, this alternate solution seems very attractive. |
| 2229 |
Simply by contracting the edges of the tunnels into the network, we |
| 2230 |
have shown a dramatic reduction in the number of tunnels needed, and |
| 2231 |
there is no requirement to apply any additional scaling techniques. |
| 2232 |
|
| 2233 |
|
| 2234 |
|
| 2235 |
|
| 2236 |
<span class="grey">Yasukawa, et al. Informational [Page 40]</span></pre> |
| 2237 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-41" ></span> |
| 2238 |
<span class="grey"><a href="./rfc5439">RFC 5439</a> Scaling in MPLS-TE February 2009</span> |
| 2239 |
|
| 2240 |
|
| 2241 |
But what of the PE-P(n) links? In the earlier sections of this |
| 2242 |
document, we have assumed that there was some requirement for PE-PE |
| 2243 |
LSPs with TE properties that extended to the PE-P(n) links at both |
| 2244 |
ends of each LSP. That means that there was a requirement to provide |
| 2245 |
reservation-based QoS on those links, to be able to discriminate |
| 2246 |
traffic flows for priority-based treatment, and to be able to |
| 2247 |
distinguish applications and sources that send data based on the LSPs |
| 2248 |
that carry the data. |
| 2249 |
|
| 2250 |
It might be argued that, since the PE-P(n) links do not offer any |
| 2251 |
routing options (each such link provides the only access to the |
| 2252 |
network for a PE), most of the benefits of tunnels are lost on these |
| 2253 |
peripheral links. However, TE is not just about routing. Just as |
| 2254 |
important are the abilities to make resource reservations, to |
| 2255 |
prioritize traffic, and to discriminate between traffic from |
| 2256 |
different applications, customers, or VPNs. |
| 2257 |
|
| 2258 |
Furthermore, in multihoming scenarios where each PE is connected to |
| 2259 |
more than one P(n) or where a PE has multiple links to a single P(n), |
| 2260 |
there may be a desire to pre-select the link to be used and to direct |
| 2261 |
the traffic to that link using a PE-PE LSP. Note that multihoming |
| 2262 |
has not been considered in this document. |
| 2263 |
|
| 2264 |
Operationally, P(n)-P(n) LSPs offer the additional management |
| 2265 |
overhead that is seen for hierarchical LSPs described in <a href="#section-6">Section 6</a>. |
| 2266 |
That is, the LSPs have to be configured and established through |
| 2267 |
additional configuration or management operations that are not |
| 2268 |
carried out at the PEs. As described in <a href="#section-6">Section 6</a>, automesh |
| 2269 |
[<a href="./rfc4972" title=""Routing Extensions for Discovery of Multiprotocol (MPLS) Label Switch Router (LSR) Traffic Engineering (TE) Mesh Membership"">RFC4972</a>] could be used to ease this task. But it must be noted |
| 2270 |
that, as mentioned above, some of the key uses of tunnels require |
| 2271 |
that traffic is classified and placed in an appropriate tunnel |
| 2272 |
according to its traffic class, end-points, originating application, |
| 2273 |
and customer (such as client VPN). This information may not be |
| 2274 |
readily available for each packet at the P(n) nodes since it is PE- |
| 2275 |
based information. Of course, it is possible to conceive of |
| 2276 |
techniques to make this information available, such as assigning a |
| 2277 |
different label for each class of traffic, but this gives rise to the |
| 2278 |
original problem of larger numbers of LSPs. |
| 2279 |
|
| 2280 |
Our conclusion is, therefore, that this alternate technique may be |
| 2281 |
suitable for the general distribution of traffic based solely on the |
| 2282 |
destination, or on a combination of the destination and key fields |
| 2283 |
carried in the IP header. In this case, it can provide a very |
| 2284 |
satisfactory answer to the scaling issues in an MPLS-TE network. But |
| 2285 |
if more sophisticated packet classification and discrimination is |
| 2286 |
required, this technique will make the desired function hard to |
| 2287 |
|
| 2288 |
|
| 2289 |
|
| 2290 |
|
| 2291 |
|
| 2292 |
<span class="grey">Yasukawa, et al. Informational [Page 41]</span></pre> |
| 2293 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-42" ></span> |
| 2294 |
<span class="grey"><a href="./rfc5439">RFC 5439</a> Scaling in MPLS-TE February 2009</span> |
| 2295 |
|
| 2296 |
|
| 2297 |
achieve, and the trade-off between scaling and feature-level will |
| 2298 |
swing too far towards solving the scaling issue at the expense of |
| 2299 |
delivery of function to the customer. |
| 2300 |
|
| 2301 |
<span class="h2"><a class="selflink" id="section-11" href="#section-11">11</a>. Management Considerations</span> |
| 2302 |
|
| 2303 |
The management issues of the models presented in this document have |
| 2304 |
been discussed in-line. No one solution is without its management |
| 2305 |
overhead. |
| 2306 |
|
| 2307 |
Note, however, that scalability of management tools is one of the |
| 2308 |
motivators for this work and that network scaling solutions that |
| 2309 |
reduce the active management of LSPs at the cost of additional effort |
| 2310 |
to manage the more static elements of the network represent a |
| 2311 |
benefit. That is, it is worth the additional effort to set up MP2P |
| 2312 |
or FA LSPs if it means that the network can be scaled to a larger |
| 2313 |
size without being constrained by the management tools. |
| 2314 |
|
| 2315 |
The MP2P technique may prove harder to debug through OAM methods than |
| 2316 |
the FA LSP approach. |
| 2317 |
|
| 2318 |
<span class="h2"><a class="selflink" id="section-12" href="#section-12">12</a>. Security Considerations</span> |
| 2319 |
|
| 2320 |
The techniques described in this document use existing or yet-to-be- |
| 2321 |
defined signaling protocol extensions and are subject to the security |
| 2322 |
provided by those extensions. Note that we are talking about |
| 2323 |
tunneling techniques used within the network and that both approaches |
| 2324 |
are vulnerable to the creation of bogus tunnels that deliver data to |
| 2325 |
an egress or consume network resources. |
| 2326 |
|
| 2327 |
The fact that the MP2P technique may prove harder to debug through |
| 2328 |
OAM methods than the FA LSP approach is a security concern since it |
| 2329 |
is important to be able to detect misconnections. |
| 2330 |
|
| 2331 |
General issues of the relationship between scaling and security are |
| 2332 |
covered in <a href="#section-1.1">Section 1.1</a>, but the details are beyond the scope of this |
| 2333 |
document. Readers are referred to [<a href="#ref-MPLS-SEC" title=""Security Framework for MPLS and GMPLS Networks"">MPLS-SEC</a>] for details of MPLS |
| 2334 |
security techniques. |
| 2335 |
|
| 2336 |
<span class="h2"><a class="selflink" id="section-13" href="#section-13">13</a>. Recommendations</span> |
| 2337 |
|
| 2338 |
The analysis in this document suggests that the ability to signal |
| 2339 |
MP2P MPLS-TE LSPs is a desirable addition to the operator's MPLS-TE |
| 2340 |
toolkit. |
| 2341 |
|
| 2342 |
At this stage, no further recommendations are made, but it would be |
| 2343 |
valuable to consult more widely to discover: |
| 2344 |
|
| 2345 |
|
| 2346 |
|
| 2347 |
|
| 2348 |
<span class="grey">Yasukawa, et al. Informational [Page 42]</span></pre> |
| 2349 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-43" ></span> |
| 2350 |
<span class="grey"><a href="./rfc5439">RFC 5439</a> Scaling in MPLS-TE February 2009</span> |
| 2351 |
|
| 2352 |
|
| 2353 |
- The concerns of other service providers with respect to network |
| 2354 |
scalability. |
| 2355 |
|
| 2356 |
- More opinions on the realistic constraints to the network |
| 2357 |
parameters listed in <a href="#section-4">Section 4</a>. |
| 2358 |
|
| 2359 |
- Desirable values for the cost-effectiveness of the network |
| 2360 |
(parameter K). |
| 2361 |
|
| 2362 |
- The applicability, manageability, and support for the two |
| 2363 |
techniques described. |
| 2364 |
|
| 2365 |
- The feasibility of combining the two techniques, as discussed in |
| 2366 |
<a href="#section-9">Section 9</a>. |
| 2367 |
|
| 2368 |
- The level of concern over the loss of functionality that would |
| 2369 |
occur if the alternate solution described in <a href="#section-10">Section 10</a> was |
| 2370 |
adopted. |
| 2371 |
|
| 2372 |
<span class="h2"><a class="selflink" id="section-14" href="#section-14">14</a>. Acknowledgements</span> |
| 2373 |
|
| 2374 |
The authors are grateful to Jean-Louis Le Roux for discussions and |
| 2375 |
review input. Thanks to Ben Niven-Jenkins, JP Vasseur, Loa |
| 2376 |
Andersson, Anders Gavler, Ben Campbell, and Tim Polk for their |
| 2377 |
comments. Thanks to Dave Allen for useful discussion of the math. |
| 2378 |
|
| 2379 |
<span class="h2"><a class="selflink" id="section-15" href="#section-15">15</a>. Normative References</span> |
| 2380 |
|
| 2381 |
[<a id="ref-RFC4206">RFC4206</a>] Kompella, K. and Y. Rekhter, "Label Switched Paths (LSP) |
| 2382 |
Hierarchy with Generalized Multi-Protocol Label Switching |
| 2383 |
(GMPLS) Traffic Engineering (TE)", <a href="./rfc4206">RFC 4206</a>, October |
| 2384 |
2005. |
| 2385 |
|
| 2386 |
<span class="h2"><a class="selflink" id="section-16" href="#section-16">16</a>. Informative References</span> |
| 2387 |
|
| 2388 |
[<a id="ref-RFC2961">RFC2961</a>] Berger, L., Gan, D., Swallow, G., Pan, P., Tommasi, F., |
| 2389 |
and S. Molendini, "RSVP Refresh Overhead Reduction |
| 2390 |
Extensions", <a href="./rfc2961">RFC 2961</a>, April 2001. |
| 2391 |
|
| 2392 |
[<a id="ref-RFC3209">RFC3209</a>] Awduche, D., Berger, L., Gan, D., Li, T., Srinivasan, V., |
| 2393 |
and G. Swallow, "RSVP-TE: Extensions to RSVP for LSP |
| 2394 |
Tunnels", <a href="./rfc3209">RFC 3209</a>, December 2001. |
| 2395 |
|
| 2396 |
[<a id="ref-RFC3270">RFC3270</a>] Le Faucheur, F., Wu, L., Davie, B., Davari, S., Vaananen, |
| 2397 |
P., Krishnan, R., Cheval, P., and J. Heinanen, "Multi- |
| 2398 |
Protocol Label Switching (MPLS) Support of Differentiated |
| 2399 |
Services", <a href="./rfc3270">RFC 3270</a>, May 2002. |
| 2400 |
|
| 2401 |
|
| 2402 |
|
| 2403 |
|
| 2404 |
<span class="grey">Yasukawa, et al. Informational [Page 43]</span></pre> |
| 2405 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-44" ></span> |
| 2406 |
<span class="grey"><a href="./rfc5439">RFC 5439</a> Scaling in MPLS-TE February 2009</span> |
| 2407 |
|
| 2408 |
|
| 2409 |
[<a id="ref-RFC3473">RFC3473</a>] Berger, L., Ed., "Generalized Multi-Protocol Label |
| 2410 |
Switching (GMPLS) Signaling Resource ReserVation |
| 2411 |
Protocol-Traffic Engineering (RSVP-TE) Extensions", <a href="./rfc3473">RFC</a> |
| 2412 |
<a href="./rfc3473">3473</a>, January 2003. |
| 2413 |
|
| 2414 |
[<a id="ref-RFC3985">RFC3985</a>] Bryant, S., Ed., and P. Pate, Ed., "Pseudo Wire Emulation |
| 2415 |
Edge-to-Edge (PWE3) Architecture", <a href="./rfc3985">RFC 3985</a>, March 2005. |
| 2416 |
|
| 2417 |
[<a id="ref-RFC4090">RFC4090</a>] Pan, P., Ed., Swallow, G., Ed., and A. Atlas, Ed., "Fast |
| 2418 |
Reroute Extensions to RSVP-TE for LSP Tunnels", <a href="./rfc4090">RFC 4090</a>, |
| 2419 |
May 2005. |
| 2420 |
|
| 2421 |
[<a id="ref-RFC4110">RFC4110</a>] Callon, R. and M. Suzuki, "A Framework for Layer 3 |
| 2422 |
Provider-Provisioned Virtual Private Networks (PPVPNs)", |
| 2423 |
<a href="./rfc4110">RFC 4110</a>, July 2005. |
| 2424 |
|
| 2425 |
[<a id="ref-RFC4972">RFC4972</a>] Vasseur, JP., Ed., Leroux, JL., Ed., Yasukawa, S., |
| 2426 |
Previdi, S., Psenak, P., and P. Mabbey, "Routing |
| 2427 |
Extensions for Discovery of Multiprotocol (MPLS) Label |
| 2428 |
Switch Router (LSR) Traffic Engineering (TE) Mesh |
| 2429 |
Membership", <a href="./rfc4972">RFC 4972</a>, July 2007. |
| 2430 |
|
| 2431 |
[<a id="ref-RFC5036">RFC5036</a>] Andersson, L., Ed., Minei, I., Ed., and B. Thomas, Ed., |
| 2432 |
"LDP Specification", <a href="./rfc5036">RFC 5036</a>, October 2007. |
| 2433 |
|
| 2434 |
[<a id="ref-MP2P-RSVP">MP2P-RSVP</a>] Yasukawa, Y., "Supporting Multipoint-to-Point Label |
| 2435 |
Switched Paths in Multiprotocol Label Switching Traffic |
| 2436 |
Engineering", Work in Progress, October 2008. |
| 2437 |
|
| 2438 |
[<a id="ref-MPLS-SEC">MPLS-SEC</a>] Fang, L., Ed., "Security Framework for MPLS and GMPLS |
| 2439 |
Networks", Work in Progress, November 2008. |
| 2440 |
|
| 2441 |
|
| 2442 |
|
| 2443 |
|
| 2444 |
|
| 2445 |
|
| 2446 |
|
| 2447 |
|
| 2448 |
|
| 2449 |
|
| 2450 |
|
| 2451 |
|
| 2452 |
|
| 2453 |
|
| 2454 |
|
| 2455 |
|
| 2456 |
|
| 2457 |
|
| 2458 |
|
| 2459 |
|
| 2460 |
<span class="grey">Yasukawa, et al. Informational [Page 44]</span></pre> |
| 2461 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-45" ></span> |
| 2462 |
<span class="grey"><a href="./rfc5439">RFC 5439</a> Scaling in MPLS-TE February 2009</span> |
| 2463 |
|
| 2464 |
|
| 2465 |
Authors' Addresses |
| 2466 |
|
| 2467 |
Seisho Yasukawa |
| 2468 |
NTT Corporation |
| 2469 |
9-11, Midori-Cho 3-Chome |
| 2470 |
Musashino-Shi, Tokyo 180-8585 Japan |
| 2471 |
Phone: +81 422 59 4769 |
| 2472 |
EMail: [email protected] |
| 2473 |
|
| 2474 |
Adrian Farrel |
| 2475 |
Old Dog Consulting |
| 2476 |
EMail: [email protected] |
| 2477 |
|
| 2478 |
Olufemi Komolafe |
| 2479 |
Cisco Systems |
| 2480 |
96 Commercial Street |
| 2481 |
Edinburgh |
| 2482 |
EH6 6LX |
| 2483 |
United Kingdom |
| 2484 |
EMail: [email protected] |
| 2485 |
|
| 2486 |
|
| 2487 |
|
| 2488 |
|
| 2489 |
|
| 2490 |
|
| 2491 |
|
| 2492 |
|
| 2493 |
|
| 2494 |
|
| 2495 |
|
| 2496 |
|
| 2497 |
|
| 2498 |
|
| 2499 |
|
| 2500 |
|
| 2501 |
|
| 2502 |
|
| 2503 |
|
| 2504 |
|
| 2505 |
|
| 2506 |
|
| 2507 |
|
| 2508 |
|
| 2509 |
|
| 2510 |
|
| 2511 |
|
| 2512 |
|
| 2513 |
|
| 2514 |
|
| 2515 |
|
| 2516 |
Yasukawa, et al. Informational [Page 45] |
| 2517 |
</pre> |