Debian Codegraph @

rfc5439.html

raw
1 <pre>Network Working Group S. Yasukawa
2 Request for Comments: 5439 NTT
3 Category: Informational A. Farrel
4 Old Dog Consulting
5 O. Komolafe
6 Cisco Systems
7 February 2009
8
9
10 <span class="h1">An Analysis of Scaling Issues in MPLS-TE Core Networks</span>
11
12 Status of This Memo
13
14 This memo provides information for the Internet community. It does
15 not specify an Internet standard of any kind. Distribution of this
16 memo is unlimited.
17
18 Copyright Notice
19
20 Copyright (c) 2009 IETF Trust and the persons identified as the
21 document authors. All rights reserved.
22
23 This document is subject to <a href="https://www.rfc-editor.org/bcp/bcp78">BCP 78</a> and the IETF Trust's Legal
24 Provisions Relating to IETF Documents (<a href="http://trustee.ietf.org/license-info">http://trustee.ietf.org/</a>
25 <a href="http://trustee.ietf.org/license-info">license-info</a>) in effect on the date of publication of this document.
26 Please review these documents carefully, as they describe your rights
27 and restrictions with respect to this document.
28
29 Abstract
30
31 Traffic engineered Multiprotocol Label Switching (MPLS-TE) is
32 deployed in providers' core networks. As providers plan to grow
33 these networks, they need to understand whether existing protocols
34 and implementations can support the network sizes that they are
35 planning.
36
37 This document presents an analysis of some of the scaling concerns
38 for the number of Label Switching Paths (LSPs) in MPLS-TE core
39 networks, and examines the value of two techniques (LSP hierarchies
40 and multipoint-to-point LSPs) for improving scaling. The intention
41 is to motivate the development of appropriate deployment techniques
42 and protocol extensions to enable the application of MPLS-TE in large
43 networks.
44
45 This document only considers the question of achieving scalability
46 for the support of point-to-point MPLS-TE LSPs. Point-to-multipoint
47 MPLS-TE LSPs are for future study.
48
49
50
51
52 <span class="grey">Yasukawa, et al. Informational [Page 1]</span></pre>
53 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-2" ></span>
54 <span class="grey"><a href="./rfc5439">RFC 5439</a> Scaling in MPLS-TE February 2009</span>
55
56
57 Table of Contents
58
59 <a href="#section-1">1</a>. Introduction ....................................................<a href="#page-3">3</a>
60 <a href="#section-1.1">1.1</a>. Overview ...................................................<a href="#page-3">3</a>
61 <a href="#section-1.2">1.2</a>. Glossary of Notation .......................................<a href="#page-5">5</a>
62 <a href="#section-2">2</a>. Issues of Concern for Scaling ...................................<a href="#page-5">5</a>
63 <a href="#section-2.1">2.1</a>. LSP State ..................................................<a href="#page-5">5</a>
64 <a href="#section-2.2">2.2</a>. Processing Overhead ........................................<a href="#page-6">6</a>
65 <a href="#section-2.3">2.3</a>. RSVP-TE Implications .......................................<a href="#page-6">6</a>
66 <a href="#section-2.4">2.4</a>. Management .................................................<a href="#page-7">7</a>
67 <a href="#section-3">3</a>. Network Topologies ..............................................<a href="#page-8">8</a>
68 <a href="#section-3.1">3.1</a>. The Snowflake Network Topology .............................<a href="#page-9">9</a>
69 <a href="#section-3.2">3.2</a>. The Ladder Network Topology ...............................<a href="#page-11">11</a>
70 <a href="#section-3.3">3.3</a>. Commercial Drivers for Selected Configurations ............<a href="#page-14">14</a>
71 <a href="#section-3.4">3.4</a>. Other Network Topologies ..................................<a href="#page-15">15</a>
72 <a href="#section-4">4</a>. Required Network Sizes .........................................<a href="#page-16">16</a>
73 <a href="#section-4.1">4.1</a>. Practical Numbers .........................................<a href="#page-16">16</a>
74 <a href="#section-5">5</a>. Scaling in Flat Networks .......................................<a href="#page-16">16</a>
75 <a href="#section-5.1">5.1</a>. Snowflake Networks ........................................<a href="#page-17">17</a>
76 <a href="#section-5.2">5.2</a>. Ladder Networks ...........................................<a href="#page-18">18</a>
77 <a href="#section-6">6</a>. Scaling Snowflake Networks with Forwarding Adjacencies .........<a href="#page-22">22</a>
78 <a href="#section-6.1">6.1</a>. Two-Layer Hierarchy .......................................<a href="#page-22">22</a>
79 6.1.1. Tuning the Network Topology to Suit the
80 Two-Layer Hierarchy ................................<a href="#page-23">23</a>
81 <a href="#section-6.2">6.2</a>. Alternative Two-Layer Hierarchy ...........................<a href="#page-24">24</a>
82 <a href="#section-6.3">6.3</a>. Three-Layer Hierarchy .....................................<a href="#page-25">25</a>
83 <a href="#section-6.4">6.4</a>. Issues with Hierarchical LSPs .............................<a href="#page-26">26</a>
84 <a href="#section-7">7</a>. Scaling Ladder Networks with Forwarding Adjacencies ............<a href="#page-27">27</a>
85 <a href="#section-7.1">7.1</a>. Two-Layer Hierarchy .......................................<a href="#page-27">27</a>
86 <a href="#section-7.2">7.2</a>. Three-Layer Hierarchy .....................................<a href="#page-28">28</a>
87 <a href="#section-7.3">7.3</a>. Issues with Hierarchical LSPs .............................<a href="#page-29">29</a>
88 <a href="#section-8">8</a>. Scaling Improvements through Multipoint-to-Point LSPs ..........<a href="#page-30">30</a>
89 <a href="#section-8.1">8.1</a>. Overview of MP2P LSPs .....................................<a href="#page-30">30</a>
90 <a href="#section-8.2">8.2</a>. LSP State: A Better Measure of Scalability ................<a href="#page-31">31</a>
91 <a href="#section-8.3">8.3</a>. Scaling Improvements for Snowflake Networks ...............<a href="#page-32">32</a>
92 <a href="#section-8.3.1">8.3.1</a>. Comparison with Other Scenarios ....................<a href="#page-33">33</a>
93 <a href="#section-8.4">8.4</a>. Scaling Improvements for Ladder Networks ..................<a href="#page-34">34</a>
94 <a href="#section-8.4.1">8.4.1</a>. Comparison with Other Scenarios ....................<a href="#page-36">36</a>
95 <a href="#section-8.4.2">8.4.2</a>. LSP State Compared with LSP Numbers ................<a href="#page-37">37</a>
96 <a href="#section-8.5">8.5</a>. Issues with MP2P LSPs .....................................<a href="#page-37">37</a>
97 <a href="#section-9">9</a>. Combined Models ................................................<a href="#page-39">39</a>
98 <a href="#section-10">10</a>. An Alternate Solution .........................................<a href="#page-39">39</a>
99 <a href="#section-10.1">10.1</a>. Pros and Cons of the Alternate Solution ..................<a href="#page-40">40</a>
100 <a href="#section-11">11</a>. Management Considerations .....................................<a href="#page-42">42</a>
101 <a href="#section-12">12</a>. Security Considerations .......................................<a href="#page-42">42</a>
102 <a href="#section-13">13</a>. Recommendations ...............................................<a href="#page-42">42</a>
103
104
105
106
107
108 <span class="grey">Yasukawa, et al. Informational [Page 2]</span></pre>
109 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-3" ></span>
110 <span class="grey"><a href="./rfc5439">RFC 5439</a> Scaling in MPLS-TE February 2009</span>
111
112
113 <a href="#section-14">14</a>. Acknowledgements ..............................................<a href="#page-43">43</a>
114 <a href="#section-15">15</a>. Normative References ..........................................<a href="#page-43">43</a>
115 <a href="#section-16">16</a>. Informative References ........................................<a href="#page-43">43</a>
116
117 <span class="h2"><a class="selflink" id="section-1" href="#section-1">1</a>. Introduction</span>
118
119 Network operators and service providers are examining scaling issues
120 as they look to deploy ever-larger traffic engineered Multiprotocol
121 Label Switching (MPLS-TE) networks. Concerns have been raised about
122 the number of Label Switched Paths (LSPs) that need to be supported
123 at the edge and at the core of the network. The impact on control
124 plane and management plane resources threatens to outweigh the
125 benefits and popularity of MPLS-TE, while the physical limitations of
126 the routers may constrain the deployment options.
127
128 Historically, it has been assumed that all MPLS-TE scaling issues can
129 be addressed using hierarchical LSP [<a href="./rfc4206" title="&quot;Label Switched Paths (LSP) Hierarchy with Generalized Multi-Protocol Label Switching (GMPLS) Traffic Engineering (TE)&quot;">RFC4206</a>]. However, analysis
130 shows that the improvement gained by LSP hierarchies is not as
131 significant in all topologies and at all points in the network as
132 might have been presumed. Further, additional management issues are
133 introduced to determine the end-points of the hierarchical LSPs and
134 to operate them. Although this does not invalidate the benefits of
135 LSP hierarchies, it does indicate that additional techniques may be
136 desirable in order to fully scale MPLS-TE networks.
137
138 This document examines the scaling properties of two generic MPLS-TE
139 network topologies and investigates the benefits of two scaling
140 techniques.
141
142 <span class="h3"><a class="selflink" id="section-1.1" href="#section-1.1">1.1</a>. Overview</span>
143
144 Physical topology scaling concerns are addressed by building networks
145 that are not fully meshed. Network topologies tend to be meshed in
146 the core but tree-shaped at the edges, giving rise to a snowflake
147 design. Alternatively, the core may be more of a ladder shape with
148 tree-shaped edges.
149
150 MPLS-TE, however, establishes a logical full mesh between all edge
151 points in the network, and this is where the scaling problems arise
152 since the structure of the network tends to focus a large number of
153 LSPs within the core of the network.
154
155 This document presents two generic network topologies (the snowflake
156 and the ladder) and attempts to parameterize the networks by making
157 some generalities. It introduces terminology for the different
158 scaling parameters and examines how many LSPs might be required to be
159 carried within the core of a network.
160
161
162
163
164 <span class="grey">Yasukawa, et al. Informational [Page 3]</span></pre>
165 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-4" ></span>
166 <span class="grey"><a href="./rfc5439">RFC 5439</a> Scaling in MPLS-TE February 2009</span>
167
168
169 Two techniques (hierarchical LSPs and multipoint-to-point LSPs) are
170 introduced and an examination is made of the scaling benefits that
171 they offer as well as of some of the concerns with using these
172 techniques.
173
174 Of necessity, this document makes many generalizations. Not least
175 among these is a set of assumptions about the symmetry and
176 connectivity of the physical network. It is hoped that these
177 generalizations will not impinge on the usefulness of the overview of
178 the scaling properties that this document attempts to give. Indeed,
179 the symmetry of the example topologies tends to highlight the scaling
180 issues of the different solution models, and this may be useful in
181 exposing the worst case scenarios.
182
183 Although protection mechanisms like Fast Reroute (FRR) [<a href="./rfc4090" title="&quot;Fast Reroute Extensions to RSVP-TE for LSP Tunnels&quot;">RFC4090</a>] are
184 briefly discussed, the main body of this document considers stable
185 network cases. It should be noted that make-before-break
186 re-optimisation after link failure may result in a significant number
187 of 'duplicate' LSPs. This issue is not addressed in this document.
188
189 It should also be understood that certain deployment models where
190 separate traffic engineered LSPs are used to provide different
191 services (such as layer 3 Virtual Private Networks (VPNs) [<a href="./rfc4110" title="&quot;A Framework for Layer 3 Provider-Provisioned Virtual Private Networks (PPVPNs)&quot;">RFC4110</a>]
192 or pseudowires [<a href="./rfc3985" title="&quot;Pseudo Wire Emulation Edge-to-Edge (PWE3) Architecture&quot;">RFC3985</a>]) or different classes of service [<a href="./rfc3270" title="&quot;Multi- Protocol Label Switching (MPLS) Support of Differentiated Services&quot;">RFC3270</a>]
193 may result in 'duplicate' or 'parallel' LSPs running between any pair
194 of provider edge nodes (PEs). This scaling factor is also not
195 considered in this document, but may be easily applied as a linear
196 factor by the reader.
197
198 The operation of security mechanisms in MPLS-TE networks [<a href="#ref-MPLS-SEC" title="&quot;Security Framework for MPLS and GMPLS Networks&quot;">MPLS-SEC</a>]
199 may have an impact on the ability of the network to scale. For
200 example, they may increase both the size and number of control plane
201 messages. Additionally, they may increase the processing overhead as
202 control plane messages are subject to processing algorithms (such as
203 encryption), and security keys need to be managed. Deployers will
204 need to consider the trade-offs between scaling objectives and
205 security objectives in their networks, and should resist the
206 temptation to respond to a degradation of scaling performance by
207 turning off security techniques that have previously been deemed as
208 necessary. Further analysis of the effects of security measures on
209 scalability are not considered further in this document.
210
211 This document is designed to help service providers discover whether
212 existing protocols and implementations can support the network sizes
213 that they are planning. To do this, it presents an analysis of some
214 of the scaling concerns for MPLS-TE core networks and examines the
215
216
217
218
219
220 <span class="grey">Yasukawa, et al. Informational [Page 4]</span></pre>
221 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-5" ></span>
222 <span class="grey"><a href="./rfc5439">RFC 5439</a> Scaling in MPLS-TE February 2009</span>
223
224
225 value of two techniques for improving scaling. This should motivate
226 the development of appropriate deployment techniques and protocol
227 extensions to enable the application of MPLS-TE in large networks.
228
229 This document only considers the question of achieving scalability
230 for the support of point-to-point MPLS-TE LSPs. Point-to-multipoint
231 MPLS-TE LSPs are for future study.
232
233 <span class="h3"><a class="selflink" id="section-1.2" href="#section-1.2">1.2</a>. Glossary of Notation</span>
234
235 This document applies consistent notation to define various
236 parameters of the networks that are analyzed. These terms are
237 defined as they are introduced throughout the document, but are
238 grouped together here for quick reference. Refer to the full
239 definitions in the text for detailed explanations.
240
241 n A network level. n = 1 is the core of the network.
242 See <a href="#section-3">Section 3</a> for more details on the definition of a level.
243 P(n) A node at level n in the network.
244 S(n) The number of nodes at level n. That is, the number of P(n)
245 nodes.
246 L(n) The number of LSPs seen by a P(n) node.
247 X(n) The number of LSP segment states held by a P(n) node.
248 M(n) The number of P(n+1) nodes subtended to a P(n) node.
249 R The number of rungs in a ladder network.
250 E The number of edge nodes (PEs) subtended below (directly or
251 indirectly) a spar-node in a ladder network.
252 K The cost-effectiveness of the network expressed in terms of
253 the ratio of the number of PEs to the number of network nodes.
254
255 <span class="h2"><a class="selflink" id="section-2" href="#section-2">2</a>. Issues of Concern for Scaling</span>
256
257 This section presents some of the issues associated with the support
258 of LSPs at a Label Switching Router (LSR) or within the network.
259 These issues may mean that there is a limit to the number of LSPs
260 that can be supported.
261
262 <span class="h3"><a class="selflink" id="section-2.1" href="#section-2.1">2.1</a>. LSP State</span>
263
264 LSP state is the data (information) that must be stored at an LSR in
265 order to maintain an LSP. Here, we refer to the information that is
266 necessary to maintain forwarding plane state and the additional
267 information required when LSPs are established through control plane
268 protocols. While the size of the LSP state is implementation-
269 dependent, it is clear that any implementation will require some data
270 in order to maintain LSP state.
271
272
273
274
275
276 <span class="grey">Yasukawa, et al. Informational [Page 5]</span></pre>
277 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-6" ></span>
278 <span class="grey"><a href="./rfc5439">RFC 5439</a> Scaling in MPLS-TE February 2009</span>
279
280
281 Thus, LSP state becomes a scaling concern because as the number of
282 LSPs at an LSR increases, so the amount of memory required to
283 maintain the LSPs increases in direct proportion. Since the memory
284 capacity of an LSR is limited, there is a related limit placed on the
285 number LSPs that can be supported.
286
287 Note that techniques to reduce the memory requirements (such as data
288 compression) may serve to increase the number of LSPs that can be
289 supported, but this will only achieve a moderate multiplier and may
290 significantly decrease the ability to process the state rapidly.
291
292 In this document, we define X(n) as "the number of LSP segment states
293 held by a P(n) node." This definition observes that an LSR at the
294 end of an LSP only has to maintain state in one direction (i.e., into
295 the network), while a transit LSR must maintain state in both
296 directions (i.e., toward both ends of the LSP). Furthermore, in
297 multipoint-to-point (MP2P) LSPs (see <a href="#section-8">Section 8</a>), a transit LSR may
298 need to maintain LSP state for one downstream segment (toward the
299 destination) and multiple upstream segments (from multiple sources).
300 That is, we define LSP segment state as the state necessary to
301 maintain an LSP in one direction to one adjacent node.
302
303 <span class="h3"><a class="selflink" id="section-2.2" href="#section-2.2">2.2</a>. Processing Overhead</span>
304
305 Depending largely on implementation issues, the number of LSPs
306 passing through an LSR may impact the processing speed for each LSP.
307 For example, control block search times can increase with the number
308 of control blocks to be searched, and even excellent implementations
309 cannot completely mitigate this fact. Thus, since CPU power is
310 constrained in any LSR, there may be a practical limit to the number
311 of LSPs that can be supported.
312
313 Further processing overhead considerations depend on issues specific
314 to the control plane protocols, and are discussed in the next
315 section.
316
317 <span class="h3"><a class="selflink" id="section-2.3" href="#section-2.3">2.3</a>. RSVP-TE Implications</span>
318
319 Like many connection-oriented signaling protocols, RSVP-TE (Resource
320 Reservation Protocol - Traffic Engineering) requires that state is
321 held within the network in order to maintain LSPs. The impact of
322 this is described in <a href="#section-2.1">Section 2.1</a>. Note that RSVP-TE requires that
323 separate information is maintained for upstream and downstream
324 relationships, but does not require any specific implementation of
325 that state.
326
327
328
329
330
331
332 <span class="grey">Yasukawa, et al. Informational [Page 6]</span></pre>
333 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-7" ></span>
334 <span class="grey"><a href="./rfc5439">RFC 5439</a> Scaling in MPLS-TE February 2009</span>
335
336
337 RSVP-TE is a soft-state protocol, which means that protocol messages
338 (refresh messages) must be regularly exchanged between signaling
339 neighbors in order to maintain the state for each LSP that runs
340 between the neighbors. A common period for the transmission (and
341 receipt) of refresh messages is 30 seconds, meaning that each LSR
342 must send and receive one message in each direction (upstream and
343 downstream) every 30 seconds for every LSP it supports. This has the
344 potential to be a significant constraint on the scaling of the
345 network, but various improvements [<a href="./rfc2961" title="&quot;RSVP Refresh Overhead Reduction Extensions&quot;">RFC2961</a>] mean that this refresh
346 processing can be significantly reduced, allowing an implementation
347 to be optimized to remove nearly all concerns about soft-state
348 scaling in a stable network.
349
350 Observations of existing implementations indicate that there may be a
351 threshold of around 50,000 LSPs above which an LSR struggles to
352 achieve sufficient processing to maintain LSP state. Although
353 refresh reduction [<a href="./rfc2961" title="&quot;RSVP Refresh Overhead Reduction Extensions&quot;">RFC2961</a>] may substantially improve this situation,
354 it has also been observed that under these circumstances the size of
355 the Srefresh may become very large, and the processing required may
356 still cause significant disruption to an LSR.
357
358 Another approach is to increase the refresh time. There is a
359 correlation between the percentage increase in refresh time and the
360 improvement in performance for the LSR. However, it should be noted
361 that RSVP-TE's soft-state nature depends on regular refresh messages;
362 thus, a degree of functionality is lost by increasing the refresh
363 time. This loss may be partially mitigated by the use of the RSVP-TE
364 Hello message, and can also be reduced by the use of various GMPLS
365 extensions [<a href="./rfc3473" title="&quot;Generalized Multi-Protocol Label Switching (GMPLS) Signaling Resource ReserVation Protocol-Traffic Engineering (RSVP-TE) Extensions&quot;">RFC3473</a>], such as the use of [<a href="./rfc2961" title="&quot;RSVP Refresh Overhead Reduction Extensions&quot;">RFC2961</a>] message
366 acknowledgements on all messages.
367
368 RSVP-TE also requires that signaling adjacencies be maintained
369 through the use of Hello message exchanges. Although [<a href="./rfc3209" title="&quot;RSVP-TE: Extensions to RSVP for LSP Tunnels&quot;">RFC3209</a>]
370 suggests that Hello messages should be retransmitted every 5 ms, in
371 practice, values of around 3 seconds are more common. Nevertheless,
372 the support of Hello messages can represent a scaling limitation on
373 an RSVP-TE implementation since one message must be sent and received
374 to/from each signaling adjacency every time period. This can impose
375 limits on the number of neighbors (physical or logical) that an LSR
376 supports, but does not impact the number of LSPs that the LSR can
377 handle.
378
379 <span class="h3"><a class="selflink" id="section-2.4" href="#section-2.4">2.4</a>. Management</span>
380
381 Another practical concern for the scalability of large MPLS-TE
382 networks is the ability to manage the network. This may be
383 constrained by the available tools, the practicality of managing
384 large numbers of LSPs, and the management protocols in use.
385
386
387
388 <span class="grey">Yasukawa, et al. Informational [Page 7]</span></pre>
389 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-8" ></span>
390 <span class="grey"><a href="./rfc5439">RFC 5439</a> Scaling in MPLS-TE February 2009</span>
391
392
393 Management tools are software implementations. Although such
394 implementations should not constrain the control plane protocols, it
395 is realistic to appreciate that network deployments will be limited
396 by the scalability of the available tools. In practice, most
397 existing tools have a limit to the number of LSPs that they can
398 support. While a Network Management System (NMS) may be able to
399 support a large number of LSPs, the number that can be supported by
400 an Element Management System (EMS) (or the number supported by an NMS
401 per-LSR) is more likely to be limited.
402
403 Similarly, practical constraints may be imposed by the operation of
404 management protocols. For example, an LSR may be swamped by
405 management protocol requests to read information about the LSPs that
406 it supports, and this might impact its ability to sustain those LSPs
407 in the control plane. OAM (Operations, Administration, and
408 Management), alarms, and notifications can further add to the burden
409 placed on an LSR and limit the number of LSPs it can support.
410
411 All of these considerations encourage a reduction in the number of
412 LSPs supported within the network and at any particular LSR.
413
414 <span class="h2"><a class="selflink" id="section-3" href="#section-3">3</a>. Network Topologies</span>
415
416 In order to provide some generic analysis of the potential scaling
417 issues for MPLS-TE networks, this document explores two network
418 topology models. These topologies are selected partly because of
419 their symmetry, which makes them more tractable to a formulaic
420 approach, and partly because they represent generalizations of real
421 deployment models. <a href="#section-3.3">Section 3.3</a> provides a discussion of the
422 commercial drivers for deployed topologies and gives more analysis of
423 why it is reasonable to consider these two topologies.
424
425 The first topology is the snowflake model. In this type of network,
426 only the very core of the network is meshed. The edges of the
427 network are formed as trees rooted in the core.
428
429 The second network topology considered is the ladder model. In this
430 type of network, the core of the network is shaped and meshed in the
431 form of a ladder and trees are attached rooted to the edge of the
432 ladder.
433
434 The sections that follow examine these topologies in detail in order
435 to parameterize them.
436
437
438
439
440
441
442
443
444 <span class="grey">Yasukawa, et al. Informational [Page 8]</span></pre>
445 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-9" ></span>
446 <span class="grey"><a href="./rfc5439">RFC 5439</a> Scaling in MPLS-TE February 2009</span>
447
448
449 <span class="h3"><a class="selflink" id="section-3.1" href="#section-3.1">3.1</a>. The Snowflake Network Topology</span>
450
451 The snowflake topologies considered in this document are based on a
452 hierarchy of connectivity within the core network. PE nodes have
453 connectivity to P-nodes as shown in Figure 1. There is no direct
454 connectivity between the PEs. Dual homing of PEs to multiple P-nodes
455 is not considered in this document, although it may be a valuable
456 addition to a network configuration.
457
458 P
459 /|\
460 / | \
461 / | \
462 / | \
463 PE PE PE
464
465 Figure 1 : PE to P-Node Connectivity
466
467 The relationship between P-nodes is also structured in a hierarchical
468 way. Thus, as shown in Figure 2, multiple P-nodes at one level are
469 connected to a P-node at a higher level. We number the levels such
470 that level 1 is the top level (top in our figure, and nearest to the
471 core of the network) and level (n) is immediately above level (n+1);
472 we denote a P-node at level n as a P(n).
473
474 As with PEs, there is no direct connectivity between P(n+1) nodes.
475 Again, dual homing of P(n+1) nodes to multiple P(n) nodes is not
476 considered in this document, although it may be a valuable addition
477 to a network configuration.
478
479 P(n)
480 /|\
481 / | \
482 / | \
483 / | \
484 P(n+1) P(n+1) P(n+1)
485
486 Figure 2 : Relationship between P-Nodes
487
488 At the top level, P(1) nodes are connected in a full mesh. In
489 reality, the level 1 part of the network may be slightly less well-
490 connected than this, but assuming a full mesh provides for
491 generality. Thus, the snowflake topology comprises a clique with
492 topologically equivalent trees subtended from each node in the
493 clique.
494
495
496
497
498
499
500 <span class="grey">Yasukawa, et al. Informational [Page 9]</span></pre>
501 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-10" ></span>
502 <span class="grey"><a href="./rfc5439">RFC 5439</a> Scaling in MPLS-TE February 2009</span>
503
504
505 The key multipliers for scalability are the number of P(1) nodes and
506 the multiplier relationship between P(n) and P(n+1) at each level,
507 down to and including PEs.
508
509 We define the multiplier M(n) as the number of P(n+1) nodes at level
510 (n+1) attached to any one P(n). Assume that M(n) is constant for all
511 nodes at level n. Since nodes at the same level are not
512 interconnected (except at the top level), and since each P(n+1) node
513 is connected to precisely one P(n) node, M(n) is one less than the
514 degree of the node at level n (that is, the P(n) node is attached to
515 M(n) nodes at level (n+1) and to 1 node at level (n-1)).
516
517 We define S(n) as the number of nodes at level (n).
518
519 Thus:
520
521 S(n) = S(1)*M(1)*M(2)*...*M(n-1)
522
523 So the number of PEs can be expressed as:
524
525 S(PE) = S(1)*M(1)*M(2)*...*M(n)
526
527 where the network has (n) layers of P-nodes.
528
529 Thus, we may depict an example snowflake network as shown in Figure
530 3. In this case:
531
532 S(1) = 3
533 M(1) = 3
534 S(2) = S(1)*M(1) = 9
535 M(2) = 2
536 S(PE) = S(1)*M(1)*M(2) = 18
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556 <span class="grey">Yasukawa, et al. Informational [Page 10]</span></pre>
557 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-11" ></span>
558 <span class="grey"><a href="./rfc5439">RFC 5439</a> Scaling in MPLS-TE February 2009</span>
559
560
561 PE PE PE PE PE PE
562 \ \/ \/ /
563 PE--P(2) P(2) P(2) P(2)--PE
564 \ | | /
565 \| |/
566 PE--P(2)---P(1)------P(1)---P(2)--PE
567 / \ / \
568 PE \ / PE
569 \/
570 P(1)
571 /|\
572 / | \
573 / | \
574 PE--P(2) P(2) P(2)--PE
575 / /\ \
576 PE PE PE PE
577
578 Figure 3 : An Example Snowflake Network
579
580 <span class="h3"><a class="selflink" id="section-3.2" href="#section-3.2">3.2</a>. The Ladder Network Topology</span>
581
582 The ladder networks considered in this section are based on an
583 arrangement of routers in the core network that resembles a ladder.
584
585 Ladder networks typically have long and thin cores that are arranged
586 as conventional ladders. That is, they have one or more spars
587 connected by rungs. Each node on a spar may have:
588
589 - connection to one or more other spars,
590 - connection to a tree of other core nodes,
591 - connection to customer nodes.
592
593 Figure 4 shows a simplified example of a ladder network. A core of
594 twelve nodes makes up two spars connected by six rungs.
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612 <span class="grey">Yasukawa, et al. Informational [Page 11]</span></pre>
613 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-12" ></span>
614 <span class="grey"><a href="./rfc5439">RFC 5439</a> Scaling in MPLS-TE February 2009</span>
615
616
617 PE PE PE PE
618 PE PE PE | PE | PE PE PE | PE | PE
619 \| \|/ |/ | \| \|/
620 PE-P-----P-----P-----P------P-----P--PE
621 | | | | | |\
622 | | | | | | PE
623 | | | | | |
624 PE-P-----P-----P-----P------P-----P
625 /| /|\ |\ |\ |\ \
626 PE PE PE | PE | PE | PE | PE PE
627 PE PE PE PE
628
629 Figure 4 : A Simplified Ladder Network
630
631 In practice, not all nodes on a spar (call them spar-nodes) need to
632 have subtended PEs. That is, they can exist simply to give
633 connectivity along the spar to other spar-nodes, or across a rung to
634 another spar. Similarly, the connectivity between spars can be more
635 complex with multiple connections from one spar-node to another spar.
636 Lastly, the network may be complicated by the inclusion of more than
637 two spars (or simplified by reduction to a single spar).
638
639 These variables make the ladder network non-trivial to model. For
640 the sake of simplicity, we will make the following restrictions:
641
642 - There are precisely two spars in the core network.
643
644 - Every spar-node connects to precisely one spar-node on the other
645 spar. That is, each spar-node is attached to precisely one rung.
646
647 - Each spar-node connects to either one (end-spar) or two (core-spar)
648 other spar-nodes on the same spar.
649
650 - Every spar-node has the same number of PEs subtended. This does
651 not mean that there are no P-nodes subtended to the spar-nodes, but
652 does mean that the edge tree subtended to each spar-node is
653 identical.
654
655 From these restrictions, we are able to quantify a ladder network as
656 follows:
657
658 R - The number of rungs. That is, the number of spar-nodes on
659 each spar.
660 S(1) - The number of spar-nodes in the network. S(1)=2*R.
661 E - The number of subtended edge nodes (PEs) to each spar-node.
662
663
664
665
666
667
668 <span class="grey">Yasukawa, et al. Informational [Page 12]</span></pre>
669 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-13" ></span>
670 <span class="grey"><a href="./rfc5439">RFC 5439</a> Scaling in MPLS-TE February 2009</span>
671
672
673 The number of rungs may vary considerably. A number less than 3 is
674 unlikely (since that would not be a significantly connected network),
675 and a number greater than 100 seems improbable (because that would
676 represent a very long, thin network).
677
678 E can be treated as for the snowflake network. That is, we can
679 consider a number of levels of attachment from P(1) nodes, which are
680 the spar-nodes, through P(i) down to P(n), which are the PEs.
681 Practically, we need to only consider n=2 (PEs attached direct to the
682 spar-nodes) and n=3 (one level of P-nodes between the PEs and the
683 spar-nodes).
684
685 Let M(i) be the ratio of P(i) nodes to P(i-1) nodes, i.e., the
686 connectivity between levels of P-node as defined for the snowflake
687 topology. Hence, the number of nodes at any level (n) is:
688
689 S(n) = S(1)*M(1)*M(2)*...*M(n-1)
690
691 So the number of PEs subtended to a spar-node is:
692
693 E = M(1)*M(2)*...*M(n)
694
695 And the number of PEs can be expressed as:
696
697 S(PE) = S(1)*M(1)*M(2)*...*M(n)
698 = S(1)*E
699
700 Thus, we may depict an example ladder network as shown in Figure 5.
701 In this case:
702
703 R = 5
704 S(1) = 10
705 M(1) = 2
706 S(2) = S(1)*M(1) = 20
707 M(2) = 2
708 E = M(1)*M(2) = 4
709 S(PE) = S(1)*E = 40
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724 <span class="grey">Yasukawa, et al. Informational [Page 13]</span></pre>
725 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-14" ></span>
726 <span class="grey"><a href="./rfc5439">RFC 5439</a> Scaling in MPLS-TE February 2009</span>
727
728
729 PE PE PE PE PE PE PE PE PE PE PE PE PE PE PE PE
730 \| \| \| \| |/ |/ |/ |/
731 P(2) P(2) P(2) P(2) P(2) P(2) P(2) P(2)
732 \ \ | \ / | / /
733 PE \ \ | \ / | / / PE
734 \ \ \| \/ |/ / /
735 PE-P(2)---P(1)---P(1)---P(1)---P(1)---P(1)---P(2)-PE
736 | | | | |
737 | | | | |
738 | | | | |
739 PE-P(2)---P(1)---P(1)---P(1)---P(1)---P(1)---P(2)-PE
740 / / / | /\ |\ \ \
741 PE / / | / \ | \ \ PE
742 / / | / \ | \ \
743 P(2) P(2) P(2) P(2) P(2) P(2) P(2) P(2)
744 /| /| /| /| |\ |\ |\ |\
745 PE PE PE PE PE PE PE PE PE PE PE PE PE PE PE PE
746
747 Figure 5 : An Example Ladder Network
748
749 <span class="h3"><a class="selflink" id="section-3.3" href="#section-3.3">3.3</a>. Commercial Drivers for Selected Configurations</span>
750
751 It is reasonable to ask why these two particular network topologies
752 have been chosen.
753
754 The most important consideration is physical scalability. Each node
755 (Label Switching Router - LSR) is only able to support a limited
756 number of physical interfaces. This necessarily reduces the ability
757 to fully mesh a network and leads to the tree-like structure of the
758 network toward the PEs.
759
760 A realistic commercial consideration for an operator is the fact that
761 the only revenue-generating nodes in the network are the PEs. Other
762 nodes are needed only to support connectivity and scalability.
763 Therefore, there is a desire to maximize S(PE) while minimizing the
764 sum of S(n) for all values of (n). This could be achieved by
765 minimizing the number of levels and maximizing the connectivity at
766 each layer, M(n). Ultimately, however, this would produce a network
767 of just interconnected PEs, which is clearly in conflict with the
768 physical scaling situation.
769
770 Therefore, the solution calls for a "few" levels with "relatively
771 large" connectivity at each level. We might say that the cost-
772 effectiveness of the network can be stated as:
773
774 K = S(PE)/(S(1)+S(2) + ... + S(n)) where n is the level above the PEs
775
776
777
778
779
780 <span class="grey">Yasukawa, et al. Informational [Page 14]</span></pre>
781 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-15" ></span>
782 <span class="grey"><a href="./rfc5439">RFC 5439</a> Scaling in MPLS-TE February 2009</span>
783
784
785 We should observe, however, that this equation may be naive in that
786 the cost of a network is not actually a function of the number of
787 routers (since a router chassis is often free or low cost), but is
788 really a function of the cost of the line cards, which is, itself, a
789 product of the capacity of the line cards. Thus, the relatively high
790 connectivity decreases the cost-effectiveness, while a topology that
791 tends to channel data through a network core tends to demand higher
792 capacity (and so, more expensive) line cards.
793
794 A further consideration is the availability of connectivity (usually
795 fibers) between LSR sites. Although it is always possible to lay new
796 fiber, this may not be cost-effective or timely. The physical shape
797 and topography of the country in which the network is laid is likely
798 to be as much of a problem. If the country is 'long and thin', then
799 a ladder network is likely to be used.
800
801 This document examines the implications for control plane and data
802 plane scalability of this type of network when MPLS-TE LSPs are used
803 to provide full connectivity between all PEs.
804
805 <span class="h3"><a class="selflink" id="section-3.4" href="#section-3.4">3.4</a>. Other Network Topologies</span>
806
807 As explained in <a href="#section-1">Section 1</a>, this document is using two symmetrical and
808 generalized network topologies for simplicity of modelling. In
809 practice, there are two other topological considerations.
810
811 a. Multihoming
812 It is relatively common for a node at level (n) to be attached to
813 more than one node at level (n-1). This is particularly common at
814 PEs that may be connected to more than one P(n).
815
816 b. Meshing within a level
817 A level in the network will often include links between P-nodes at
818 the same level, including the possibility of links between PEs.
819 This may result in a network that looks like a series of
820 concentric circles with spokes.
821
822 Both of these features are likely to have some impact on the scaling
823 of the networks. However, for the purposes of establishing the
824 ground rules for scaling, this document restricts itself to the
825 consideration of the symmetrical networks described in Sections <a href="#section-2.1">2.1</a>
826 and 2.2. Discussion of other network formats is for future study.
827
828
829
830
831
832
833
834
835
836 <span class="grey">Yasukawa, et al. Informational [Page 15]</span></pre>
837 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-16" ></span>
838 <span class="grey"><a href="./rfc5439">RFC 5439</a> Scaling in MPLS-TE February 2009</span>
839
840
841 <span class="h2"><a class="selflink" id="section-4" href="#section-4">4</a>. Required Network Sizes</span>
842
843 An important question for this evaluation and analysis is the size of
844 the network that operators require. How many PEs are required? What
845 ratio of P to PE is acceptable? How many ports do devices have for
846 physical connectivity? What type of MPLS-TE connectivity between PEs
847 is required?
848
849 Although presentation of figures for desired network sizes must be
850 treated with caution because history shows that networks grow beyond
851 all projections, it is useful to set some acceptable lower bounds.
852 That is, we can state that we are interested in networks of at least
853 a certain size.
854
855 The most important features are:
856
857 - The network should have at least 1000 PEs.
858 - Each pair of PEs should be connected by at least one LSP in each
859 direction.
860
861 <span class="h3"><a class="selflink" id="section-4.1" href="#section-4.1">4.1</a>. Practical Numbers</span>
862
863 In practice, reasonable target numbers are as follows.
864
865 S(PE) &gt;= 1000
866 Number of levels is 3. That is: 1, 2, and PE.
867 M(2) &lt;= 20
868 M(1) &lt;= 20
869 S(1) &lt;= 100
870
871 <span class="h2"><a class="selflink" id="section-5" href="#section-5">5</a>. Scaling in Flat Networks</span>
872
873 Before proceeding to examine potential scaling improvements, we need
874 to examine how well the flat networks described in the previous
875 sections scale.
876
877 Consider the requirement for a full mesh of LSPs linking all PEs.
878 That is, each PE has an LSP to and from every other LSP. Thus, if
879 there are S(PE) PEs in the network, there are S(PE)*(S(PE) - 1) LSPs.
880
881 Define L(n) as the number of LSPs handled by a level (n) LSR.
882
883 L(PE) = 2*(S(PE) - 1)
884
885
886
887
888
889
890
891
892 <span class="grey">Yasukawa, et al. Informational [Page 16]</span></pre>
893 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-17" ></span>
894 <span class="grey"><a href="./rfc5439">RFC 5439</a> Scaling in MPLS-TE February 2009</span>
895
896
897 <span class="h3"><a class="selflink" id="section-5.1" href="#section-5.1">5.1</a>. Snowflake Networks</span>
898
899 There are a total of S(PE) PEs in the network and, since each PE
900 establishes an LSP with every other PE, it would be expected that
901 there are S(PE) - 1 LSPs incoming to each PE and the same number of
902 LSPs outgoing from the same PE, giving a total of 2(S(PE) - 1) on the
903 incident link. Hence, in a snowflake topology (see Figure 3), since
904 there are M(2) PEs attached to each P(2) node, it may tempting to
905 think that L(2) (the number of LSPs traversing each P(2) node) is
906 simply 2*(S(PE) - 1)*M(2). However, it should be noted that of the
907 S(PE) - 1 LSPs incoming to each PE, M(2) - 1 originated from nodes
908 attached to the same P(2) node, and so this value would count the
909 LSPs between the M(2) PEs attached to each P(2) node twice: once when
910 outgoing from the M(2) - 1 other nodes and once when incoming into a
911 particular PE.
912
913 There are a total of M(2)*(M(2) - 1) LSPs between these M(2) PEs and,
914 since this value is erroneously included twice in 2*(S(PE) - 1)*M(2),
915 the correct value is:
916
917 L(2) = 2*M(2)*(S(PE) - 1) - M(2)*(M(2) - 1)
918 = M(2)*(2*S(PE) - M(2) - 1)
919
920 An alternative way of looking at this, that proves extensible for the
921 calculation of L(1), is to observe that each PE subtended to a P(2)
922 node has an LSP in each direction to all S(PE) - M(2) PEs in the rest
923 of the system, and there are M(2) such locally subtended PEs; thus,
924 2*M(2)*(S(PE) - M(2)). Additionally, there are M(2)*(M(2) - 1) LSPs
925 between the locally subtended PEs. So:
926
927 L(2) = 2*M(2)*(S(PE) - M(2)) + M(2)*(M(2) - 1)
928 = M(2)*(2*S(PE) - M(2) - 1)
929
930 L(1) can be computed in the same way as this second evaluation of
931 L(2). Each PE subtended below a P(1) node has an LSP in each
932 direction to all PEs not below the P(1) node. There are M(1)*M(2)
933 PEs below the P(1) node, so this accounts for 2*M(1)*M(2)*(S(PE) -
934 M(1)*M(2)) LSPs. To this, we need to add the number of LSPs that
935 pass through the P(1) node and that run between the PEs subtended
936 below the P(1). Consider each P(2): it has M(2) PEs, each of which
937 has an LSP going to all of the PEs subtended to the other P(2) nodes
938 subtended to the P(1). There are M(1) - 1 such other P(2) nodes, and
939 so M(2)*(M(1) - 1) other such PEs. So the number of LSPs from the
940 PEs below a P(2) node is M(2)*M(2)*(M(1) - 1). And there are M(1)
941 P(2) nodes below the P(1), giving rise to a total of
942 M(2)*M(2)*M(1)*(M(1) - 1) LSPs. Thus:
943
944
945
946
947
948 <span class="grey">Yasukawa, et al. Informational [Page 17]</span></pre>
949 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-18" ></span>
950 <span class="grey"><a href="./rfc5439">RFC 5439</a> Scaling in MPLS-TE February 2009</span>
951
952
953 L(1) = 2*M(1)*M(2)*(S(PE) - M(1)*M(2)) + M(2)*M(2)*M(1)*(M(1) - 1)
954 = M(1)*M(2)*(2*S(PE) - M(2)*(M(1) + 1))
955
956 So, for example, with S(1) = 5, M(1) = 10, and M(2) = 20, we see:
957
958 S(PE) = 1000
959 L(PE) = 1998
960 L(2) = 39580
961 L(1) = 356000
962
963 Alternatively, with S(1) = 10, M(1) = 10, and M(2) = 20, we see:
964
965 S(PE) = 2000
966 L(PE) = 3998
967 L(2) = 79580
968 L(1) = 756000
969
970 In both examples, the number of LSPs at the core (P(1)) nodes is
971 probably unacceptably large, even though there are only a relatively
972 modest number of PEs. In fact, L(2) may even be too large in the
973 second example.
974
975 <span class="h3"><a class="selflink" id="section-5.2" href="#section-5.2">5.2</a>. Ladder Networks</span>
976
977 In ladder networks, L(PE) remains the same at 2*(S(PE) - 1).
978
979 L(2) can be computed using the same mechanism as for the snowflake
980 topology because the subtended tree is the same format. Hence,
981
982 L(2) = 2*M(2)*(S(PE) - 1) - M(2)*(M(2) - 1)
983
984 But L(1) requires a different computation because each P(1) not only
985 sees LSPs for the subtended PEs, but is also a transit node for some
986 of the LSPs that cross the core (the core is not fully meshed).
987
988 Each P(1) sees:
989
990 o all of the LSPs between locally attached PEs,
991 o less those LSPs between locally attached PEs that can be served
992 exclusively by the attached P(2) nodes,
993 o all LSPs between locally attached PEs and remote PEs, and
994 o LSPs in transit that pass through the P(1).
995
996 The first three numbers are easily determined and match what we have
997 seen from the snowflake network. They are:
998
999
1000
1001
1002
1003
1004 <span class="grey">Yasukawa, et al. Informational [Page 18]</span></pre>
1005 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-19" ></span>
1006 <span class="grey"><a href="./rfc5439">RFC 5439</a> Scaling in MPLS-TE February 2009</span>
1007
1008
1009 o E*(E-1)
1010 o M(1)*M(2)*(M(2)-1) = E*(M(2) - 1)
1011 o 2*E*E*(S(1) - 1)
1012
1013 The number of LSPs in transit is more complicated to compute. It is
1014 simplified by not considering the ends of the ladders but by
1015 examining an arbitrary segment of the middle of the ladder, such as
1016 shown in Figure 6. We look to compute and generalize the number of
1017 LSPs traversing each core link (labeled a and b in Figure 6) and so
1018 determine the number of transit LSPs seen by each P(1).
1019
1020 : : : : : :
1021 : : : : : :
1022 P(2) P(2) P(2) P(2) P(2) P(2)
1023 \ | \ / | /
1024 \ | \ / | /
1025 \| \/ |/
1026 ......P(1)---P(1)---P(1)......
1027 | a | |
1028 | |b |
1029 | | |
1030 ......P(1)---P(1)---P(1)......
1031 /| /\ |\
1032 / | / \ | \
1033 / | / \ | \
1034 P(2) P(2) P(2) P(2) P(2) P(2)
1035 : : : : : :
1036 : : : : : :
1037
1038 Figure 6 : An Arbitrary Section of a Ladder Network
1039
1040 Of course, the number of LSPs carried on links a and b in Figure 6
1041 depends on how LSPs are routed through the core network. But if we
1042 assume a symmetrical routing policy and an even distribution of LSPs
1043 across all shortest paths, the result is the same.
1044
1045 Now we can see that each P(1) sees half of 2a+b LSPs (since each LSP
1046 would otherwise be counted twice as it passed through the P(1)),
1047 except that some of the LSPs are locally terminated and so are only
1048 included once in the sum 2a+b.
1049
1050 So L(1) = a + b/2 -
1051 (locally terminated transit LSPs)/2 +
1052 (locally contained LSPs)
1053
1054
1055
1056
1057
1058
1059
1060 <span class="grey">Yasukawa, et al. Informational [Page 19]</span></pre>
1061 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-20" ></span>
1062 <span class="grey"><a href="./rfc5439">RFC 5439</a> Scaling in MPLS-TE February 2009</span>
1063
1064
1065 Thus:
1066
1067 L(1) = a + b/2 -
1068 2*E*E*(S(1) - 1)/2 +
1069 E*(E-1) - E*(M(2) - 1)
1070 = a + b/2 +
1071 E*E*(2 - S(1)) - E*M(2)
1072
1073 So all we have to do is work out a and b.
1074
1075 Recall that the ladder length R = S(1)/2, and define X = E*E.
1076
1077 Consider the contribution made by all of the LSPs that make n hops on
1078 the ladder to the totals of each of a and b. If the ladder was
1079 unbounded, then we could say that in the case of a, there are n*2X
1080 LSPs along the spar only, and n(n-1)*2X/n = 2X(n-1) LSPs use a rung
1081 and the spar. Thus, the LSPs that make n hops on the ladder
1082 contribute (4n-2)X LSPs to a. Note that the edge cases are special
1083 because LSPs that make only one hop on the ladder cannot transit a
1084 P(1) but only start or end there.
1085
1086 So with a ladder of length R = S(1)/2, we could say:
1087
1088 R
1089 a = SUM[(4i-2)*X] + 2RX
1090 i=2
1091
1092 = 2*X*R*(R+1)
1093
1094 And similarly, considering b in an unbounded ladder, the LSPs that
1095 only travel one hop on the LSP are a special case, contributing 2X
1096 LSPs, and every other LSP that traverses n hops on the ladder
1097 contributes 2n*2X/n = 4X LSPs. So:
1098
1099 R+1
1100 b = 2X + SUM[4X]
1101 i=2
1102
1103 = 2*X + 4*X*R
1104
1105 In fact, the ladders are bounded, and so the number of LSPs is
1106 reduced because of the effect of the ends of the ladders. The links
1107 that see the most LSPs are in the middle of the ladder. Consider a
1108 ladder of length R; a node in the middle of the ladder is R/2 hops
1109 away from the end of the ladder. So we see that the formula for the
1110 contribution to the count of spar-only LSPs for a is only valid up to
1111 n=R/2, and for spar-and-rung LSPs, up to n=1+R/2. Above these
1112 limits, the contribution made by spar-only LSPs decays as (n-R/2)*2X.
1113
1114
1115
1116 <span class="grey">Yasukawa, et al. Informational [Page 20]</span></pre>
1117 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-21" ></span>
1118 <span class="grey"><a href="./rfc5439">RFC 5439</a> Scaling in MPLS-TE February 2009</span>
1119
1120
1121 However, for a first-order approximation, we will use the values of a
1122 and b as computed above. This gives us an upper bound of the number
1123 of LSPs without using a more complex formula for the reduction made
1124 by the effect of the ends of the ladder.
1125
1126 From this:
1127
1128 L(1) = a + b/2 +
1129 E*E*(2 - S(1)) - E*M(2)
1130 = 2*X*R*(R+1) +
1131 X + 2*X*R +
1132 E*E*(2 - S(1)) - E*M(2)
1133 = E*E*S(1)*(1 + S(1)/2) +
1134 E*E + E*E*S(1) +
1135 2*E+E - E*E*S(1) - E*M(2)
1136 = E*E*S(1)*(1 + S(1)/2) + 3*E+E - E*M(2)
1137 = E*E*S(1)*S(1)/2 + E*E*S(1) + 3*E*E - E*M(2)
1138
1139 So, for example, with S(1) = 6, M(1) = 10, and M(2) = 17, we see:
1140
1141 E = 170
1142 S(PE) = 1020
1143 L(PE) = 2038
1144 L(2) = 34374
1145 L(1) = 777410
1146
1147 Alternatively, with S(1) = 10, M(1) = 10, and M(2) = 20, we see:
1148
1149 E = 200
1150 S(PE) = 2000
1151 L(PE) = 3998
1152 L(2) = 79580
1153 L(1) = 2516000
1154
1155 In both examples, the number of LSPs at the core (P(1)) nodes is
1156 probably unacceptably large, even though there are only a relatively
1157 modest number of PEs. In fact, L(2) may even be too large in the
1158 second example.
1159
1160 Compare the L(1) values with the total number of LSPs in the system
1161 S(PE)*(S(PE) - 1), which is 1039380 and 3998000, respectively.
1162
1163
1164
1165
1166
1167
1168
1169
1170
1171
1172 <span class="grey">Yasukawa, et al. Informational [Page 21]</span></pre>
1173 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-22" ></span>
1174 <span class="grey"><a href="./rfc5439">RFC 5439</a> Scaling in MPLS-TE February 2009</span>
1175
1176
1177 <span class="h2"><a class="selflink" id="section-6" href="#section-6">6</a>. Scaling Snowflake Networks with Forwarding Adjacencies</span>
1178
1179 One of the purposes of LSP hierarchies [<a href="./rfc4206" title="&quot;Label Switched Paths (LSP) Hierarchy with Generalized Multi-Protocol Label Switching (GMPLS) Traffic Engineering (TE)&quot;">RFC4206</a>] is to improve the
1180 scaling properties of MPLS-TE networks. LSP tunnels (sometimes known
1181 as Forwarding Adjacencies (FAs)) may be established to provide
1182 connectivity over the core of the network, and multiple edge-to-edge
1183 LSPs may be tunneled down a single FA LSP.
1184
1185 In our network we consider a mesh of FA LSPs between all core nodes
1186 at the same level. We consider two possibilities here. In the
1187 first, all P(2) nodes are connected to all other P(2) nodes by LSP
1188 tunnels, and the PE-to-PE LSPs are tunneled across the core of the
1189 network. In the second, an extra layer of LSP hierarchy is
1190 introduced by connecting all P(1) nodes in an LSP mesh and tunneling
1191 the P(2)-to-P(2) tunnels through these.
1192
1193 <span class="h3"><a class="selflink" id="section-6.1" href="#section-6.1">6.1</a>. Two-Layer Hierarchy</span>
1194
1195 In this hierarchy model, the P(2) nodes are connected by a mesh of
1196 tunnels. This means that the P(1) nodes do not see the PE-to-PE
1197 LSPs.
1198
1199 It remains the case that:
1200
1201 L(PE) = 2*(S(PE) - 1)
1202
1203 L(2) is slightly increased. It can be computed as the sum of all
1204 LSPs for all attached PEs, including the LSPs between the attached PE
1205 (this figure is unchanged from <a href="#section-5.1">Section 5.1</a>, i.e., M(2)*(2*S(PE) -
1206 M(2) - 1)), plus the number of FA LSPs providing a mesh to the other
1207 P(2) nodes. Since the number of P(2) nodes is S(2), each P(2) node
1208 sees 2*(S(2) - 1) FA LSPs. Thus:
1209
1210 L(2) = M(2)*(2*S(PE) - M(2) - 1) + 2*(S(2) - 1)
1211
1212 L(1), however, is significantly reduced and can be computed as the
1213 sum of the number of FA LSPs to and from each attached P(2) to each
1214 other P(2) in the network, including (but counting only once) the FA
1215 LSPs between attached P(2) nodes. In fact, the problem is identical
1216 to the L(2) computation in <a href="#section-5.1">Section 5.1</a>. So:
1217
1218 L(1) = M(1)*(2*S(2) - M(1) - 1)
1219
1220
1221
1222
1223
1224
1225
1226
1227
1228 <span class="grey">Yasukawa, et al. Informational [Page 22]</span></pre>
1229 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-23" ></span>
1230 <span class="grey"><a href="./rfc5439">RFC 5439</a> Scaling in MPLS-TE February 2009</span>
1231
1232
1233 So, for example, with S(1) = 5, M(1) = 10, and M(2) = 20, we see:
1234
1235 S(PE) = 1000
1236 S(2) = 50
1237 L(PE) = 1998
1238 L(2) = 39678
1239 L(1) = 890
1240
1241 Alternatively, with S(1) = 10, M(1) = 10, and M(2) = 20, we see:
1242
1243 S(PE) = 2000
1244 S(2) = 100
1245 L(PE) = 3998
1246 L(2) = 79778
1247 L(1) = 1890
1248
1249 So, in both examples, potential problems at the core (P(1)) nodes
1250 caused by an excessive number of LSPs can be avoided, but any problem
1251 with L(2) is made slightly worse, as can be seen from the table
1252 below.
1253
1254 Example| Count | Unmodified | 2-Layer
1255 | | (<a href="#section-5.1">Section 5.1</a>) | Hierarchy
1256 -------+-------+---------------+----------
1257 A | L(2) | 39580 | 39678
1258 | L(1) | 356000 | 890
1259 -------+-------+---------------+----------
1260 B | L(2) | 79580 | 79778
1261 | L(1) | 756000 | 1890
1262
1263 <span class="h4"><a class="selflink" id="section-6.1.1" href="#section-6.1.1">6.1.1</a>. Tuning the Network Topology to Suit the Two-Layer Hierarchy</span>
1264
1265 Clearly, we can reduce L(2) by selecting appropriate values of S(1),
1266 M(1), and M(2). We can do this without negative consequences, since
1267 no change will affect L(PE) and since a large percentage increase in
1268 L(1) is sustainable now that L(1) is so small.
1269
1270 Observe that:
1271
1272 L(2) = M(2)*(2*S(PE) - M(2) - 1) + 2*(S(2) - 1)
1273
1274 where S(PE) = S(1)*M(1)*M(2) and S(2) = S(1)*M(1). So L(2) scales
1275 with M(2)^2 and we can have the most impact by reducing M(2) while
1276 keeping S(PE) constant.
1277
1278
1279
1280
1281
1282
1283
1284 <span class="grey">Yasukawa, et al. Informational [Page 23]</span></pre>
1285 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-24" ></span>
1286 <span class="grey"><a href="./rfc5439">RFC 5439</a> Scaling in MPLS-TE February 2009</span>
1287
1288
1289 For example, with S(1) = 10, M(1) = 10, and M(2) = 10, we see:
1290
1291 S(PE) = 1000
1292 S(2) = 100
1293 L(PE) = 1998
1294 L(2) = 20088
1295 L(1) = 1890
1296
1297 And similarly, with S(1) = 20, M(1) = 20, and M(2) = 5, we see:
1298
1299 S(PE) = 2000
1300 S(2) = 400
1301 L(PE) = 3998
1302 L(2) = 20768
1303 L(1) = 15580
1304
1305 These considerable scaling benefits must be offset against the cost-
1306 effectiveness of the network. Recall from <a href="#section-3.3">Section 3.3</a> that:
1307
1308 K = S(PE)/(S(1)+S(2) ... + S(n))
1309
1310 where n is the level above the PEs, so that for our network:
1311
1312 K = S(PE) / (S(1) + S(2))
1313
1314 Thus, in the first example the cost-effectiveness has been halved
1315 from 1000/55 to 1000/110. In the second example, it has been reduced
1316 to roughly one quarter, changing from 2000/110 to 2000/420.
1317
1318 So, although the tuning changes may be necessary to reach the desired
1319 network size, they come at a considerable cost to the operator.
1320
1321 <span class="h3"><a class="selflink" id="section-6.2" href="#section-6.2">6.2</a>. Alternative Two-Layer Hierarchy</span>
1322
1323 An alternative to the two-layer hierarchy presented in <a href="#section-6.1">Section 6.1</a> is
1324 to provide a full mesh of FA LSPs between P(1) nodes. This technique
1325 is only of benefit to any nodes in the core of the level 1 network.
1326 It makes no difference to the PE and P(2) nodes since they continue
1327 to see only the PE-to-PE LSPs. Furthermore, this approach increases
1328 the burden at the P(1) nodes since they have to support all of the
1329 PE-to-PE LSPs as in the flat model plus the additional 2*(S(1) - 1)
1330 P(1)-to-P(1) FA LSPs. Thus, this approach should only be considered
1331 where there is a mesh of P-nodes within the ring of P(1) nodes, and
1332 is not considered further in this document.
1333
1334
1335
1336
1337
1338
1339
1340 <span class="grey">Yasukawa, et al. Informational [Page 24]</span></pre>
1341 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-25" ></span>
1342 <span class="grey"><a href="./rfc5439">RFC 5439</a> Scaling in MPLS-TE February 2009</span>
1343
1344
1345 <span class="h3"><a class="selflink" id="section-6.3" href="#section-6.3">6.3</a>. Three-Layer Hierarchy</span>
1346
1347 As demonstrated by <a href="#section-6.2">Section 6.2</a>, introducing a mesh of FA LSPs at the
1348 top level (P(1)) has no benefit, but if we introduce an additional
1349 level in the network (P(3) between P(2) and PE) to make a four-level
1350 snowflake, we can introduce a new layer of FA LSPs so that we have a
1351 full mesh of FA LSPs between all P(3) nodes to carry the PE-to-PE
1352 LSPs, and a full mesh of FA LSPs between all P(2) nodes to carry the
1353 P(3)-to-P(3) LSPs.
1354
1355 The number of PEs is S(PE) = S(1)*M(1)*M(2)*M(3), and the number of
1356 PE-to-PE LSPs at a PE remains L(PE) = 2*(S(PE) - 1).
1357
1358 The number of LSPs at a P(3) can be deduced from <a href="#section-6.1">Section 6.1</a>. It is
1359 the sum of all LSPs for all attached PEs, including the LSPs between
1360 the attached PE, plus the number of FA LSPs providing a mesh to the
1361 other P(3) nodes.
1362
1363 L(3) = M(3)*(2*S(PE) - M(3) - 1) + 2*(S(3) - 1)
1364
1365 The number of LSPs at P(2) can also be deduced from <a href="#section-6.1">Section 6.1</a> since
1366 it is the sum of all LSPs for all attached P(3) nodes, including the
1367 LSPs between the attached PE plus the number of FA LSPs providing a
1368 mesh to the other P(2) nodes.
1369
1370 L(2) = M(2)*(2*S(3) - M(2) - 1) + 2*(S(2) - 1)
1371
1372 Finally, L(1) can be copied straight from 6.1.
1373
1374 L(1) = M(1)*(2*S(2) - M(1) - 1)
1375
1376 For example, with S(1) = 5, M(1) = 5, M(2) = 5, and M(3) = 8, we see:
1377
1378 S(PE) = 1000
1379 S(3) = 125
1380 S(2) = 25
1381 L(PE) = 1998
1382 L(3) = 16176
1383 L(2) = 1268
1384 L(1) = 220
1385
1386
1387
1388
1389
1390
1391
1392
1393
1394
1395
1396 <span class="grey">Yasukawa, et al. Informational [Page 25]</span></pre>
1397 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-26" ></span>
1398 <span class="grey"><a href="./rfc5439">RFC 5439</a> Scaling in MPLS-TE February 2009</span>
1399
1400
1401 Similarly, with S(1) = 5, M(1) = 5, M(2) = 8, and M(3) = 10, we see:
1402
1403 S(PE) = 2000
1404 S(3) = 200
1405 S(2) = 25
1406 L(PE) = 3998
1407 L(3) = 40038
1408 L(2) = 3184
1409 L(1) = 220
1410
1411 Clearly, there are considerable scaling improvements with this three-
1412 layer hierarchy, and all of the numbers (even L(3) in the second
1413 example) are manageable.
1414
1415 Of course, the extra level in the network tends to reduce the cost-
1416 effectiveness of the networks with values of K = 1000/155 and K =
1417 2000/230 (from 1000/55 and 2000/110) for the examples above. That is
1418 a reduction by a factor of 3 in the first case and 2 in the second
1419 case. Such a change in cost-effectiveness has to be weighed against
1420 the desire to deploy such a large network. If LSP hierarchies are
1421 the only scaling tool available, and networks this size are required,
1422 the cost-effectiveness may need to be sacrificed.
1423
1424 <span class="h3"><a class="selflink" id="section-6.4" href="#section-6.4">6.4</a>. Issues with Hierarchical LSPs</span>
1425
1426 A basic observation for hierarchical scaling techniques is that it is
1427 hard to have any impact on the number of LSPs that must be supported
1428 by the level of P(n) nodes adjacent to the PEs (for example, it is
1429 hard to reduce L(3) in <a href="#section-6.3">Section 6.3</a>). In fact, the only way we can
1430 change the number of LSPs supported by these nodes is to change the
1431 scaling ratio M(n) in the network -- in other words, to change the
1432 number of PEs subtended to any P(n). But such a change has a direct
1433 effect on the number of PEs in the network and so the cost-
1434 effectiveness is impacted.
1435
1436 Another concern with the hierarchical approach is that it must be
1437 configured and managed. This may not seem like a large burden, but
1438 it must be recalled that the P(n) nodes are not at the edge of the
1439 network -- they are a set of nodes that must be identified so that
1440 the FA LSPs can be configured and provisioned. Effectively, the
1441 operator must plan and construct a layered network with a ring of
1442 P(n) nodes giving access to the level (n) network. This design
1443 activity is open to considerable risk as failing to close the ring
1444 (i.e., allowing a node to be at both level (n+1) and at level (n))
1445 may cause operational confusion.
1446
1447
1448
1449
1450
1451
1452 <span class="grey">Yasukawa, et al. Informational [Page 26]</span></pre>
1453 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-27" ></span>
1454 <span class="grey"><a href="./rfc5439">RFC 5439</a> Scaling in MPLS-TE February 2009</span>
1455
1456
1457 Protocol techniques (such as IGP automesh [<a href="./rfc4972" title="&quot;Routing Extensions for Discovery of Multiprotocol (MPLS) Label Switch Router (LSR) Traffic Engineering (TE) Mesh Membership&quot;">RFC4972</a>]) have been
1458 developed to reduce the configuration necessary to build this type of
1459 multi-level network. In the case of automesh, the routing protocol
1460 is used to advertise the membership of a 'mesh group', and all
1461 members of the mesh group can discover each other and connect with
1462 LSP tunnels. Thus, the P(n) nodes giving access to level (n) can
1463 advertise their existence to each other, and it is not necessary to
1464 configure each with information about all of the others. Although
1465 this process can help to reduce the configuration overhead, it does
1466 not eliminate it, as each member of the mesh group must still be
1467 planned and configured for membership.
1468
1469 An important consideration for the use of hierarchical LSPs is how
1470 they can be protected using MPLS Fast Reroute (FRR) [<a href="./rfc4090" title="&quot;Fast Reroute Extensions to RSVP-TE for LSP Tunnels&quot;">RFC4090</a>]. FRR
1471 may provide link protection either by protecting the tunnels as they
1472 traverse a broken link or by treating each level (n) tunnel LSP as a
1473 link in level (n+1) and providing protection for the level (n+1) LSPs
1474 (although in this model, fault detection and propagation time may be
1475 an issue). Node protection may be performed in a similar way, but
1476 protection of the first and last nodes of a hierarchical LSP is
1477 particularly difficult. Additionally, the whole notion of scaling
1478 with regard to FRR gives rise to separate concerns that are outside
1479 the scope of this document as currently formulated.
1480
1481 Finally, observe that we have been explaining these techniques using
1482 conveniently symmetrical networks. Consider how we would arrange the
1483 hierarchical LSPs in a network where some PEs are connected closer to
1484 the center of the network than others.
1485
1486 <span class="h2"><a class="selflink" id="section-7" href="#section-7">7</a>. Scaling Ladder Networks with Forwarding Adjacencies</span>
1487
1488 <span class="h3"><a class="selflink" id="section-7.1" href="#section-7.1">7.1</a>. Two-Layer Hierarchy</span>
1489
1490 In <a href="#section-6.2">Section 6.2</a>, we observed that there is no value to placing FA LSPs
1491 between the P(1) nodes of our example snowflake topologies. This is
1492 because those LSPs would be just one hop long and would, in fact,
1493 only serve to increase the burden at the P(1) nodes. However, in the
1494 ladder model, there is value to this approach. The P(1) nodes are
1495 the spar-nodes of the ladder, and they are not all mutually adjacent.
1496 That is, the P(1)-to-P(1) hierarchical LSPs can create a full mesh of
1497 P(1) nodes where one does not exist in the physical topology.
1498
1499 The number of LSPs seen by a P(1) node is then:
1500
1501 o all of the tunnels terminating at the P(1) node,
1502 o any transit tunnels, and
1503 o all of the LSPs due to subtended PEs.
1504
1505
1506
1507
1508 <span class="grey">Yasukawa, et al. Informational [Page 27]</span></pre>
1509 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-28" ></span>
1510 <span class="grey"><a href="./rfc5439">RFC 5439</a> Scaling in MPLS-TE February 2009</span>
1511
1512
1513 This is a substantial reduction; all of the transit LSPs are reduced
1514 to just one per remote P(1) that causes any transit LSP. So:
1515
1516 L(1) = 2*(S(1) - 1) +
1517 O(S(1)*S(1)/2) +
1518 2*E*E*(S(1) - 1) + E*(E-1) - E*(M(2) - 1)
1519
1520 where O(S(1)*S(1)/2) gives an upper bound order of magnitude. So:
1521
1522 L(1) = S(1)*S(1)/2 + 2*S(1) + 2*E*E*(S(1) - 1) - E*M(2) - 2
1523
1524 So, in our two examples:
1525
1526 With S(1) = 6, M(1) = 10, and M(2) = 17, we see:
1527
1528 E = 170
1529 S(PE) = 1020
1530 L(PE) = 2038
1531 L(2) = 34374
1532 L(1) = 286138
1533
1534 Alternatively, with S(1) = 10, M(1) = 10, and M(2) = 20, we see:
1535
1536 E = 200
1537 S(PE) = 2000
1538 L(PE) = 3998
1539 L(2) = 79580
1540 L(1) = 716060
1541
1542 Both of these show significant improvements over the previous L(1)
1543 figures of 777410 and 2516000. But the numbers are still too large
1544 to manage, and there is no improvement in the L(2) figures.
1545
1546 <span class="h3"><a class="selflink" id="section-7.2" href="#section-7.2">7.2</a>. Three-Layer Hierarchy</span>
1547
1548 We can also apply the three-layer hierarchy to the ladder model. In
1549 this case, the number of LSPs between P(1) nodes is not reduced, but
1550 tunnels are also set up between all P(2) nodes. Thus, the number of
1551 LSPs seen by a P(1) node is:
1552
1553 o all of the tunnels terminating at the P(1) node,
1554 o any transit tunnels between P(1) nodes, and
1555 o all of the LSPs due to subtended P(2) nodes.
1556
1557 No PE-to-PE LSPs are seen at the P(1) nodes.
1558
1559
1560
1561
1562
1563
1564 <span class="grey">Yasukawa, et al. Informational [Page 28]</span></pre>
1565 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-29" ></span>
1566 <span class="grey"><a href="./rfc5439">RFC 5439</a> Scaling in MPLS-TE February 2009</span>
1567
1568
1569 L(1) = 2*(S(1) - 1) +
1570 O(S(1)*S(1)/2) +
1571 2*(S(1) - 1)*M(1)*M(1) + M(1)*(M(1) - 1)
1572
1573 where O(S(1)*S(1)/2) gives an upper bound order of magnitude. So:
1574
1575 L(1) = S(1)*S(1)/2 + 2*S(1) + 2*M(1)*M(1)*S(1) - M(1)(M(1) + 1) - 2
1576
1577 Unfortunately, there is a small increase in the number of LSPs seen
1578 by the P(2) nodes. Each P(2) now sees all of the PE-to-PE LSPs that
1579 it saw before and is also an end-point for a set of P(2)-to-P(2)
1580 tunnels. Thus, L(2) increases to:
1581
1582 L(2) = 2*M(2)*(S(PE) - 1) - M(2)*(M(2) - 1) + 2*(S(1)*M(1) - 1)
1583
1584 So, in our two examples:
1585
1586 With S(1) = 6, M(1) = 10, and M(2) = 17, we see:
1587
1588 E = 170
1589 S(PE) = 1020
1590 L(PE) = 2038
1591 L(2) = 34492
1592 L(1) = 1118
1593
1594 Alternatively, with S(1) = 10, M(1) = 10, and M(2) = 20, we see:
1595
1596 E = 200
1597 S(PE) = 2000
1598 L(PE) = 3998
1599 L(2) = 79778
1600 L(1) = 1958
1601
1602 This represents a very dramatic decrease in LSPs across the core.
1603
1604 <span class="h3"><a class="selflink" id="section-7.3" href="#section-7.3">7.3</a>. Issues with Hierarchical LSPs</span>
1605
1606 The same issues exist for hierarchical LSPs as described in <a href="#section-6.4">Section</a>
1607 <a href="#section-6.4">6.4</a>. Although dramatic improvements can be made to the scaling
1608 numbers for the number of LSPs at core nodes, this can only be done
1609 at the cost of configuring P(2) to P(2) tunnels. The mesh of P(1)
1610 tunnels is not enough.
1611
1612 But the sheer number of P(2) to P(2) tunnels that must be configured
1613 is a significant management burden that can only be eased by using a
1614 technique like automesh [<a href="./rfc4972" title="&quot;Routing Extensions for Discovery of Multiprotocol (MPLS) Label Switch Router (LSR) Traffic Engineering (TE) Mesh Membership&quot;">RFC4972</a>].
1615
1616
1617
1618
1619
1620 <span class="grey">Yasukawa, et al. Informational [Page 29]</span></pre>
1621 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-30" ></span>
1622 <span class="grey"><a href="./rfc5439">RFC 5439</a> Scaling in MPLS-TE February 2009</span>
1623
1624
1625 It is significant, however, that the scaling problem at the P(2)
1626 nodes cannot be improved by using tunnels and that the only solution
1627 to ease this in the hierarchical approach would be to institute
1628 another layer of hierarchy (that is, P(3) nodes) between the P(2)
1629 nodes and the PEs. This is, of course, a significant expense.
1630
1631 <span class="h2"><a class="selflink" id="section-8" href="#section-8">8</a>. Scaling Improvements through Multipoint-to-Point LSPs</span>
1632
1633 An alternative (or complementary) scaling technique has been proposed
1634 using multipoint-to-point (MP2P) LSPs. The fundamental improvement
1635 in this case is achieved by reducing the number of LSPs toward the
1636 destination as LSPs toward the same destination are merged.
1637
1638 This section presents an overview of MP2P LSPs and describes their
1639 applicability and scaling benefits.
1640
1641 <span class="h3"><a class="selflink" id="section-8.1" href="#section-8.1">8.1</a>. Overview of MP2P LSPs</span>
1642
1643 Note that the MP2P LSPs discussed here are for MPLS-TE and are not
1644 the same concept familiar in the Label Distribution Protocol (LDP)
1645 described in [<a href="./rfc5036" title="&quot;LDP Specification&quot;">RFC5036</a>].
1646
1647 Traffic flows generally converge toward their destination and this
1648 can be utilized by MPLS in constructing an MP2P LSP. With such an
1649 LSP, the Label Forwarding Information Base (LFIB) mappings at each
1650 LSR are many-to-one so that multiple pairs {incoming interface,
1651 incoming label} are mapped to a single pair {outgoing interface,
1652 outgoing label}. Obviously, if per-platform labels are used, this
1653 mapping may be optimized within an implementation.
1654
1655 It is important to note that with MP2P MPLS-TE LSPs, the traffic
1656 flows are merged. That is, some additional form of identifier is
1657 required if de-merging is required. For example, if the payload is
1658 IP traffic belonging to the same client network, no additional de-
1659 merging information is required since the IP packet contains
1660 sufficient data. On the other hand, if the data comes, for example,
1661 from a variety of VPN client networks, then the flows will need to be
1662 labeled in their own right as point-to-point (P2P) flows, so that
1663 traffic can be disambiguated at the egress of the MP2P LSPs.
1664
1665 Techniques for establishing MP2P MPLS-TE LSPs and for assigning the
1666 correct bandwidth downstream of LSP merge points are out of the scope
1667 of this document.
1668
1669
1670
1671
1672
1673
1674
1675
1676 <span class="grey">Yasukawa, et al. Informational [Page 30]</span></pre>
1677 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-31" ></span>
1678 <span class="grey"><a href="./rfc5439">RFC 5439</a> Scaling in MPLS-TE February 2009</span>
1679
1680
1681 <span class="h3"><a class="selflink" id="section-8.2" href="#section-8.2">8.2</a>. LSP State: A Better Measure of Scalability</span>
1682
1683 Consider the network topology shown in Figure 3. Suppose that we
1684 establish MP2P LSP tunnels such that there is one tunnel terminating
1685 at each PE, and that that tunnel has every other PE as an ingress.
1686 Thus, a PE-to-PE MP2P LSP tunnel would have S(PE)-1 ingresses and one
1687 egress, and there would be S(PE) such tunnels.
1688
1689 Note that there still remain 2*(S(PE) - 1) PE-to-PE P2P LSPs that are
1690 carried through these tunnels.
1691
1692 Let's consider the number of LSPs handled at each node in the
1693 network.
1694
1695 The PEs continue to handle the same number of PE-to-PE P2P LSPs, and
1696 must also handle the MP2P LSPs. So:
1697
1698 L(PE) = 2*(S(PE) - 1) + S(PE)
1699
1700 But all P(n) nodes in the network only handle the MP2P LSP tunnels.
1701 Nominally, this means that L(n) = S(PE) for all values of n. This
1702 would appear to be a great success with the number of LSPs cut to
1703 completely manageable levels.
1704
1705 However, the number of LSPs is not the only issue (although it may
1706 have some impact for some of the scaling concerns listed in <a href="#section-4">Section</a>
1707 <a href="#section-4">4</a>). We are more interested in the amount of LSP state that is
1708 maintained by an LSR. This reflects the amount of storage required
1709 at the LSR, the amount of protocol processing, and the amount of
1710 information that needs to be managed.
1711
1712 In fact, we were also interested in this measure of scalability in
1713 the earlier sections of this document, but in those cases we could
1714 see a direct correlation between the number of LSPs and the amount of
1715 LSP state since transit LSPs had two pieces of state information (one
1716 on the incoming and one on the outgoing interface), and ingress or
1717 egress LSPs had just one piece of state.
1718
1719 We can quantify the amount of LSP state according to the number of
1720 LSP segments managed by an LSR. So (as above), in the case of a P2P
1721 LSP, an ingress or egress has one segment to maintain, while a
1722 transit has two segments. Similarly, for an MP2P LSP, an LSR must
1723 maintain one set of state information for each upstream segment
1724 (which, we can assume, is in a one-to-one relationship with the
1725 number of upstream neighbors) and exactly one downstream segment --
1726 ingresses obviously have no upstream neighbors, and egresses have no
1727 downstream segments.
1728
1729
1730
1731
1732 <span class="grey">Yasukawa, et al. Informational [Page 31]</span></pre>
1733 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-32" ></span>
1734 <span class="grey"><a href="./rfc5439">RFC 5439</a> Scaling in MPLS-TE February 2009</span>
1735
1736
1737 So we can start again on our examination of the scaling properties of
1738 MP2P LSPs using X(n) to represent the amount of LSP state held at
1739 each P(n) node.
1740
1741 <span class="h3"><a class="selflink" id="section-8.3" href="#section-8.3">8.3</a>. Scaling Improvements for Snowflake Networks</span>
1742
1743 At the PEs, there is only connectivity to one other network node: the
1744 P(2) node. But note that if P2P LSPs need to be used to allow
1745 disambiguation of data at the MP2P LSP egresses, then these P2P LSPs
1746 are tunneled within the MP2P LSPs. So X(PE) is:
1747
1748 X(PE) = 2*(S(PE) - 1) if no disambiguation is required,
1749
1750 and
1751
1752 X(PE) = 4*(S(PE) - 1) if disambiguation is required.
1753
1754 Each P(2) node has M(2) downstream PEs. The P(2) sees a single MP2P
1755 LSP targeted at each downstream PE with one downstream segment (to
1756 that PE) and M(2) - 1 upstream segments from the other subtended PEs.
1757 Additionally, each of these LSPs has an upstream segment from the one
1758 upstream P(1). This gives a total of M(2)*(1 + M(2)) LSP segments.
1759
1760 There are also LSPs running from the subtended PEs to every other PE
1761 in the network. There are S(PE) - M(2) such PEs, and the P(2) sees
1762 one upstream segment for each of these from each subtended PE. It
1763 also has one downstream segment for each of these LSPs. This gives
1764 (M(2) + 1)*(S(PE) - M(2)) LSP segments.
1765
1766 Thus:
1767
1768 X(2) = M(2)*(1 + M(2)) + (M(2) + 1)*(S(PE) - M(2))
1769 = S(PE)*(M(2) + 1)
1770
1771 Similarly, at each P(1) node there are M(1) downstream P(2) nodes and
1772 so a total of M(1)*M(2) downstream PEs. Each P(1) is connected in a
1773 full mesh with the other P(1) nodes and so has (S(1) - 1) neighbors.
1774
1775 The P(1) sees a single MP2P LSP targeted at each downstream PE. This
1776 has one downstream segment (to the P(2) to which the PE is connected)
1777 and M(1) - 1 upstream segments from the other subtended P(2) nodes.
1778 Additionally, each of these LSPs has an upstream segment from each of
1779 the P(1) neighbors. This gives a total number of LSP segments of
1780 M(1)*M(2)*(M(1) + S(1) - 1).
1781
1782 There are also LSPs running from each of the subtended PEs to every
1783 other PE in the network. There are S(PE) - M(1)M(2) such PEs, and
1784 the P(1) sees one upstream segment for each of these from each
1785
1786
1787
1788 <span class="grey">Yasukawa, et al. Informational [Page 32]</span></pre>
1789 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-33" ></span>
1790 <span class="grey"><a href="./rfc5439">RFC 5439</a> Scaling in MPLS-TE February 2009</span>
1791
1792
1793 subtended P(2) (since the aggregation from the subtended PEs has
1794 already happened at the P(2) nodes). It also has one downstream
1795 segment to the appropriate next hop P(1) neighbor for each of these
1796 LSPs. This gives (M(1) + 1)*(S(PE) - M(1)*M(2)) LSP segments.
1797
1798 Thus:
1799
1800 X(1) = M(1)*M(2)*(M(1) + S(1) - 1) +
1801 (M(1) + 1)*(S(PE) - M(1)*M(2))
1802 = M(1)*M(2)*(S(1) - 2) + S(PE)*(M(1) + 1)
1803
1804 So, for example, with S(1) = 10, M(1) = 10, and M(2) = 10, we see:
1805
1806 S(PE) = 1000
1807 S(2) = 100
1808 X(PE) = 3996 (or 1998)
1809 X(2) = 11000
1810 X(1) = 11800
1811
1812 And similarly, with S(1) = 20, M(1) = 20, and M(2) = 5, we see:
1813
1814 S(PE) = 2000
1815 S(2) = 400
1816 X(PE) = 5996 (or 2998)
1817 X(2) = 12000
1818 X(1) = 39800
1819
1820 <span class="h4"><a class="selflink" id="section-8.3.1" href="#section-8.3.1">8.3.1</a>. Comparison with Other Scenarios</span>
1821
1822 For comparison with the examples in Sections <a href="#section-5">5</a> and <a href="#section-6">6</a>, we need to
1823 convert those LSP-based figures to our new measure of LSP state.
1824
1825 Observe that each LSP in Sections <a href="#section-5">5</a> and <a href="#section-6">6</a> generates two state units
1826 at a transit LSR and one at an ingress or egress. So we can provide
1827 conversions as follows:
1828
1829 <a href="#section-5">Section 5</a> (flat snowflake network)
1830
1831 L(PE) = 2*(S(PE) - 1)
1832 L(2) = M(2)*(2*S(PE) - M(2) - 1)
1833 L(1) = M(1)*M(2)*(2*S(PE) - M(2)*(M(1) + 1))
1834 X(PE) = 2*(S(PE) - 1)
1835 X(2) = 2*M(2)*(2*S(PE) - M(2) - 1)
1836 X(1) = 2*M(1)*M(2)*(2*S(PE) - M(2)*(M(1) + 1))
1837
1838 For the example with S(1) = 10, M(1) = 10, and M(2) = 10, this
1839 gives a comparison table as follows:
1840
1841
1842
1843
1844 <span class="grey">Yasukawa, et al. Informational [Page 33]</span></pre>
1845 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-34" ></span>
1846 <span class="grey"><a href="./rfc5439">RFC 5439</a> Scaling in MPLS-TE February 2009</span>
1847
1848
1849 Count | Unmodified | MP2P
1850 ------+-------------+----------
1851 X(PE) | 1998 | 3996
1852 X(2) | 39780 | 11000
1853 X(1) | 378000 | 11800
1854
1855 Clearly, this technique is a significant improvement over the flat
1856 network within the core of the network, although the PEs are more
1857 heavily stressed if disambiguation is required.
1858
1859 <a href="#section-6.1">Section 6.1</a> (two-layer hierarchy snowflake network)
1860
1861 L(PE) = 2*(S(PE) - 1)
1862 L(2) = M(2)*(2*S(PE) - M(2) - 1) + 2*(S(2) - 1)
1863 L(1) = M(1)*(2*S(2) - M(1) - 1)
1864 X(PE) = 2*(S(PE) - 1)
1865 X(2) = 2*M(2)*(2*S(PE) - M(2) - 1) + 2*(S(2) - 1)
1866 X(1) = 2*M(1)*(2*S(2) - M(1) - 1)
1867
1868 Note that in the computation of X(2) the hierarchical LSPs only add
1869 one state at each P(2) node.
1870
1871 For the same example with S(1) = 10, M(1) = 10, and M(2) = 10, this
1872 gives a comparison table as follows:
1873
1874 Count | 2-Layer | MP2P
1875 | Hierarchy |
1876 ------+-------------+----------
1877 X(PE) | 1998 | 3996
1878 X(2) | 39978 | 11000
1879 X(1) | 3780 | 11800
1880
1881 We can observe that the MP2P model is better at P(2), but the
1882 hierarchical model is better at P(1).
1883
1884 In fact, this comparison can be generalized to observe that the MP2P
1885 model produces its best effects toward the edge of the network, while
1886 the hierarchical model makes most impression at the core. However,
1887 the requirement for disambiguation of P2P LSPs tunneled within the
1888 MP2P LSPs does cause a double burden at the PEs.
1889
1890 <span class="h3"><a class="selflink" id="section-8.4" href="#section-8.4">8.4</a>. Scaling Improvements for Ladder Networks</span>
1891
1892 MP2P LSPs applied just within the ladder will not make a significant
1893 difference, but applying MP2P for all LSPs and at all nodes makes a
1894 very big difference without requiring any further configuration.
1895
1896
1897
1898
1899
1900 <span class="grey">Yasukawa, et al. Informational [Page 34]</span></pre>
1901 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-35" ></span>
1902 <span class="grey"><a href="./rfc5439">RFC 5439</a> Scaling in MPLS-TE February 2009</span>
1903
1904
1905 LSP state at a spar-node may be divided into those LSPs' segments
1906 that enter or leave the spar-node due to subtended PEs (local LSP
1907 segments), and those that enter or leave the spar-node due to remote
1908 PEs (remote segments).
1909
1910 The local segments may be counted as:
1911
1912 o E LSPs targeting local PEs
1913 o (S(1)-1)*E*M(1) LSPs targeting remote PEs
1914
1915 The remote segments may be counted as:
1916
1917 o (S(1)-1)*E outgoing LSPs targeting remote PEs
1918 o &lt;= 3*S(1)*E incoming LSPs targeting any PE (there are precisely
1919 P(1) nodes attached to any other P(1) node)
1920
1921 Hence, using X(1) as a measure of LSP state rather than a count of
1922 LSPs, we get:
1923
1924 X(1) &lt;= E + (S(1)-1)*E*M(1) + (S(1)-1)*E + 3*S(1)*E
1925 &lt;= (4 + M(1))*S(1)*E - M(1)*E
1926
1927 The number of LSPs at the P(2) nodes is also improved. We may also
1928 count the LSP state in the same way so that there are:
1929
1930 o M(2) LSPs targeting local PEs,
1931 o M(2)*(S(1)*E) LSPs from local PEs to all other PEs, and
1932 o S(1)*E - M(2) LSPs to remote PEs.
1933
1934 So using X(2) as a measure of LSP state and not a count of LSPs, we
1935 have:
1936
1937 X(2) = M(2) + M(2)*(S(1)*E) + S(1)*E - M(2)
1938 = (M(2) + 1)*S(1)*E
1939
1940 Our examples from <a href="#section-5.2">Section 5.2</a> give us the following numbers:
1941
1942 With S(1) = 6, M(1) = 10, and M(2) = 17, we see:
1943
1944 E = 170
1945 S(PE) = 1020
1946 X(PE) = 2038
1947 X(2) = 18360
1948 X(1) &lt;= 12580
1949
1950
1951
1952
1953
1954
1955
1956 <span class="grey">Yasukawa, et al. Informational [Page 35]</span></pre>
1957 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-36" ></span>
1958 <span class="grey"><a href="./rfc5439">RFC 5439</a> Scaling in MPLS-TE February 2009</span>
1959
1960
1961 Alternatively, with S(1) = 10, M(1) = 10, and M(2) = 20, we see:
1962
1963 E = 200
1964 S(PE) = 2000
1965 X(PE) = 3998
1966 X(2) = 42000
1967 X(1) &lt;= 26000
1968
1969 <span class="h4"><a class="selflink" id="section-8.4.1" href="#section-8.4.1">8.4.1</a>. Comparison with Other Scenarios</span>
1970
1971 The use of MP2P compares very favorably with all scaling scenarios.
1972 It is the only technique able to reduce the value of X(2), and it
1973 does this by a factor of almost two. The impact on X(1) is better
1974 than everything except the three-level hierarchy.
1975
1976 The following table provides a quick cross-reference for the figures
1977 for the example ladder networks. Note that the previous figures are
1978 modified to provide counts of LSP state rather than LSP numbers.
1979 Again, each LSP contributes one state at its end points and two
1980 states at transit nodes.
1981
1982 Thus, for the all cases we have:
1983
1984 X(PE) = 2*(S(PE) - 1) or
1985 X(PE) = 4*(S(PE) - 1) if disambiguation is required.
1986
1987 In the unmodified (flat) case, we have:
1988
1989 X(2) = 2*(M(2)*(2*S(PE) - M(2) - 1))
1990 X(1) = 2*(M(1)*M(2)*(2*S(PE) - M(2)*(M(1) + 1)))
1991
1992 In the two-level hierarchy, we have:
1993
1994 X(2) = 2*(2*M(2)*(S(PE) - 1) - M(2)*(M(2) - 1))
1995 X(1) = S(1)*S(1) + 2*S(1) + 4*E*E*(S(1) - 1) - 2*E*M(2) - 2
1996
1997 In the three-level hierarchy, we have:
1998
1999 X(2) = 2*(2*M(2)*(S(PE) - 1) - M(2)*(M(2) - 1)) + 2*(S(1)*M(1) - 1)
2000 X(1) = S(1)*S(1) + 2*S(1) + 4*M(1)*M(1)*S(1) - 2*M(1)(M(1) + 1) - 2
2001
2002 Example A: S(1) = 6, M(1) = 10, and M(2) = 17
2003 Example B: S(1) = 10, M(1) = 10, and M(2) = 20
2004
2005
2006
2007
2008
2009
2010
2011
2012 <span class="grey">Yasukawa, et al. Informational [Page 36]</span></pre>
2013 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-37" ></span>
2014 <span class="grey"><a href="./rfc5439">RFC 5439</a> Scaling in MPLS-TE February 2009</span>
2015
2016
2017 Example| Count | Unmodified | 2-Level | 3-Level | MP2P
2018 | | | Hierarchy | Hierarchy |
2019 -------+-------+------------+------------+-------------+-------
2020 A | X(2) | 68748 | 68748 | 68866 | 18360
2021 | X(1) | 1554820 | 572266 | 2226 | 12580
2022 -------+-------+------------+------------+-------------+-------
2023 B | X(2) | 159160 | 159160 | 159358 | 42000
2024 | X(1) | 5032000 | 1433998 | 3898 | 26000
2025
2026 <span class="h4"><a class="selflink" id="section-8.4.2" href="#section-8.4.2">8.4.2</a>. LSP State Compared with LSP Numbers</span>
2027
2028 Recall that in <a href="#section-8.3">Section 8.3</a>, the true benefit of MP2P was analyzed
2029 with respect to the LSP segment state required, rather than the
2030 actual number of LSPs. This proved to be a more accurate comparison
2031 of the techniques because the MP2P LSPs require state on each branch
2032 of the LSP, so the saving is not linear with the reduced number of
2033 LSPs.
2034
2035 A similar analysis could be performed here for the ladder network.
2036 The net effect is that it increases the state by an order of two for
2037 all transit LSPs in the P2P models, and by a multiplier equal to the
2038 degree of a node in the MP2P model.
2039
2040 A rough estimate shows that, as with snowflake networks, MP2P
2041 provides better scaling than the one-level hierarchical model and is
2042 considerably better at the core. But MP2P compares less will with
2043 the two-level hierarchy especially in the core.
2044
2045 <span class="h3"><a class="selflink" id="section-8.5" href="#section-8.5">8.5</a>. Issues with MP2P LSPs</span>
2046
2047 The biggest challenges for MP2P LSPs are the provision of support in
2048 the control and data planes. To some extent, support must also be
2049 provided in the management plane.
2050
2051 Control plane support is just a matter of defining the protocols and
2052 procedures [<a href="#ref-MP2P-RSVP" title="&quot;Supporting Multipoint-to-Point Label Switched Paths in Multiprotocol Label Switching Traffic Engineering&quot;">MP2P-RSVP</a>], although it must be clearly understood that
2053 this will introduce some complexity to the control plane.
2054
2055 Hardware issues may be a little more tricky. For example, the
2056 capacity of the upstream segments must never (allowing for
2057 statistical over-subscription) exceed the capacity of the downstream
2058 segment. Similarly, data planes must be equipped with sufficient
2059 buffers to handle incoming packet collisions.
2060
2061 The management plane will be impacted in several ways. Firstly, the
2062 management applications will need to handle LSPs with multiple
2063 senders. This means that, although the applications need to process
2064 fewer LSPs, they will be more complicated and will, in fact, need to
2065
2066
2067
2068 <span class="grey">Yasukawa, et al. Informational [Page 37]</span></pre>
2069 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-38" ></span>
2070 <span class="grey"><a href="./rfc5439">RFC 5439</a> Scaling in MPLS-TE February 2009</span>
2071
2072
2073 process the same number of ingresses and egresses. Other issues like
2074 diagnostics and OAM would also need to be enhanced to support MP2P,
2075 but might be borrowed heavily from LDP networks.
2076
2077 Lastly, note that when the MP2P solution is used, the receiver (the
2078 single egress PE of an MP2P tunnel) cannot use the incoming label as
2079 an indicator of the source of the data. Contrast this with P2P LSPs.
2080 Depending on deployment, this might not be an issue since the PE-PE
2081 connectivity may in any case be a tunnel with inner labels to
2082 discriminate the data flows.
2083
2084 In other deployments, it may be considered necessary to include
2085 additional PE-PE P2P LSPs and tunnel these through the MP2P LSPs.
2086 This would require the PEs to support twice as many LSPs. Since PEs
2087 are not usually as fully specified as P-routers, this may cause some
2088 concern; however, the use of penultimate hop popping on the MP2P LSPs
2089 might help to reduce this issue.
2090
2091 In all cases, care must be taken not to confuse the reduction in the
2092 number of LSPs with a reduction in the LSP state that is required.
2093 In fact, the discussion in <a href="#section-8.3">Section 8.3</a> is slightly optimistic since
2094 LSP state toward the destination will probably need to include sender
2095 information and so will increase depending on the number of senders
2096 for the MP2P LSP. <a href="#section-8.4">Section 8.4</a>, on the other hand, counts LSP state
2097 rather than LSPs. This issue is clearly dependent on the protocol
2098 solution for MP2P RSVP-TE, which is out of scope for this document.
2099
2100 MPLS Fast Reroute (FRR) [<a href="./rfc4090" title="&quot;Fast Reroute Extensions to RSVP-TE for LSP Tunnels&quot;">RFC4090</a>] is an attractive scheme for
2101 providing rapid local protection from node or link failures. Such a
2102 scheme has, however, not been designed for MP2P at the time of
2103 writing, so it remains to be seen how practical it could be,
2104 especially in the case of the failure of a merge node. Initial
2105 examination of this case suggests that FRR would not be a problem for
2106 MP2P, given that each flow can be handled separately.
2107
2108 As a final note, observe that the MP2P scenario presented in this
2109 document may be optimistic. MP2P LSP merging may be hard to achieve
2110 between LSPs with significantly different traffic and Quality of
2111 Service (QoS) parameters. Therefore, it may be necessary to increase
2112 the number of MP2P LSPs arriving at an egress.
2113
2114
2115
2116
2117
2118
2119
2120
2121
2122
2123
2124 <span class="grey">Yasukawa, et al. Informational [Page 38]</span></pre>
2125 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-39" ></span>
2126 <span class="grey"><a href="./rfc5439">RFC 5439</a> Scaling in MPLS-TE February 2009</span>
2127
2128
2129 <span class="h2"><a class="selflink" id="section-9" href="#section-9">9</a>. Combined Models</span>
2130
2131 There is nothing to prevent the combination of hierarchical and MP2P
2132 solutions within a network.
2133
2134 Note that if MP2P LSPs are tunneled through P2P FA LSPs across the
2135 core, none of the benefit of LSP merging is seen for the hops during
2136 which the MP2P LSPs are tunneled.
2137
2138 On the other hand, it is possible to construct solutions where MP2P
2139 FA LSPs are constructed within the network, resulting in savings from
2140 both modes of operation.
2141
2142 <span class="h2"><a class="selflink" id="section-10" href="#section-10">10</a>. An Alternate Solution</span>
2143
2144 A simple solution to reducing the number of LSP tunnels handled by
2145 any node in the network has been proposed. In this solution it is
2146 observed that part of the problem is caused purely by the total
2147 number of LSP in the network, and that this is a function of the
2148 number of PEs since a full mesh of PE-PE LSPs is required. The
2149 conclusion of this observation is to move the tunnel end-points
2150 further into the network so that, instead of having a full mesh of
2151 PE-PE tunnels, we have only a full mesh of P(n)-P(n) tunnels.
2152
2153 Obviously, there is no change in the physical network topology, so
2154 the PEs remain subtended to the P(n) nodes, and the consequence is
2155 that there is no TE on the links between PEs and P(n) nodes.
2156
2157 In this case, we have already done the hard work for computing the
2158 number of LSPs in the previous sections. The power of the analysis
2159 in the earlier sections is demonstrated by its applicability to this
2160 new model -- all we need to do is make minor changes to the formulae.
2161 This is most simply done by removing a layer from the network. We
2162 introduce the term "tunnel end-point" (TEP) and replace the P(n)
2163 nodes with TEPs. Thus, the example of a flat snowflake network in
2164 Figure 3 becomes as shown in Figure 7. Corresponding changes can be
2165 made to all of the sample topologies.
2166
2167
2168
2169
2170
2171
2172
2173
2174
2175
2176
2177
2178
2179
2180 <span class="grey">Yasukawa, et al. Informational [Page 39]</span></pre>
2181 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-40" ></span>
2182 <span class="grey"><a href="./rfc5439">RFC 5439</a> Scaling in MPLS-TE February 2009</span>
2183
2184
2185 PE PE PE PE PE PE
2186 \ \/ \/ /
2187 PE--TEP TEP TEP TEP--PE
2188 \ | | /
2189 \| |/
2190 PE--TEP---P(1)------P(1)---TEP--PE
2191 / \ / \
2192 PE \ / PE
2193 \/
2194 P(1)
2195 /|\
2196 / | \
2197 / | \
2198 PE--TEP TEP TEP--PE
2199 / /\ \
2200 PE PE PE PE
2201
2202 Figure 7 : An Example Snowflake Network with Tunnel End-Points
2203
2204 To perform the scaling calculations we need only replace the PE
2205 counts in the formulae with TEP counts, and observe that there is one
2206 fewer layer in the network. For example, in the flat snowflake
2207 network shown in Figure 7, we can see that the number of LSPs seen at
2208 a TEP is:
2209
2210 L(TEP) = 2*(S(TPE) - 1)
2211
2212 In our sample networks, S(TPE) is typically of the order of 50 or 100
2213 (the original values of S(2)), so L(TEP) is less than 200, which is
2214 quite manageable.
2215
2216 Similarly, the number of LSPs handled by a P(1) node can be derived
2217 from the original formula for the number of LSPs seen at a P(2) node,
2218 since all we have done is reduce n in P(n) from 2 to 1. So our new
2219 formula is:
2220
2221 L(1) = M(1)*(2*S(TEP) - M(1) - 1)
2222
2223 With figures for M(1) = 10 and S(TEP) = 100, this gives us L(1) =
2224 1890. This is also very manageable.
2225
2226 <span class="h3"><a class="selflink" id="section-10.1" href="#section-10.1">10.1</a>. Pros and Cons of the Alternate Solution</span>
2227
2228 On the face of it, this alternate solution seems very attractive.
2229 Simply by contracting the edges of the tunnels into the network, we
2230 have shown a dramatic reduction in the number of tunnels needed, and
2231 there is no requirement to apply any additional scaling techniques.
2232
2233
2234
2235
2236 <span class="grey">Yasukawa, et al. Informational [Page 40]</span></pre>
2237 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-41" ></span>
2238 <span class="grey"><a href="./rfc5439">RFC 5439</a> Scaling in MPLS-TE February 2009</span>
2239
2240
2241 But what of the PE-P(n) links? In the earlier sections of this
2242 document, we have assumed that there was some requirement for PE-PE
2243 LSPs with TE properties that extended to the PE-P(n) links at both
2244 ends of each LSP. That means that there was a requirement to provide
2245 reservation-based QoS on those links, to be able to discriminate
2246 traffic flows for priority-based treatment, and to be able to
2247 distinguish applications and sources that send data based on the LSPs
2248 that carry the data.
2249
2250 It might be argued that, since the PE-P(n) links do not offer any
2251 routing options (each such link provides the only access to the
2252 network for a PE), most of the benefits of tunnels are lost on these
2253 peripheral links. However, TE is not just about routing. Just as
2254 important are the abilities to make resource reservations, to
2255 prioritize traffic, and to discriminate between traffic from
2256 different applications, customers, or VPNs.
2257
2258 Furthermore, in multihoming scenarios where each PE is connected to
2259 more than one P(n) or where a PE has multiple links to a single P(n),
2260 there may be a desire to pre-select the link to be used and to direct
2261 the traffic to that link using a PE-PE LSP. Note that multihoming
2262 has not been considered in this document.
2263
2264 Operationally, P(n)-P(n) LSPs offer the additional management
2265 overhead that is seen for hierarchical LSPs described in <a href="#section-6">Section 6</a>.
2266 That is, the LSPs have to be configured and established through
2267 additional configuration or management operations that are not
2268 carried out at the PEs. As described in <a href="#section-6">Section 6</a>, automesh
2269 [<a href="./rfc4972" title="&quot;Routing Extensions for Discovery of Multiprotocol (MPLS) Label Switch Router (LSR) Traffic Engineering (TE) Mesh Membership&quot;">RFC4972</a>] could be used to ease this task. But it must be noted
2270 that, as mentioned above, some of the key uses of tunnels require
2271 that traffic is classified and placed in an appropriate tunnel
2272 according to its traffic class, end-points, originating application,
2273 and customer (such as client VPN). This information may not be
2274 readily available for each packet at the P(n) nodes since it is PE-
2275 based information. Of course, it is possible to conceive of
2276 techniques to make this information available, such as assigning a
2277 different label for each class of traffic, but this gives rise to the
2278 original problem of larger numbers of LSPs.
2279
2280 Our conclusion is, therefore, that this alternate technique may be
2281 suitable for the general distribution of traffic based solely on the
2282 destination, or on a combination of the destination and key fields
2283 carried in the IP header. In this case, it can provide a very
2284 satisfactory answer to the scaling issues in an MPLS-TE network. But
2285 if more sophisticated packet classification and discrimination is
2286 required, this technique will make the desired function hard to
2287
2288
2289
2290
2291
2292 <span class="grey">Yasukawa, et al. Informational [Page 41]</span></pre>
2293 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-42" ></span>
2294 <span class="grey"><a href="./rfc5439">RFC 5439</a> Scaling in MPLS-TE February 2009</span>
2295
2296
2297 achieve, and the trade-off between scaling and feature-level will
2298 swing too far towards solving the scaling issue at the expense of
2299 delivery of function to the customer.
2300
2301 <span class="h2"><a class="selflink" id="section-11" href="#section-11">11</a>. Management Considerations</span>
2302
2303 The management issues of the models presented in this document have
2304 been discussed in-line. No one solution is without its management
2305 overhead.
2306
2307 Note, however, that scalability of management tools is one of the
2308 motivators for this work and that network scaling solutions that
2309 reduce the active management of LSPs at the cost of additional effort
2310 to manage the more static elements of the network represent a
2311 benefit. That is, it is worth the additional effort to set up MP2P
2312 or FA LSPs if it means that the network can be scaled to a larger
2313 size without being constrained by the management tools.
2314
2315 The MP2P technique may prove harder to debug through OAM methods than
2316 the FA LSP approach.
2317
2318 <span class="h2"><a class="selflink" id="section-12" href="#section-12">12</a>. Security Considerations</span>
2319
2320 The techniques described in this document use existing or yet-to-be-
2321 defined signaling protocol extensions and are subject to the security
2322 provided by those extensions. Note that we are talking about
2323 tunneling techniques used within the network and that both approaches
2324 are vulnerable to the creation of bogus tunnels that deliver data to
2325 an egress or consume network resources.
2326
2327 The fact that the MP2P technique may prove harder to debug through
2328 OAM methods than the FA LSP approach is a security concern since it
2329 is important to be able to detect misconnections.
2330
2331 General issues of the relationship between scaling and security are
2332 covered in <a href="#section-1.1">Section 1.1</a>, but the details are beyond the scope of this
2333 document. Readers are referred to [<a href="#ref-MPLS-SEC" title="&quot;Security Framework for MPLS and GMPLS Networks&quot;">MPLS-SEC</a>] for details of MPLS
2334 security techniques.
2335
2336 <span class="h2"><a class="selflink" id="section-13" href="#section-13">13</a>. Recommendations</span>
2337
2338 The analysis in this document suggests that the ability to signal
2339 MP2P MPLS-TE LSPs is a desirable addition to the operator's MPLS-TE
2340 toolkit.
2341
2342 At this stage, no further recommendations are made, but it would be
2343 valuable to consult more widely to discover:
2344
2345
2346
2347
2348 <span class="grey">Yasukawa, et al. Informational [Page 42]</span></pre>
2349 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-43" ></span>
2350 <span class="grey"><a href="./rfc5439">RFC 5439</a> Scaling in MPLS-TE February 2009</span>
2351
2352
2353 - The concerns of other service providers with respect to network
2354 scalability.
2355
2356 - More opinions on the realistic constraints to the network
2357 parameters listed in <a href="#section-4">Section 4</a>.
2358
2359 - Desirable values for the cost-effectiveness of the network
2360 (parameter K).
2361
2362 - The applicability, manageability, and support for the two
2363 techniques described.
2364
2365 - The feasibility of combining the two techniques, as discussed in
2366 <a href="#section-9">Section 9</a>.
2367
2368 - The level of concern over the loss of functionality that would
2369 occur if the alternate solution described in <a href="#section-10">Section 10</a> was
2370 adopted.
2371
2372 <span class="h2"><a class="selflink" id="section-14" href="#section-14">14</a>. Acknowledgements</span>
2373
2374 The authors are grateful to Jean-Louis Le Roux for discussions and
2375 review input. Thanks to Ben Niven-Jenkins, JP Vasseur, Loa
2376 Andersson, Anders Gavler, Ben Campbell, and Tim Polk for their
2377 comments. Thanks to Dave Allen for useful discussion of the math.
2378
2379 <span class="h2"><a class="selflink" id="section-15" href="#section-15">15</a>. Normative References</span>
2380
2381 [<a id="ref-RFC4206">RFC4206</a>] Kompella, K. and Y. Rekhter, "Label Switched Paths (LSP)
2382 Hierarchy with Generalized Multi-Protocol Label Switching
2383 (GMPLS) Traffic Engineering (TE)", <a href="./rfc4206">RFC 4206</a>, October
2384 2005.
2385
2386 <span class="h2"><a class="selflink" id="section-16" href="#section-16">16</a>. Informative References</span>
2387
2388 [<a id="ref-RFC2961">RFC2961</a>] Berger, L., Gan, D., Swallow, G., Pan, P., Tommasi, F.,
2389 and S. Molendini, "RSVP Refresh Overhead Reduction
2390 Extensions", <a href="./rfc2961">RFC 2961</a>, April 2001.
2391
2392 [<a id="ref-RFC3209">RFC3209</a>] Awduche, D., Berger, L., Gan, D., Li, T., Srinivasan, V.,
2393 and G. Swallow, "RSVP-TE: Extensions to RSVP for LSP
2394 Tunnels", <a href="./rfc3209">RFC 3209</a>, December 2001.
2395
2396 [<a id="ref-RFC3270">RFC3270</a>] Le Faucheur, F., Wu, L., Davie, B., Davari, S., Vaananen,
2397 P., Krishnan, R., Cheval, P., and J. Heinanen, "Multi-
2398 Protocol Label Switching (MPLS) Support of Differentiated
2399 Services", <a href="./rfc3270">RFC 3270</a>, May 2002.
2400
2401
2402
2403
2404 <span class="grey">Yasukawa, et al. Informational [Page 43]</span></pre>
2405 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-44" ></span>
2406 <span class="grey"><a href="./rfc5439">RFC 5439</a> Scaling in MPLS-TE February 2009</span>
2407
2408
2409 [<a id="ref-RFC3473">RFC3473</a>] Berger, L., Ed., "Generalized Multi-Protocol Label
2410 Switching (GMPLS) Signaling Resource ReserVation
2411 Protocol-Traffic Engineering (RSVP-TE) Extensions", <a href="./rfc3473">RFC</a>
2412 <a href="./rfc3473">3473</a>, January 2003.
2413
2414 [<a id="ref-RFC3985">RFC3985</a>] Bryant, S., Ed., and P. Pate, Ed., "Pseudo Wire Emulation
2415 Edge-to-Edge (PWE3) Architecture", <a href="./rfc3985">RFC 3985</a>, March 2005.
2416
2417 [<a id="ref-RFC4090">RFC4090</a>] Pan, P., Ed., Swallow, G., Ed., and A. Atlas, Ed., "Fast
2418 Reroute Extensions to RSVP-TE for LSP Tunnels", <a href="./rfc4090">RFC 4090</a>,
2419 May 2005.
2420
2421 [<a id="ref-RFC4110">RFC4110</a>] Callon, R. and M. Suzuki, "A Framework for Layer 3
2422 Provider-Provisioned Virtual Private Networks (PPVPNs)",
2423 <a href="./rfc4110">RFC 4110</a>, July 2005.
2424
2425 [<a id="ref-RFC4972">RFC4972</a>] Vasseur, JP., Ed., Leroux, JL., Ed., Yasukawa, S.,
2426 Previdi, S., Psenak, P., and P. Mabbey, "Routing
2427 Extensions for Discovery of Multiprotocol (MPLS) Label
2428 Switch Router (LSR) Traffic Engineering (TE) Mesh
2429 Membership", <a href="./rfc4972">RFC 4972</a>, July 2007.
2430
2431 [<a id="ref-RFC5036">RFC5036</a>] Andersson, L., Ed., Minei, I., Ed., and B. Thomas, Ed.,
2432 "LDP Specification", <a href="./rfc5036">RFC 5036</a>, October 2007.
2433
2434 [<a id="ref-MP2P-RSVP">MP2P-RSVP</a>] Yasukawa, Y., "Supporting Multipoint-to-Point Label
2435 Switched Paths in Multiprotocol Label Switching Traffic
2436 Engineering", Work in Progress, October 2008.
2437
2438 [<a id="ref-MPLS-SEC">MPLS-SEC</a>] Fang, L., Ed., "Security Framework for MPLS and GMPLS
2439 Networks", Work in Progress, November 2008.
2440
2441
2442
2443
2444
2445
2446
2447
2448
2449
2450
2451
2452
2453
2454
2455
2456
2457
2458
2459
2460 <span class="grey">Yasukawa, et al. Informational [Page 44]</span></pre>
2461 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-45" ></span>
2462 <span class="grey"><a href="./rfc5439">RFC 5439</a> Scaling in MPLS-TE February 2009</span>
2463
2464
2465 Authors' Addresses
2466
2467 Seisho Yasukawa
2468 NTT Corporation
2469 9-11, Midori-Cho 3-Chome
2470 Musashino-Shi, Tokyo 180-8585 Japan
2471 Phone: +81 422 59 4769
2472 EMail: [email protected]
2473
2474 Adrian Farrel
2475 Old Dog Consulting
2476 EMail: [email protected]
2477
2478 Olufemi Komolafe
2479 Cisco Systems
2480 96 Commercial Street
2481 Edinburgh
2482 EH6 6LX
2483 United Kingdom
2484 EMail: [email protected]
2485
2486
2487
2488
2489
2490
2491
2492
2493
2494
2495
2496
2497
2498
2499
2500
2501
2502
2503
2504
2505
2506
2507
2508
2509
2510
2511
2512
2513
2514
2515
2516 Yasukawa, et al. Informational [Page 45]
2517 </pre>