Debian Codegraph @

rfc6198.html

raw
1 <pre>Internet Engineering Task Force (IETF) B. Decraene
2 Request for Comments: 6198 France Telecom
3 Category: Informational P. Francois
4 ISSN: 2070-1721 UCL
5 C. Pelsser
6 IIJ
7 Z. Ahmad
8 Orange Business Services
9 A.J. Elizondo Armengol
10 Telefonica I+D
11 T. Takeda
12 NTT
13 April 2011
14
15
16 <span class="h1">Requirements for the Graceful Shutdown of BGP Sessions</span>
17
18 Abstract
19
20 The Border Gateway Protocol (BGP) is heavily used in Service Provider
21 networks for both Internet and BGP/MPLS VPN services. For resiliency
22 purposes, redundant routers and BGP sessions can be deployed to
23 reduce the consequences of an Autonomous System Border Router (ASBR)
24 or BGP session breakdown on customers' or peers' traffic. However,
25 simply taking down or even bringing up a BGP session for maintenance
26 purposes may still induce connectivity losses during the BGP
27 convergence. This is no longer satisfactory for new applications
28 (e.g., voice over IP, online gaming, VPN). Therefore, a solution is
29 required for the graceful shutdown of a (set of) BGP session(s) in
30 order to limit the amount of traffic loss during a planned shutdown.
31 This document expresses requirements for such a solution.
32
33 Status of This Memo
34
35 This document is not an Internet Standards Track specification; it is
36 published for informational purposes.
37
38 This document is a product of the Internet Engineering Task Force
39 (IETF). It represents the consensus of the IETF community. It has
40 received public review and has been approved for publication by the
41 Internet Engineering Steering Group (IESG). Not all documents
42 approved by the IESG are a candidate for any level of Internet
43 Standard; see <a href="./rfc5741#section-2">Section&nbsp;2 of RFC 5741</a>.
44
45 Information about the current status of this document, any errata,
46 and how to provide feedback on it may be obtained at
47 <a href="http://www.rfc-editor.org/info/rfc6198">http://www.rfc-editor.org/info/rfc6198</a>.
48
49
50
51
52 <span class="grey">Decraene, et al. Informational [Page 1]</span></pre>
53 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-2" ></span>
54 <span class="grey"><a href="./rfc6198">RFC 6198</a> Reqs for Graceful BGP Shutdown April 2011</span>
55
56
57 Copyright Notice
58
59 Copyright (c) 2011 IETF Trust and the persons identified as the
60 document authors. All rights reserved.
61
62 This document is subject to <a href="https://www.rfc-editor.org/bcp/bcp78">BCP 78</a> and the IETF Trust's Legal
63 Provisions Relating to IETF Documents
64 (<a href="http://trustee.ietf.org/license-info">http://trustee.ietf.org/license-info</a>) in effect on the date of
65 publication of this document. Please review these documents
66 carefully, as they describe your rights and restrictions with respect
67 to this document. Code Components extracted from this document must
68 include Simplified BSD License text as described in Section 4.e of
69 the Trust Legal Provisions and are provided without warranty as
70 described in the Simplified BSD License.
71
72 Table of Contents
73
74 <a href="#section-1">1</a>. Introduction ....................................................<a href="#page-2">2</a>
75 <a href="#section-2">2</a>. Conventions Used in This Document ...............................<a href="#page-3">3</a>
76 <a href="#section-3">3</a>. Problem Statement ...............................................<a href="#page-4">4</a>
77 <a href="#section-3.1">3.1</a>. Example of Undesirable BGP Routing Behavior ................<a href="#page-4">4</a>
78 <a href="#section-3.2">3.2</a>. Causes of Packet Loss ......................................<a href="#page-5">5</a>
79 <a href="#section-4">4</a>. Terminology .....................................................<a href="#page-6">6</a>
80 <a href="#section-5">5</a>. Goals and Requirements ..........................................<a href="#page-7">7</a>
81 <a href="#section-6">6</a>. Security Considerations ........................................<a href="#page-10">10</a>
82 <a href="#section-7">7</a>. References .....................................................<a href="#page-10">10</a>
83 <a href="#section-7.1">7.1</a>. Normative References ......................................<a href="#page-10">10</a>
84 <a href="#section-7.2">7.2</a>. Informative References ....................................<a href="#page-10">10</a>
85 Acknowledgments ...................................................<a href="#page-11">11</a>
86 <a href="#appendix-A">Appendix A</a>. Reference BGP Topologies ..............................<a href="#page-12">12</a>
87 <a href="#appendix-A.1">A.1</a>. EBGP Topologies ...........................................<a href="#page-12">12</a>
88 <a href="#appendix-A.2">A.2</a>. IBGP Topologies ...........................................<a href="#page-15">15</a>
89 <a href="#appendix-A.3">A.3</a>. Routing Decisions .........................................<a href="#page-19">19</a>
90
91 <span class="h2"><a class="selflink" id="section-1" href="#section-1">1</a>. Introduction</span>
92
93 The Border Gateway Protocol (BGP) [<a href="./rfc4271" title="&quot;A Border Gateway Protocol 4 (BGP-4)&quot;">RFC4271</a>] is heavily used in
94 Service Provider networks for both Internet and BGP/MPLS VPN services
95 [<a href="./rfc4364" title="&quot;BGP/MPLS IP Virtual Private Networks (VPNs)&quot;">RFC4364</a>]. For resiliency purposes, redundant routers and BGP
96 sessions can be deployed to reduce the consequences of an Autonomous
97 System Border Router (ASBR) or BGP session breakdown on customers' or
98 peers' traffic.
99
100 We place ourselves in the context where a Service Provider performs a
101 maintenance operation and needs to shut down one or multiple BGP
102 peering link(s) or a whole ASBR. If an alternate path is available
103 within the Autonomous System (AS), the requirement is to avoid or
104 reduce customer or peer traffic loss during the BGP convergence.
105
106
107
108 <span class="grey">Decraene, et al. Informational [Page 2]</span></pre>
109 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-3" ></span>
110 <span class="grey"><a href="./rfc6198">RFC 6198</a> Reqs for Graceful BGP Shutdown April 2011</span>
111
112
113 Indeed, as an alternate path is available in the AS, it should be
114 made possible to reroute the customer or peer traffic on this backup
115 path before the BGP session(s) is/are torn down, the nominal path
116 withdrawn, and the forwarding stopped.
117
118 The requirements also cover the subsequent re-establishment of the
119 BGP session as even this "UP" case can currently trigger route loss,
120 and thus traffic loss, at some routers.
121
122 BGP [<a href="./rfc4271" title="&quot;A Border Gateway Protocol 4 (BGP-4)&quot;">RFC4271</a>] and MP-BGP [<a href="./rfc4760" title="&quot;Multiprotocol Extensions for BGP-4&quot;">RFC4760</a>] do not currently have a mechanism
123 to gracefully migrate traffic from one BGP next-hop to another
124 without interrupting the flow of traffic. When a BGP session is
125 taken down, BGP behaves as if there were a sudden link or router
126 failure and withdraws the prefixes learned over that session, which
127 may trigger traffic loss. While still being advertised as reachable,
128 there is no mechanism to advertise to its BGP peers that the prefix
129 will soon be unreachable. When applicable, such mechanism would
130 reduce or prevent traffic loss. It would typically be applicable in
131 case of a maintenance operation requiring the shutdown of a
132 forwarding resource. Typical examples would be a link or line card
133 maintenance, replacement, or upgrade. It may also be applicable for
134 a software upgrade, as it may involve a firmware reset on the line
135 cards and hence forwarding interruption.
136
137 The introduction of route reflectors (RRs) as per [<a href="./rfc4456" title="&quot;BGP Route Reflection: An Alternative to Full Mesh Internal BGP (IBGP)&quot;">RFC4456</a>] to solve
138 scalability issues bound to Internal BGP (IBGP) full-meshes has
139 worsened the duration of routing convergence as some route reflectors
140 may hide the backup path. Thus, depending on RR topology, more IBGP
141 hops may be involved in the IBGP convergence.
142
143 Note that these planned maintenance operations cannot be addressed by
144 Graceful Restart (GR) extensions [<a href="./rfc4724" title="&quot;Graceful Restart Mechanism for BGP&quot;">RFC4724</a>] as GR only applies when
145 the forwarding is preserved during the control plane restart. On the
146 contrary, graceful shutdown applies when the forwarding is
147 interrupted.
148
149 Also, note that some protocols are already considering such a
150 graceful shutdown procedure (e.g., GMPLS in [<a href="./rfc5817" title="&quot;Graceful Shutdown in MPLS and Generalized MPLS Traffic Engineering Networks&quot;">RFC5817</a>]).
151
152 A metric of success is the degree to which such a mechanism
153 eliminates traffic loss during maintenance operations.
154
155 <span class="h2"><a class="selflink" id="section-2" href="#section-2">2</a>. Conventions Used in This Document</span>
156
157 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
158 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
159 document are to be interpreted as described in <a href="./rfc2119">RFC 2119</a> [<a href="./rfc2119" title="&quot;Key words for use in RFCs to Indicate Requirement Levels&quot;">RFC2119</a>].
160
161
162
163
164 <span class="grey">Decraene, et al. Informational [Page 3]</span></pre>
165 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-4" ></span>
166 <span class="grey"><a href="./rfc6198">RFC 6198</a> Reqs for Graceful BGP Shutdown April 2011</span>
167
168
169 <span class="h2"><a class="selflink" id="section-3" href="#section-3">3</a>. Problem Statement</span>
170
171 As per [<a href="./rfc4271" title="&quot;A Border Gateway Protocol 4 (BGP-4)&quot;">RFC4271</a>], when one (or many) BGP session(s) are shut down, a
172 BGP NOTIFICATION message is sent to the peer and the session is then
173 closed. A protocol convergence is then triggered both by the local
174 router and by the peer. Alternate paths to the destination are
175 selected, if known. If those alternate paths are not known prior to
176 the BGP session shutdown, additional BGP convergence steps are
177 required in each AS to search for an alternate path.
178
179 This behavior is not satisfactory in a maintenance situation because
180 the traffic that was directed towards the removed next-hops may be
181 lost until the end of the BGP convergence. As it is a planned
182 operation, a make-before-break solution should be made possible.
183
184 As maintenance operations are frequent in large networks [<a href="#ref-Reliable" title="&quot;Reliable IP Nodes: A prerequisite to profitable IP services&quot;">Reliable</a>],
185 the global availability of the network is significantly impaired by
186 this BGP maintenance issue.
187
188 <span class="h3"><a class="selflink" id="section-3.1" href="#section-3.1">3.1</a>. Example of Undesirable BGP Routing Behavior</span>
189
190 To illustrate these problems, let us consider the following simple
191 example where one customer router "CUST" is dual-attached to two
192 Service Providers' routers, "ASBR1" and "ASBR2".
193
194 ASBR1 and ASBR2 are in the same AS and are owned by the same Service
195 Provider. Both are IBGP clients of the route reflector R1.
196
197 '
198 AS1 ' AS2
199 '
200
201 /-----------ASBR1---
202 / \
203 / \
204 CUST R1
205 \ /
206 Z/z \ /
207 \-----------ASBR2---
208
209 '
210 AS1 ' AS2
211 '
212
213 Figure 1. Dual-Attached Customer
214
215
216
217
218
219
220 <span class="grey">Decraene, et al. Informational [Page 4]</span></pre>
221 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-5" ></span>
222 <span class="grey"><a href="./rfc6198">RFC 6198</a> Reqs for Graceful BGP Shutdown April 2011</span>
223
224
225 Before the maintenance, packets for destination Z/z use the ASBR1-
226 CUST link because R1 selects ASBR1's route based on the IGP cost.
227
228 Let's assume the Service Provider wants to shut down the ASBR1-CUST
229 link for maintenance purposes. Currently, when the shutdown is
230 performed on ASBR1, the following steps are performed:
231
232 1. ASBR1 withdraws its prefix Z/z to its route reflector, R1.
233
234 2. R1 runs its decision process, selects the route from ASBR2, and
235 advertises the new path to ASBR1.
236
237 3. ASBR1 runs its decision process and recovers the reachability
238 of Z/z.
239
240 Traffic is lost at step 1 when ASBR1 looses its route until step 3
241 when it discovers a new path.
242
243 Note that this is a simplified description for illustrative purposes.
244 In a bigger AS, multiple steps of BGP convergence may be required to
245 find and select the best alternate path (e.g., ASBR1 may be chosen
246 based on a higher LOCAL_PREF, hierarchical route reflectors may be
247 used, etc.). When multiple BGP routers are involved and plenty of
248 prefixes are affected, the recovery process can take longer than
249 application requirements.
250
251 <span class="h3"><a class="selflink" id="section-3.2" href="#section-3.2">3.2</a>. Causes of Packet Loss</span>
252
253 The loss of packets during maintenance has two main causes:
254
255 - lack of an alternate path on some routers, and
256
257 - transient routing inconsistency.
258
259 Some routers may lack an alternate path because another router is
260 hiding the backup path. This router can be:
261
262 - a route reflector only propagating its best path.
263
264 - the backup ASBR not advertising the backup path because it
265 prefers the nominal path.
266
267 This lack of knowledge regarding the alternate path is the first
268 target of this requirements document.
269
270 Transient routing inconsistencies happen during IBGP convergence
271 because routers do not simultaneously update their Routing
272 Information Bases (RIBs) and hence do not simultaneously update their
273
274
275
276 <span class="grey">Decraene, et al. Informational [Page 5]</span></pre>
277 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-6" ></span>
278 <span class="grey"><a href="./rfc6198">RFC 6198</a> Reqs for Graceful BGP Shutdown April 2011</span>
279
280
281 Forwarding Information Bases (FIBs) entries. This can lead to
282 forwarding loops, which result in both link congestion and packet
283 drops. The duration of these transient micro-loops is dependent on
284 the IBGP topology (e.g., number of route reflectors between ingress
285 and egress ASBR), implementation differences among router platforms
286 (which result in differences in the time taken to update specific
287 prefix in the FIB), and forwarding mode (hop-by-hop IP forwarding
288 versus tunneling).
289
290 Note that when an IP lookup is only performed on entry to the AS, for
291 example, prior to entry into a tunnel across the AS, micro-loops will
292 not occur. An example of this is when BGP is being used as the
293 routing protocol for MPLS VPN as defined in [<a href="./rfc4364" title="&quot;BGP/MPLS IP Virtual Private Networks (VPNs)&quot;">RFC4364</a>].
294
295 Note that [<a href="./rfc5715" title="&quot;A Framework for Loop-Free Convergence&quot;">RFC5715</a>] defines a framework for loop-free convergence.
296 It has been written in the context of IP fast reroute for link state
297 IGP [<a href="./rfc5714" title="&quot;IP Fast Reroute Framework&quot;">RFC5714</a>], but some concepts are also of interest for BGP
298 convergence.
299
300 <span class="h2"><a class="selflink" id="section-4" href="#section-4">4</a>. Terminology</span>
301
302 g-shut: Graceful shutdown. A method for explicitly notifying the BGP
303 routers that a BGP session (and hence the prefixes learned over that
304 session) is going to be disabled.
305
306 g-noshut: Graceful no shutdown. A method for explicitly notifying
307 the BGP routers that a BGP session (and hence the prefixes learned
308 over that session) is going to be enabled.
309
310 g-shut initiator: the router on which the session(s) shutdown(s) is
311 (are) performed for maintenance.
312
313 g-shut neighbor: a router that peers with the g-shut initiator via
314 (one of) the session(s) undergoing maintenance.
315
316 affected prefixes: a prefix initially reached via the peering link(s)
317 undergoing maintenance.
318
319 affected router: a router reaching an affected prefix via a peering
320 link undergoing maintenance.
321
322 initiator AS: the autonomous system of the g-shut initiator router.
323
324 neighbor AS(es): the autonomous system(s) of the g-shut neighbor
325 router(s).
326
327
328
329
330
331
332 <span class="grey">Decraene, et al. Informational [Page 6]</span></pre>
333 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-7" ></span>
334 <span class="grey"><a href="./rfc6198">RFC 6198</a> Reqs for Graceful BGP Shutdown April 2011</span>
335
336
337 <span class="h2"><a class="selflink" id="section-5" href="#section-5">5</a>. Goals and Requirements</span>
338
339 Currently, when a BGP session of the router under maintenance is shut
340 down, the router removes the routes and then triggers the BGP
341 convergence on its BGP peers by withdrawing its route.
342
343 The goal of BGP graceful shutdown of a (set of) BGP session(s) is to
344 minimize traffic loss during a planned shutdown. Ideally, a solution
345 should reduce this traffic loss to zero.
346
347 Another goal is to minimize and, preferably, to eliminate packet loss
348 when the BGP session is re-established following the maintenance.
349
350 As the event is known in advance, a make-before-break solution can be
351 used in order to initiate the BGP convergence, find and install the
352 alternate paths before the nominal paths are removed. As a result,
353 before the nominal BGP session is shut down, all affected routers
354 learn and use the alternate paths. Those alternate paths are
355 computed by BGP, taking into account the known status of the network,
356 which includes known failures that the network is processing
357 concurrently with the BGP session graceful shutdown and possibly
358 other known graceful shutdowns under way. Therefore, multiple BGP
359 graceful shutdowns overlapping within a short time frame are
360 gracefully handled. Indeed, a given graceful shutdown takes into
361 account all previous ones.
362
363 As a result, provided an alternate path with enough remaining
364 capacity is available, the packets are rerouted before the BGP
365 session termination and fewer packets (possibly none) are lost during
366 the BGP convergence process since, at any time, all routers have a
367 valid path.
368
369 From the above goals, we can derive the following requirements:
370
371 a) A mechanism to advertise the maintenance action to all affected
372 routers is REQUIRED. Such a mechanism may be either implicit or
373 explicit. Note that affected routers can be located both in the
374 local AS and in neighboring ASes. Note also that the
375 maintenance action can either be the shutdown of a BGP session
376 or the establishment of a BGP session.
377
378 The mechanism SHOULD allow BGP routers to minimize and,
379 preferably, eliminate packet loss when a path is removed or
380 advertised. In particular, it SHOULD be ensured that the old
381 path is not removed from the routing tables of the affected
382 routers before the new path is known.
383
384
385
386
387
388 <span class="grey">Decraene, et al. Informational [Page 7]</span></pre>
389 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-8" ></span>
390 <span class="grey"><a href="./rfc6198">RFC 6198</a> Reqs for Graceful BGP Shutdown April 2011</span>
391
392
393 The solution mechanism MUST significantly reduce and, ideally,
394 eliminate packet loss. A trade-off may be made between the
395 degree of packet loss and the simplicity of the solution.
396
397 b) An Internet-wide convergence is OPTIONAL. However, if the
398 initiator AS and the neighbor AS(es) have a backup path, they
399 SHOULD be able to gracefully converge before the nominal path is
400 shut down.
401
402 c) The proposed solution SHOULD be applicable to any kind of BGP
403 sessions (External BGP (EBGP), IBGP, IBGP route reflector
404 client, EBGP confederations, EBGP multi hop, MultiProtocol BGP
405 extension, etc.) and any address family. If a BGP
406 implementation allows the closing or enabling of a subset of
407 Address Family Identifiers (AFIs) carried in an MP-BGP session,
408 this mechanism MAY be applicable to this subset of AFIs.
409
410 Depending on the kind of session, there may be some variations
411 in the proposed solution in order to fulfill the requirements.
412
413 The following cases should be handled in priority:
414
415 - The shutdown of an inter-AS link and therefore the shutdown of
416 an EBGP session;
417
418 - The shutdown of an ASBR and therefore the shutdown of all its
419 BGP sessions.
420
421 Service Providers and platforms implementing a graceful shutdown
422 solution should note that in BGP/MPLS VPN as per [<a href="./rfc4364" title="&quot;BGP/MPLS IP Virtual Private Networks (VPNs)&quot;">RFC4364</a>], the
423 Provider Edge - Customer Edge (PE-CE) routing can be performed
424 by protocols other than BGP (e.g., static routes, RIPv2, OSPF,
425 IS-IS). This is out of scope of this document.
426
427 d) The proposed solution SHOULD NOT change the BGP convergence
428 behavior for the ASes exterior to the maintenance process,
429 namely, ASes other than the initiator AS and its neighbor
430 AS(es).
431
432 e) An incremental deployment on a per-AS or per-BGP session basis
433 MUST be made possible. In case of partial deployment, the
434 proposed solution SHOULD incrementally improve the maintenance
435 process. It should be noted that in an inter-domain relation,
436 one AS may have more incentive to use graceful shutdown than the
437 other. Similarly, in a BGP/MPLS VPN environment, it's much
438 easier to upgrade the PE routers than the CE ones, mainly
439 because there is at least an order of magnitude more CE and CE
440 locations than PE and PE locations. As a consequence, when
441
442
443
444 <span class="grey">Decraene, et al. Informational [Page 8]</span></pre>
445 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-9" ></span>
446 <span class="grey"><a href="./rfc6198">RFC 6198</a> Reqs for Graceful BGP Shutdown April 2011</span>
447
448
449 splitting the cost of the solution between the g-shut initiator
450 and the g-shut neighbor, the solution SHOULD favor a low-cost
451 solution on the neighbor AS side in order to reduce the impact
452 on the g-shut neighbor. Impact should be understood as a
453 generic term that includes first hardware, then software, then
454 configuration upgrade.
455
456 f) Redistribution or advertisement of (static) IP routes into BGP
457 SHOULD also be covered.
458
459 g) The proposed solution MAY be designed in order to avoid
460 transient forwarding loops. Indeed, forwarding loops increase
461 packet transit-delay and may lead to link saturation.
462
463 h) The specific procedure SHOULD end when the BGP session is closed
464 following the g-shut and once the BGP session is gracefully
465 opened following the g-noshut. In the end, once the planned
466 maintenance is finished, the nominal BGP routing MUST be re-
467 established. The duration of the g-shut procedure, and hence
468 the time before the BGP session is safely closed, SHOULD be
469 discussed by the solution document. Examples of possible
470 solutions are the use of a pre-configured timer, the use of a
471 message to signal the end of the BGP convergence, or the
472 monitoring of the traffic on the g-shut interface.
473
474 i) The solution SHOULD be simple and simple to operate. Hence, it
475 MAY only cover a subset of the cases. As a consequence, most of
476 the above requirements are expressed as "SHOULD" rather than
477 "MUST".
478
479 The metrics to evaluate and compare the proposed solutions are:
480
481 - The duration of the remaining loss of connectivity when the
482 BGP session is brought down or up;
483
484 - The applicability to a wide range of BGP and network
485 topologies;
486
487 - The simplicity;
488
489 - The duration of transient forwarding loops;
490
491 - The additional load introduced in BGP (e.g., BGP messages sent
492 to peer routers, peer ASes, the Internet).
493
494
495
496
497
498
499
500 <span class="grey">Decraene, et al. Informational [Page 9]</span></pre>
501 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-10" ></span>
502 <span class="grey"><a href="./rfc6198">RFC 6198</a> Reqs for Graceful BGP Shutdown April 2011</span>
503
504
505 <span class="h2"><a class="selflink" id="section-6" href="#section-6">6</a>. Security Considerations</span>
506
507 At the requirements stage, this graceful shutdown mechanism is not
508 expected to affect the security of the BGP protocol, especially if it
509 can be kept simple. No new sessions are required and the additional
510 ability to signal the graceful shutdown is not expected to bring
511 additional attack vectors, as BGP neighbors already have the ability
512 to send incorrect or misleading information or even shut down the
513 session.
514
515 Security considerations MUST be addressed by the proposed solutions.
516 In particular, they SHOULD address the issues of bogus g-shut
517 messages and how they would affect the network(s), as well as the
518 impact of hiding a g-shut message so that g-shut is not performed.
519
520 The solution SHOULD NOT increase the ability of one AS to selectively
521 influence routing decision in the peer AS (inbound Traffic
522 Engineering) outside of the case of the BGP session shutdown.
523 Otherwise, the peer AS SHOULD have means to detect such behavior.
524
525 <span class="h2"><a class="selflink" id="section-7" href="#section-7">7</a>. References</span>
526
527 <span class="h3"><a class="selflink" id="section-7.1" href="#section-7.1">7.1</a>. Normative References</span>
528
529 [<a id="ref-RFC2119">RFC2119</a>] Bradner, S., "Key words for use in RFCs to Indicate
530 Requirement Levels", <a href="https://www.rfc-editor.org/bcp/bcp14">BCP 14</a>, <a href="./rfc2119">RFC 2119</a>, March 1997.
531
532 [<a id="ref-RFC4271">RFC4271</a>] Rekhter, Y., Ed., Li, T., Ed., and S. Hares, Ed., "A
533 Border Gateway Protocol 4 (BGP-4)", <a href="./rfc4271">RFC 4271</a>, January
534 2006.
535
536 [<a id="ref-RFC4760">RFC4760</a>] Bates, T., Chandra, R., Katz, D., and Y. Rekhter,
537 "Multiprotocol Extensions for BGP-4", <a href="./rfc4760">RFC 4760</a>, January
538 2007.
539
540 [<a id="ref-RFC4456">RFC4456</a>] Bates, T., Chen, E., and R. Chandra, "BGP Route
541 Reflection: An Alternative to Full Mesh Internal BGP
542 (IBGP)", <a href="./rfc4456">RFC 4456</a>, April 2006.
543
544 [<a id="ref-RFC4364">RFC4364</a>] Rosen, E. and Y. Rekhter, "BGP/MPLS IP Virtual Private
545 Networks (VPNs)", <a href="./rfc4364">RFC 4364</a>, February 2006.
546
547 <span class="h3"><a class="selflink" id="section-7.2" href="#section-7.2">7.2</a>. Informative References</span>
548
549 [<a id="ref-RFC5817">RFC5817</a>] Ali, Z., Vasseur, JP., Zamfir, A., and J. Newton,
550 "Graceful Shutdown in MPLS and Generalized MPLS Traffic
551 Engineering Networks", <a href="./rfc5817">RFC 5817</a>, April 2010.
552
553
554
555
556 <span class="grey">Decraene, et al. Informational [Page 10]</span></pre>
557 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-11" ></span>
558 <span class="grey"><a href="./rfc6198">RFC 6198</a> Reqs for Graceful BGP Shutdown April 2011</span>
559
560
561 [<a id="ref-RFC5715">RFC5715</a>] Shand, M. and S. Bryant, "A Framework for Loop-Free
562 Convergence", <a href="./rfc5715">RFC 5715</a>, January 2010.
563
564 [<a id="ref-RFC5714">RFC5714</a>] Shand, M. and S. Bryant, "IP Fast Reroute Framework", <a href="./rfc5714">RFC</a>
565 <a href="./rfc5714">5714</a>, January 2010.
566
567 [<a id="ref-RFC4724">RFC4724</a>] Sangli, S., Chen, E., Fernando, R., Scudder, J., and Y.
568 Rekhter, "Graceful Restart Mechanism for BGP", <a href="./rfc4724">RFC 4724</a>,
569 January 2007.
570
571 [<a id="ref-Reliable">Reliable</a>] Network Strategy Partners, LLC. "Reliable IP Nodes: A
572 prerequisite to profitable IP services", November 2002.
573 <a href="http://www.nspllc.com/NewPages/Reliable_IP_Nodes.pdf">http://www.nspllc.com/NewPages/Reliable_IP_Nodes.pdf</a>
574
575 Acknowledgments
576
577 The authors would like to thank Nicolas Dubois, Benoit Fondeviole,
578 Christian Jacquenet, Olivier Bonaventure, Steve Uhlig, Xavier Vinet,
579 Vincent Gillet, Jean-Louis le Roux, Pierre Alain Coste, and Ronald
580 Bonica for their useful discussions on this subject, review, and
581 comments.
582
583 This document has been partly sponsored by the European project IST
584 AGAVE.
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612 <span class="grey">Decraene, et al. Informational [Page 11]</span></pre>
613 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-12" ></span>
614 <span class="grey"><a href="./rfc6198">RFC 6198</a> Reqs for Graceful BGP Shutdown April 2011</span>
615
616
617 <span class="h2"><a class="selflink" id="appendix-A" href="#appendix-A">Appendix A</a>. Reference BGP Topologies</span>
618
619 This section describes some frequent BGP topologies used both within
620 the AS (IBGP) and between ASes (EBGP). Solutions should be
621 applicable to the following topologies and their combinations.
622
623 <span class="h3"><a class="selflink" id="appendix-A.1" href="#appendix-A.1">A.1</a>. EBGP Topologies</span>
624
625 This section describes some frequent BGP topologies used between
626 ASes. In each figure, a line represents a BGP session.
627
628 <span class="h4"><a class="selflink" id="appendix-A.1.1" href="#appendix-A.1.1">A.1.1</a>. One ASBR in AS1 Connected to Two ASBRs in the Neighboring AS2</span>
629
630 In this topology, we have an asymmetric protection scheme between AS1
631 and AS2:
632
633 - On the AS2 side, two different routers are used to connect to
634 AS1.
635
636 - On the AS1 side, one single router with two BGP sessions is
637 used.
638
639 '
640 AS1 ' AS2
641 '
642 /----------- ASBR2.1
643 / '
644 / '
645 ASBR1.1 '
646 \ '
647 \ '
648 \----------- ASBR2.2
649 '
650 '
651 AS1 ' AS2
652 '
653
654 Figure 2. EBGP Topology with Redundant ASBR in One of the ASes
655
656 BGP graceful shutdown is expected to be applicable for the
657 maintenance of:
658
659 - one of the routers of AS2;
660
661 - one link between AS1 and AS2, performed either on an AS1 or AS2
662 router.
663
664
665
666
667
668 <span class="grey">Decraene, et al. Informational [Page 12]</span></pre>
669 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-13" ></span>
670 <span class="grey"><a href="./rfc6198">RFC 6198</a> Reqs for Graceful BGP Shutdown April 2011</span>
671
672
673 Note that in the case of maintenance of the whole router, all its BGP
674 sessions need to be gracefully shutdown at the beginning of the
675 maintenance and gracefully brought up at the end of the maintenance.
676
677 <span class="h4"><a class="selflink" id="appendix-A.1.2" href="#appendix-A.1.2">A.1.2</a>. Two ASBRs in AS1 Connected to Two ASBRs in AS2</span>
678
679 In this topology, we have a symmetric protection scheme between AS1
680 and AS2: on both sides, two different routers are used to connect AS1
681 to AS2.
682
683 '
684 AS1 ' AS2
685 '
686 ASBR1.1----------- ASBR2.1
687 '
688 '
689 '
690 '
691 '
692 ASBR1.2----------- ASBR2.2
693 '
694 AS1 ' AS2
695 '
696
697 Figure 3. EBGP Topology with Redundant ASBRs in Both ASes
698
699 BGP graceful shutdown is expected to be applicable for the
700 maintenance of:
701
702 - any of the ASBR routers (in AS1 or AS2);
703
704 - one link between AS1 and AS2, performed either on an AS1 or AS2
705 router.
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724 <span class="grey">Decraene, et al. Informational [Page 13]</span></pre>
725 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-14" ></span>
726 <span class="grey"><a href="./rfc6198">RFC 6198</a> Reqs for Graceful BGP Shutdown April 2011</span>
727
728
729 <span class="h4"><a class="selflink" id="appendix-A.1.3" href="#appendix-A.1.3">A.1.3</a>. Two ASBRs in AS2 Each Connected to Two Different ASes</span>
730
731 In this topology, at least three ASes are involved.
732
733 '
734 AS1 ' AS2
735 '
736 ASBR1.1----------- ASBR2.1
737 | '
738 | '
739 '''''|''''''''''
740 | '
741 | '
742 ASBR3.1----------- ASBR2.2
743 '
744 AS3 ' AS2
745
746 Figure 4. EBGP Topology of a Dual-Homed Customer
747
748 As the requirement expressed in <a href="#section-5">Section 5</a> is to advertise the
749 maintenance only within the initiator and neighbor ASes, not
750 Internet-wide, BGP graceful shutdown solutions may not be applicable
751 to this topology. Depending on which routes are exchanged between
752 these ASes, some protection for some of the traffic may be possible.
753
754 For instance, if ASBR2.2 performs a maintenance affecting ASBR3.1,
755 then ASBR3.1 will be notified. However, ASBR1.1 may not be notified
756 of the maintenance of the EBGP session between ASBR3.1 and ASBR2.2.
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780 <span class="grey">Decraene, et al. Informational [Page 14]</span></pre>
781 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-15" ></span>
782 <span class="grey"><a href="./rfc6198">RFC 6198</a> Reqs for Graceful BGP Shutdown April 2011</span>
783
784
785 <span class="h3"><a class="selflink" id="appendix-A.2" href="#appendix-A.2">A.2</a>. IBGP Topologies</span>
786
787 This section describes some frequent BGP topologies used within an
788 AS. In each figure, a line represents a BGP session.
789
790 <span class="h4"><a class="selflink" id="appendix-A.2.1" href="#appendix-A.2.1">A.2.1</a>. IBGP Full-Mesh</span>
791
792 In this topology, we have a full-mesh of IBGP sessions:
793
794 P1 ----- P2
795 | \ / |
796 | \ / |
797 | \/ | AS1
798 | /\ |
799 | / \ |
800 | / \ |
801 ASBR1.1--ASBR1.2
802 \ /
803 \ /
804 ''''''\'''/''''''''''''
805 \ / AS2
806 ASBR2.1
807
808 Figure 5. IBGP Full-Mesh
809
810 When the session between ASBR1.1 and ASBR2.1 is gracefully shut down,
811 it is required that all affected routers of AS1 reroute traffic to
812 ASBR1.2 before the session between ASBR1.1 and ASBR2.1 is shut down.
813
814 Similarly, when the session between ASBR1.1 and ASBR2.1 is gracefully
815 brought up, all affected routers of AS1 preferring ASBR1.1 over
816 ASBR1.2 need to reroute traffic to ASBR1.1 before the less preferred
817 path through ASBR1.2 is possibly withdrawn.
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836 <span class="grey">Decraene, et al. Informational [Page 15]</span></pre>
837 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-16" ></span>
838 <span class="grey"><a href="./rfc6198">RFC 6198</a> Reqs for Graceful BGP Shutdown April 2011</span>
839
840
841 <span class="h4"><a class="selflink" id="appendix-A.2.2" href="#appendix-A.2.2">A.2.2</a>. Route Reflector</span>
842
843 In this topology, route reflectors are used to limit the number of
844 IBGP sessions. There is a single level of route reflectors and the
845 route reflectors are fully meshed.
846
847 P1 (RR)-- P2 (RR)
848 | \ / |
849 | \ / |
850 | \ / | AS1
851 | \/ |
852 | /\ |
853 | / \ |
854 | / \ |
855 | / \ |
856 ASBR1.1 ASBR1.2
857 \ /
858 \ /
859 ''''''\''''''/''''''''''''
860 \ /
861 \ / AS2
862 ASBR2.1
863
864 Figure 6. Route Reflector
865
866 When the session between ASBR1.1 and ASBR2.1 is gracefully shut down,
867 all BGP routers of AS1 need to reroute traffic to ASBR1.2 before the
868 session between ASBR1.1 and ASBR2.1 is shut down.
869
870 Similarly, when the session between ASBR1.1 and ASBR2.1 is gracefully
871 brought up, all affected routers of AS1 preferring ASBR1.1 over
872 ASBR1.2 need to reroute traffic to ASBR1.1 before the less preferred
873 path through ASBR1.2 is possibly withdrawn.
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892 <span class="grey">Decraene, et al. Informational [Page 16]</span></pre>
893 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-17" ></span>
894 <span class="grey"><a href="./rfc6198">RFC 6198</a> Reqs for Graceful BGP Shutdown April 2011</span>
895
896
897 <span class="h4"><a class="selflink" id="appendix-A.2.3" href="#appendix-A.2.3">A.2.3</a>. Hierarchical Route Reflector</span>
898
899 In this topology, hierarchical route reflectors are used to limit the
900 number of IBGP sessions. There could be more than two levels of
901 route reflectors and the top-level route reflectors are fully meshed.
902
903
904 P1 (RR) -------- P2 (RR)
905 | |
906 | |
907 | | AS1
908 | |
909 | |
910
911 P3 (RR) P4 (RR)
912 | |
913 | |
914 | | AS1
915 | |
916 | |
917 ASBR1.1 ASBR1.2
918 \ /
919 \ /
920 ''''''\'''''''''/''''''''''''
921 \ /
922 \ / AS2
923 ASBR2.1
924
925 Figure 7. Hierarchical Route Reflector
926
927 When the session between ASBR1.1 and ASBR2.1 is gracefully shut down,
928 all BGP routers of AS1 need to reroute traffic to ASBR1.2 before the
929 session between ASBR1.1 and ASBR2.1 is shut down.
930
931 Similarly, when the session between ASBR1.1 and ASBR2.1 is gracefully
932 brought up, all affected routers of AS1 preferring ASBR1.1 over
933 ASBR1.2 need to reroute traffic to ASBR1.1 before the less preferred
934 path through ASBR1.2 is possibly withdrawn.
935
936 <span class="h4"><a class="selflink" id="appendix-A.2.4" href="#appendix-A.2.4">A.2.4</a>. Confederations</span>
937
938 In this topology, a confederation of ASes is used to limit the number
939 of IBGP sessions. Moreover, RRs may be present in the member ASes of
940 the confederation.
941
942
943
944
945
946
947
948 <span class="grey">Decraene, et al. Informational [Page 17]</span></pre>
949 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-18" ></span>
950 <span class="grey"><a href="./rfc6198">RFC 6198</a> Reqs for Graceful BGP Shutdown April 2011</span>
951
952
953 Confederations may be run with different sub-options. Regarding the
954 IGP, each member AS can run its own IGP or they can all share the
955 same IGP. Regarding BGP, LOCAL_PREF may or may not cross the member
956 AS boundaries.
957
958 A solution should support the graceful shutdown and graceful bringing
959 up of EBGP sessions between member ASes in the confederation in
960 addition to the graceful shutdown and graceful bringing up of EBGP
961 sessions between a member-AS and an AS outside of the confederation.
962
963 ASBR1C.1 ---------- ASBR1C.2
964 | |
965 | |
966 | AS1C |
967 | |
968 | |
969 """|"""""""""""""""""""|"""
970 | " |
971 ASBR1A.2 " ASBR1B.2
972 | " |
973 | " |
974 | AS1A " AS1B | AS1
975 | " |
976 | " |
977 ASBR1A.1 " ASBR1B.1
978 \ " /
979 \ " /
980 ''''''\'''''''''''''/''''''''''''
981 \ /
982 \ / AS2
983 ASBR2.1
984
985 Figure 8. Confederation
986
987 In the above figure, member ASes AS1A, AS1B, and AS1C belong to a
988 confederation of ASes in AS1. AS1A and AS1B are connected to AS2.
989
990 In normal operation, for the traffic toward AS2:
991
992 - AS1A sends the traffic directly to AS2 through ASBR1A.1.
993
994 - AS1B sends the traffic directly to AS2 through ASBR1B.1.
995
996 - AS1C load balances the traffic between AS1A and AS1B.
997
998 When the session between ASBR1A.1 and ASBR2.1 is gracefully shut
999 down, all BGP routers of AS1 need to reroute traffic to ASBR1B.1
1000 before the session between ASBR1A.1 and ASBR2.1 is shut down.
1001
1002
1003
1004 <span class="grey">Decraene, et al. Informational [Page 18]</span></pre>
1005 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-19" ></span>
1006 <span class="grey"><a href="./rfc6198">RFC 6198</a> Reqs for Graceful BGP Shutdown April 2011</span>
1007
1008
1009 Similarly, when the session between ASBR1A.1 and ASBR2.1 is
1010 gracefully brought up, all affected routers of AS1 preferring
1011 ASBR1A.1 over ASBR1B.1 need to reroute traffic to ASBR1A.1 before the
1012 less preferred path through ASBR1B.1 is possibly withdrawn.
1013
1014 <span class="h3"><a class="selflink" id="appendix-A.3" href="#appendix-A.3">A.3</a>. Routing Decisions</span>
1015
1016 Here we describe some routing engineering choices that are frequently
1017 used in ASes and that should be supported by the solution.
1018
1019 <span class="h4"><a class="selflink" id="appendix-A.3.1" href="#appendix-A.3.1">A.3.1</a>. Hot Potato (IGP Cost)</span>
1020
1021 The ingress router selects the nominal egress ASBR (AS exit point)
1022 based on the IGP cost to reach the BGP next-hop.
1023
1024 <span class="h4"><a class="selflink" id="appendix-A.3.2" href="#appendix-A.3.2">A.3.2</a>. Cold Potato (BGP LOCAL_PREF)</span>
1025
1026 The ingress router selects the nominal egress ASBR based on the BGP
1027 LOCAL_PREF value set and advertised by the exit point.
1028
1029 <span class="h4"><a class="selflink" id="appendix-A.3.3" href="#appendix-A.3.3">A.3.3</a>. Cold Potato (BGP Preference Set on Ingress)</span>
1030
1031 The ingress router selects the nominal egress ASBR based on
1032 preconfigured policy information. (Typically, this is done by
1033 locally setting the BGP LOCAL_PREF based on the BGP communities
1034 attached on the routes).
1035
1036 As per [<a href="./rfc4271" title="&quot;A Border Gateway Protocol 4 (BGP-4)&quot;">RFC4271</a>], note that if tunnels are not used to forward
1037 packets between the ingress and egress ASBR; this can lead to
1038 persistent forwarding loops.
1039
1040
1041
1042
1043
1044
1045
1046
1047
1048
1049
1050
1051
1052
1053
1054
1055
1056
1057
1058
1059
1060 <span class="grey">Decraene, et al. Informational [Page 19]</span></pre>
1061 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-20" ></span>
1062 <span class="grey"><a href="./rfc6198">RFC 6198</a> Reqs for Graceful BGP Shutdown April 2011</span>
1063
1064
1065 Authors' Addresses
1066
1067 Bruno Decraene
1068 France Telecom
1069 38-40 rue du General Leclerc
1070 92794 Issy Moulineaux cedex 9
1071 France
1072 EMail: [email protected]
1073
1074 Pierre Francois
1075 Universite catholique de Louvain
1076 Place Ste Barbe, 2
1077 Louvain-la-Neuve 1348
1078 BE
1079 EMail: [email protected]
1080
1081 Cristel Pelsser
1082 Internet Initiative Japan
1083 Jinbocho Mitsui Building
1084 1-105 Kanda jinbo-cho
1085 Chiyoda-ku, Tokyo 101-0051
1086 Japan
1087 EMail: [email protected]
1088
1089 Zubair Ahmad
1090 Orange Business Services
1091 13775 McLearen Road, Oak Hill VA 20171
1092 USA
1093 EMail: [email protected]
1094
1095 Antonio Jose Elizondo Armengol
1096 Division de Analisis Tecnologicos
1097 Technology Analysis Division
1098 Telefonica I+D
1099 C/ Emilio Vargas 6
1100 28043, Madrid
1101 EMail: [email protected]
1102
1103 Tomonori Takeda
1104 NTT Corporation
1105 9-11, Midori-Cho 3 Chrome
1106 Musashino-Shi, Tokyo 180-8585
1107 Japan
1108 EMail: [email protected]
1109
1110
1111
1112
1113
1114
1115
1116 Decraene, et al. Informational [Page 20]
1117 </pre>