Debian Codegraph @

rfc8039.html

raw
1 <pre>Internet Engineering Task Force (IETF) A. Shpiner
2 Request for Comments: 8039 Mellanox
3 Category: Experimental R. Tse
4 ISSN: 2070-1721 Microsemi
5 C. Schelp
6 Oracle
7 T. Mizrahi
8 Marvell
9 December 2016
10
11
12 <span class="h1">Multipath Time Synchronization</span>
13
14 Abstract
15
16 Clock synchronization protocols are very widely used in IP-based
17 networks. The Network Time Protocol (NTP) has been commonly deployed
18 for many years, and the last few years have seen an increasingly
19 rapid deployment of the Precision Time Protocol (PTP). As time-
20 sensitive applications evolve, clock accuracy requirements are
21 becoming increasingly stringent, requiring the time synchronization
22 protocols to provide high accuracy. This memo describes a multipath
23 approach to PTP and NTP over IP networks, allowing the protocols to
24 run concurrently over multiple communication paths between the master
25 and slave clocks, without modifying these protocols. The multipath
26 approach can significantly contribute to clock accuracy, security,
27 and fault tolerance. The multipath approach that is presented in
28 this document enables backward compatibility with nodes that do not
29 support the multipath functionality.
30
31 Status of This Memo
32
33 This document is not an Internet Standards Track specification; it is
34 published for examination, experimental implementation, and
35 evaluation.
36
37 This document defines an Experimental Protocol for the Internet
38 community. This document is a product of the Internet Engineering
39 Task Force (IETF). It represents the consensus of the IETF
40 community. It has received public review and has been approved for
41 publication by the Internet Engineering Steering Group (IESG). Not
42 all documents approved by the IESG are a candidate for any level of
43 Internet Standard; see <a href="./rfc7841#section-2">Section&nbsp;2 of RFC 7841</a>.
44
45 Information about the current status of this document, any errata,
46 and how to provide feedback on it may be obtained at
47 <a href="http://www.rfc-editor.org/info/rfc8039">http://www.rfc-editor.org/info/rfc8039</a>.
48
49
50
51
52 <span class="grey">Shpiner, et al. Experimental [Page 1]</span></pre>
53 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-2" ></span>
54 <span class="grey"><a href="./rfc8039">RFC 8039</a> Multipath Time Synchronization December 2016</span>
55
56
57 Copyright Notice
58
59 Copyright (c) 2016 IETF Trust and the persons identified as the
60 document authors. All rights reserved.
61
62 This document is subject to <a href="https://www.rfc-editor.org/bcp/bcp78">BCP 78</a> and the IETF Trust's Legal
63 Provisions Relating to IETF Documents
64 (<a href="http://trustee.ietf.org/license-info">http://trustee.ietf.org/license-info</a>) in effect on the date of
65 publication of this document. Please review these documents
66 carefully, as they describe your rights and restrictions with respect
67 to this document. Code Components extracted from this document must
68 include Simplified BSD License text as described in Section 4.e of
69 the Trust Legal Provisions and are provided without warranty as
70 described in the Simplified BSD License.
71
72 Table of Contents
73
74 <a href="#section-1">1</a>. Introduction ....................................................<a href="#page-3">3</a>
75 <a href="#section-2">2</a>. Conventions Used in This Document ...............................<a href="#page-4">4</a>
76 <a href="#section-2.1">2.1</a>. Abbreviations ..............................................<a href="#page-4">4</a>
77 <a href="#section-2.2">2.2</a>. Terminology ................................................<a href="#page-4">4</a>
78 <a href="#section-3">3</a>. Multiple Paths in IP Networks ...................................<a href="#page-5">5</a>
79 <a href="#section-3.1">3.1</a>. Load Balancing .............................................<a href="#page-5">5</a>
80 <a href="#section-3.2">3.2</a>. Using Multiple Paths Concurrently ..........................<a href="#page-5">5</a>
81 <a href="#section-3.3">3.3</a>. Two-Way Paths ..............................................<a href="#page-5">5</a>
82 <a href="#section-4">4</a>. Solution Overview ...............................................<a href="#page-6">6</a>
83 <a href="#section-4.1">4.1</a>. Path Configuration and Identification ......................<a href="#page-6">6</a>
84 <a href="#section-4.2">4.2</a>. Combining ..................................................<a href="#page-6">6</a>
85 <a href="#section-5">5</a>. Multipath Time Synchronization over IP Networks .................<a href="#page-7">7</a>
86 <a href="#section-5.1">5.1</a>. Overview ...................................................<a href="#page-7">7</a>
87 <a href="#section-5.2">5.2</a>. Single-Ended Multipath Synchronization .....................<a href="#page-8">8</a>
88 5.2.1. Single-Ended MPPTP Synchronization Message
89 Exchange ............................................<a href="#page-8">8</a>
90 5.2.2. Single-Ended MPNTP Synchronization Message
91 Exchange ............................................<a href="#page-9">9</a>
92 <a href="#section-5.3">5.3</a>. Dual-Ended Multipath Synchronization ......................<a href="#page-10">10</a>
93 <a href="#section-5.3.1">5.3.1</a>. Dual-Ended MPPTP Synchronization Message Exchange ..10
94 <a href="#section-5.3.2">5.3.2</a>. Dual-Ended MPNTP Synchronization Message Exchange ..11
95 <a href="#section-5.4">5.4</a>. Using Traceroute for Path Discovery .......................<a href="#page-12">12</a>
96 <a href="#section-5.5">5.5</a>. Using Unicast Discovery for MPPTP .........................<a href="#page-13">13</a>
97 <a href="#section-6">6</a>. Combining Algorithm ............................................<a href="#page-13">13</a>
98 <a href="#section-7">7</a>. Security Considerations ........................................<a href="#page-14">14</a>
99 <a href="#section-8">8</a>. Scope of the Experiment ........................................<a href="#page-14">14</a>
100 <a href="#section-9">9</a>. References .....................................................<a href="#page-15">15</a>
101 <a href="#section-9.1">9.1</a>. Normative References ......................................<a href="#page-15">15</a>
102 <a href="#section-9.2">9.2</a>. Informative References ....................................<a href="#page-15">15</a>
103 Acknowledgments ...................................................<a href="#page-17">17</a>
104 Authors' Addresses ................................................<a href="#page-17">17</a>
105
106
107
108 <span class="grey">Shpiner, et al. Experimental [Page 2]</span></pre>
109 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-3" ></span>
110 <span class="grey"><a href="./rfc8039">RFC 8039</a> Multipath Time Synchronization December 2016</span>
111
112
113 <span class="h2"><a class="selflink" id="section-1" href="#section-1">1</a>. Introduction</span>
114
115 The two most common time synchronization protocols in IP networks are
116 (1) the Network Time Protocol [<a href="#ref-NTP" title="&quot;Network Time Protocol Version 4: Protocol and Algorithms Specification&quot;">NTP</a>] and (2) the Precision Time
117 Protocol (PTP) as defined in the IEEE 1588 standard [<a href="#ref-IEEE1588" title="&quot;IEEE Standard for a Precision Clock Synchronization Protocol for Networked Measurement and Control Systems&quot;">IEEE1588</a>].
118
119 The accuracy of the time synchronization protocols directly depends
120 on the stability and the symmetry of propagation delays in both
121 directions between the master and slave clocks. Depending on the
122 nature of the underlying network, time synchronization protocol
123 packets can be subject to variable network latency or path asymmetry
124 (e.g., [<a href="#ref-ASYMMETRY">ASYMMETRY</a>] [<a href="#ref-ASYMMETRY2">ASYMMETRY2</a>]). As time-sensitive applications
125 evolve, accuracy requirements are becoming increasingly stringent.
126 Using a single network path in a clock synchronization protocol
127 closely ties the slave clock accuracy to the behavior of the specific
128 path, which may suffer from temporal congestion, faults, or malicious
129 attacks. Relying on multiple clock servers, as in NTP, solves these
130 problems but requires active maintenance of multiple accurate sources
131 in the network, which is not always possible. The usage of
132 Transparent Clocks (TCs) in PTP solves the congestion problem by
133 eliminating the queuing time from the delay calculations but does not
134 address security or fault-tolerance aspects.
135
136 ____
137 ______/ \_____
138 ___/ \____
139 ____/ \
140 ____ / path 1 / ___
141 / \ / ________________________ \ / \
142 /Master\____\___/ \____\________/Slave\
143 \Clock / / \________ _______________/ \ \Clock/
144 \____/ \ / \__ /
145 \____ path 2 __/
146 \_______ ______/
147 \_________/
148
149 Figure 1: Multipath Connection
150
151 Since master and slave clocks are often connected through more than
152 one path in the network, as shown in Figure 1, [<a href="#ref-SLAVEDIV" title="&quot;Slave Diversity: Using Multiple Paths to Improve the Accuracy of Clock Synchronization Protocols&quot;">SLAVEDIV</a>] suggested
153 that a time synchronization protocol can be run over multiple paths,
154 providing several advantages. First, it can significantly increase
155 the clock accuracy as shown in [<a href="#ref-SLAVEDIV" title="&quot;Slave Diversity: Using Multiple Paths to Improve the Accuracy of Clock Synchronization Protocols&quot;">SLAVEDIV</a>]. Second, this approach
156 provides additional security, allowing the mitigation of
157 man-in-the-middle attacks against the time synchronization protocol
158 [<a href="#ref-DELAY-ATT">DELAY-ATT</a>]. Third, using multiple paths concurrently provides an
159 inherent failure protection mechanism.
160
161
162
163
164 <span class="grey">Shpiner, et al. Experimental [Page 3]</span></pre>
165 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-4" ></span>
166 <span class="grey"><a href="./rfc8039">RFC 8039</a> Multipath Time Synchronization December 2016</span>
167
168
169 This document introduces Multipath PTP (MPPTP) and Multipath NTP
170 (MPNTP). The functionality of the multipath approach is defined at
171 the network layer and does not require any changes in PTP or NTP.
172
173 MPPTP and MPNTP are defined over IP networks. As IP networks
174 typically combine ECMP routing, this property is leveraged for the
175 multiple paths used in MPPTP and MPNTP. The key property of the
176 multipath approach is that clocks in the network can use more than
177 one IP address. Each {master IP, slave IP} address pair defines a
178 path. Depending on the network topology and configuration, the IP
179 combination pairs can form multiple diverse paths used by the
180 multipath synchronization protocols. It has been shown [<a href="#ref-MULTI" title="&quot;Multi-path Time Protocols&quot;">MULTI</a>] that
181 using multiple IP addresses over the wide Internet indeed allows two
182 endpoints to attain multiple diverse paths.
183
184 This document introduces two variants of the multipath approach:
185 (1) a variant that requires both master and slave nodes to support
186 the multipath functionality, referred to as the dual-ended variant,
187 and (2) a backward-compatible variant that allows a multipath clock
188 to connect to a conventional single-path clock, referred to as the
189 single-ended variant.
190
191 <span class="h2"><a class="selflink" id="section-2" href="#section-2">2</a>. Conventions Used in This Document</span>
192
193 <span class="h3"><a class="selflink" id="section-2.1" href="#section-2.1">2.1</a>. Abbreviations</span>
194
195 BMC Best Master Clock [<a href="#ref-IEEE1588" title="&quot;IEEE Standard for a Precision Clock Synchronization Protocol for Networked Measurement and Control Systems&quot;">IEEE1588</a>]
196
197 ECMP Equal-Cost Multipath
198
199 LAN Local Area Network
200
201 MPNTP Multipath Network Time Protocol
202
203 MPPTP Multipath Precision Time Protocol
204
205 NTP Network Time Protocol [<a href="#ref-NTP" title="&quot;Network Time Protocol Version 4: Protocol and Algorithms Specification&quot;">NTP</a>]
206
207 PTP Precision Time Protocol [<a href="#ref-IEEE1588" title="&quot;IEEE Standard for a Precision Clock Synchronization Protocol for Networked Measurement and Control Systems&quot;">IEEE1588</a>]
208
209 <span class="h3"><a class="selflink" id="section-2.2" href="#section-2.2">2.2</a>. Terminology</span>
210
211 In the NTP terminology, a time synchronization protocol is run
212 between a client and a server, while PTP uses the terms 'master' and
213 'slave'. Throughout this document, the sections that refer to both
214 PTP and NTP generically use the terms 'master' and 'slave'.
215
216
217
218
219
220 <span class="grey">Shpiner, et al. Experimental [Page 4]</span></pre>
221 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-5" ></span>
222 <span class="grey"><a href="./rfc8039">RFC 8039</a> Multipath Time Synchronization December 2016</span>
223
224
225 <span class="h2"><a class="selflink" id="section-3" href="#section-3">3</a>. Multiple Paths in IP Networks</span>
226
227 <span class="h3"><a class="selflink" id="section-3.1" href="#section-3.1">3.1</a>. Load Balancing</span>
228
229 Traffic sent across IP networks is often load-balanced across
230 multiple paths. The load-balancing decisions are typically based on
231 packet header fields: source and destination addresses, Layer 4
232 ports, the Flow Label field in IPv6, etc.
233
234 Three common load-balancing criteria are per-destination, per-flow,
235 and per-packet. The per-destination load balancers take a
236 load-balancing decision based on the destination IP address.
237 Per-flow load balancers use various fields in the packet header,
238 e.g., IP addresses and Layer 4 ports, for the load-balancing
239 decision. Per-packet load balancers use flow-blind techniques such
240 as round-robin without basing the choice on the packet content.
241
242 <span class="h3"><a class="selflink" id="section-3.2" href="#section-3.2">3.2</a>. Using Multiple Paths Concurrently</span>
243
244 To utilize the diverse paths that traverse per-destination
245 load balancers or per-flow load balancers, the packet transmitter can
246 vary the IP addresses in the packet header. The analysis in [<a href="#ref-PARIS2" title="&quot;Measuring Multipath Routing in the Internet&quot;">PARIS2</a>]
247 shows that a significant majority of the flows on the Internet
248 traverse per-destination or per-flow load balancing. It presents
249 statistics that 72% of the flows traverse per-destination
250 load balancing and 39% of the flows traverse per-flow load balancing,
251 while only a negligible part of the flows traverse per-packet
252 load balancing. These statistics show that the vast majority of the
253 traffic on the Internet is load-balanced based on packet header
254 fields.
255
256 The approaches in this document are based on varying the source and
257 destination IP addresses in the packet header. Possible extensions
258 have been considered that also vary the UDP ports. However, some of
259 the existing implementations of PTP and NTP use fixed UDP port values
260 in both the source and destination UDP port fields and thus do not
261 allow this approach.
262
263 <span class="h3"><a class="selflink" id="section-3.3" href="#section-3.3">3.3</a>. Two-Way Paths</span>
264
265 A key property of IP networks is that packets forwarded from A to B
266 do not necessarily traverse the same path as packets from B to A.
267 Thus, we define a two-way path for a master-slave connection as a
268 pair of one-way paths: the first from master to slave and the second
269 from slave to master.
270
271
272
273
274
275
276 <span class="grey">Shpiner, et al. Experimental [Page 5]</span></pre>
277 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-6" ></span>
278 <span class="grey"><a href="./rfc8039">RFC 8039</a> Multipath Time Synchronization December 2016</span>
279
280
281 If possible, a traffic engineering approach can be used to verify
282 that time synchronization traffic is always forwarded through
283 bidirectional two-way paths, i.e., that each two-way path uses the
284 same route in the forward and reverse directions, thus allowing
285 propagation time symmetry. However, in the general case, two-way
286 paths do not necessarily use the same path for the forward and
287 reverse directions.
288
289 <span class="h2"><a class="selflink" id="section-4" href="#section-4">4</a>. Solution Overview</span>
290
291 The multipath time synchronization protocols we present here are
292 comprised of two building blocks: one is the path configuration and
293 identification, and the other is the algorithm used by the slave to
294 combine the information received from the various paths.
295
296 <span class="h3"><a class="selflink" id="section-4.1" href="#section-4.1">4.1</a>. Path Configuration and Identification</span>
297
298 The master and slave clocks must be able to determine the path of
299 transmitted protocol packets and to identify the path of incoming
300 protocol packets. A path is determined by a {master IP, slave IP}
301 address pair. The synchronization protocol message exchange is run
302 independently through each path.
303
304 Each IP address pair defines a two-way path and thus allows the
305 clocks to bind a transmitted packet to a specific path or to identify
306 the path of an incoming packet.
307
308 If possible, the routing tables across the network should be
309 configured with multiple traffic-engineered paths between the pair of
310 clocks. By carefully configuring the routers in such networks, it is
311 possible to create diverse paths for each of the IP address pairs
312 between two clocks in the network. However, in public and provider
313 networks, the load-balancing behavior is hidden from the end users.
314 In this case, the actual number of paths may be less than the number
315 of IP address pairs, since some of the address pairs may share common
316 paths.
317
318 <span class="h3"><a class="selflink" id="section-4.2" href="#section-4.2">4.2</a>. Combining</span>
319
320 Various methods can be used for combining the time information
321 received from the different paths. The output of the combining
322 algorithm is the accurate time offset. Combining methods are further
323 discussed in <a href="#section-6">Section 6</a>.
324
325
326
327
328
329
330
331
332 <span class="grey">Shpiner, et al. Experimental [Page 6]</span></pre>
333 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-7" ></span>
334 <span class="grey"><a href="./rfc8039">RFC 8039</a> Multipath Time Synchronization December 2016</span>
335
336
337 <span class="h2"><a class="selflink" id="section-5" href="#section-5">5</a>. Multipath Time Synchronization over IP Networks</span>
338
339 <span class="h3"><a class="selflink" id="section-5.1" href="#section-5.1">5.1</a>. Overview</span>
340
341 This section presents two variants of MPPTP and MPNTP: single-ended
342 multipath time synchronization and dual-ended multipath time
343 synchronization. In the first variant, the multipath approach is
344 only implemented by the slave, and the master is not aware of its
345 usage. In the second variant, all clocks use multiple paths.
346
347 The dual-ended variant provides higher path diversity by using
348 multiple IP addresses at both ends, the master and slave, while the
349 single-ended variant only uses multiple addresses at the slave.
350 Consequently, the single-ended approach can interoperate with
351 existing implementations that do not use multiple paths. The
352 dual-ended and single-ended approaches can coexist in the same
353 network; each slave selects the connection(s) it wants to make with
354 the available masters. A dual-ended slave could switch to
355 single-ended mode if it does not see any dual-ended masters
356 available. A single-ended slave could connect to a single IP address
357 of a dual-ended master.
358
359 Multipath time synchronization, in both variants, requires clocks to
360 use multiple IP addresses. Using multiple IP addresses introduces a
361 trade-off. A large number of IP addresses allows a large number of
362 diverse paths, providing the advantages of slave diversity discussed
363 in <a href="#section-1">Section 1</a>. On the other hand, a large number of IP addresses is
364 more costly, requires the network topology to be more redundant, and
365 exacts extra management overhead.
366
367 If possible, the set of IP addresses for each clock should be chosen
368 in a way that enables the establishment of paths that are the most
369 different. If the load-balancing rules in the network are known, it
370 is possible to choose the IP addresses in a way that enforces path
371 diversity. However, even if the load-balancing scheme is not known,
372 a careful choice of the IP addresses can increase the probability of
373 path diversity. For example, choosing multiple addresses with
374 different prefixes is likely to produce higher path diversity, as BGP
375 routers are more likely to route these different prefixes through
376 different routes.
377
378 The use of Network Address Translation (NAT) may significantly reduce
379 the effectiveness of multipath synchronization in some cases. For
380 example, if a master uses multiple IP addresses that are translated
381 to a single IP address, the path diversity can be dramatically
382 reduced compared to a network that does not use NAT. Thus, path
383
384
385
386
387
388 <span class="grey">Shpiner, et al. Experimental [Page 7]</span></pre>
389 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-8" ></span>
390 <span class="grey"><a href="./rfc8039">RFC 8039</a> Multipath Time Synchronization December 2016</span>
391
392
393 discovery should be used to identify the possible paths between the
394 master and slave. Path discovery is further discussed in
395 <a href="#section-5.4">Section 5.4</a>.
396
397 The concept of using multiple IP addresses or multiple interfaces is
398 well established and is being used today by various applications and
399 protocols, e.g., [<a href="#ref-MPTCP" title="&quot;TCP Extensions for Multipath Operation with Multiple Addresses&quot;">MPTCP</a>]. Using multiple interfaces introduces some
400 challenges and issues, which were presented and discussed in [<a href="#ref-MIF" title="&quot;Multiple Interfaces and Provisioning Domains Problem Statement&quot;">MIF</a>].
401
402 The descriptions in this section refer to the end-to-end scheme of
403 PTP but are similarly applicable to the peer-to-peer scheme. MPNTP,
404 as described in this document, refers to the NTP client-server mode,
405 although the concepts described here can be extended to include the
406 symmetric variant as well.
407
408 Multipath synchronization by nature requires protocol messages to be
409 sent as unicast. Specifically in PTP, the following messages must be
410 sent as unicast in MPPTP: Sync, Delay_Req, Delay_Resp, PDelay_Req,
411 PDelay_Resp, Follow_Up, and PDelay_Resp_Follow_Up. Note that
412 [<a href="#ref-IEEE1588" title="&quot;IEEE Standard for a Precision Clock Synchronization Protocol for Networked Measurement and Control Systems&quot;">IEEE1588</a>] allows these messages to be sent either as multicast or as
413 unicast.
414
415 <span class="h3"><a class="selflink" id="section-5.2" href="#section-5.2">5.2</a>. Single-Ended Multipath Synchronization</span>
416
417 In the single-ended approach, only the slave is aware of the fact
418 that multiple paths are used, while the master is agnostic to the
419 usage of multiple paths. This approach allows a hybrid network,
420 where some of the clocks are multipath clocks and others are
421 conventional one-path clocks. A single-ended multipath clock
422 presents itself to the network as N independent clocks, using N IP
423 addresses, as well as N clockIdentity [<a href="#ref-IEEE1588" title="&quot;IEEE Standard for a Precision Clock Synchronization Protocol for Networked Measurement and Control Systems&quot;">IEEE1588</a>] values (in PTP).
424 Thus, the usage of multiple slave identities by a slave clock is
425 transparent from the master's point of view, such that it treats each
426 of the identities as a separate slave clock.
427
428 <span class="h4"><a class="selflink" id="section-5.2.1" href="#section-5.2.1">5.2.1</a>. Single-Ended MPPTP Synchronization Message Exchange</span>
429
430 The single-ended MPPTP message exchange procedure is as follows.
431
432 o Each single-ended MPPTP clock has a fixed set of N IP addresses
433 and N corresponding clockIdentities. Each clock arbitrarily
434 defines one of its IP addresses and clockIdentity values as the
435 clock primary identity.
436
437 o A single-ended MPPTP port sends Announce messages only from its
438 primary identity, according to the BMC algorithm.
439
440
441
442
443
444 <span class="grey">Shpiner, et al. Experimental [Page 8]</span></pre>
445 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-9" ></span>
446 <span class="grey"><a href="./rfc8039">RFC 8039</a> Multipath Time Synchronization December 2016</span>
447
448
449 o The BMC algorithm at each clock determines the master, based on
450 the received Announce messages.
451
452 o A single-ended MPPTP port that is in the 'slave' state uses
453 unicast negotiation to request the master to transmit unicast
454 messages to each of the N slave clockIdentity values. The slave
455 port periodically sends N Signaling messages to the master, using
456 each of its N identities. The Signaling message includes the
457 REQUEST_UNICAST_TRANSMISSION TLV [<a href="#ref-IEEE1588" title="&quot;IEEE Standard for a Precision Clock Synchronization Protocol for Networked Measurement and Control Systems&quot;">IEEE1588</a>].
458
459 o The master periodically sends unicast Sync messages from its
460 primary identity, identified by the sourcePortIdentity [<a href="#ref-IEEE1588" title="&quot;IEEE Standard for a Precision Clock Synchronization Protocol for Networked Measurement and Control Systems&quot;">IEEE1588</a>]
461 and IP address, to each of the slave identities.
462
463 o The slave, upon receiving a Sync message, identifies its path
464 according to the destination IP address. The slave sends a
465 Delay_Req unicast message to the primary identity of the master.
466 The Delay_Req is sent using the slave identity corresponding to
467 the path through which the Sync was received. Note that the rate
468 of Delay_Req messages may be lower than the Sync message rate, and
469 thus a Sync message is not necessarily followed by a Delay_Req.
470
471 o The master, in response to a Delay_Req message from the slave,
472 responds with a Delay_Resp message using the IP address and
473 sourcePortIdentity from the Delay_Req message.
474
475 o Upon receiving the Delay_Resp message, the slave identifies the
476 path using the destination IP address and the
477 requestingPortIdentity [<a href="#ref-IEEE1588" title="&quot;IEEE Standard for a Precision Clock Synchronization Protocol for Networked Measurement and Control Systems&quot;">IEEE1588</a>]. The slave can then compute the
478 corresponding path delay and the offset from the master.
479
480 o The slave combines the information from all negotiated paths.
481
482 <span class="h4"><a class="selflink" id="section-5.2.2" href="#section-5.2.2">5.2.2</a>. Single-Ended MPNTP Synchronization Message Exchange</span>
483
484 The single-ended MPNTP message exchange procedure is as follows.
485
486 o A single-ended MPNTP client has N separate identities, i.e., N IP
487 addresses. The assumption is that the server information,
488 including its IP address, is known to the NTP clients. This is a
489 fair assumption, as typically the address(es) of the NTP server(s)
490 is provided to the NTP client by configuration.
491
492 o A single-ended MPNTP client initiates NTP with an NTP server
493 N times, using each of its N identities.
494
495 o NTP is maintained between the server and each of the N client
496 identities.
497
498
499
500 <span class="grey">Shpiner, et al. Experimental [Page 9]</span></pre>
501 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-10" ></span>
502 <span class="grey"><a href="./rfc8039">RFC 8039</a> Multipath Time Synchronization December 2016</span>
503
504
505 o The client sends NTP messages to the master using each of its
506 N identities.
507
508 o The server responds to the client's NTP messages using the IP
509 address from the received NTP packet.
510
511 o The client, upon receiving an NTP packet, uses the IP destination
512 address to identify the path through which it came, and it uses
513 the time information accordingly.
514
515 o The client combines the information from all paths.
516
517 <span class="h3"><a class="selflink" id="section-5.3" href="#section-5.3">5.3</a>. Dual-Ended Multipath Synchronization</span>
518
519 In dual-ended multipath synchronization, each clock has N IP
520 addresses. Time synchronization messages are exchanged between some
521 of the combinations of {master IP, slave IP} addresses, allowing
522 multiple paths between the master and slave. Note that the actual
523 number of paths between the master and slave may be less than the
524 number of chosen {master IP, slave IP} address pairs.
525
526 Once the multiple two-way connections are established, a separate
527 synchronization protocol exchange instance is run through each
528 of them.
529
530 <span class="h4"><a class="selflink" id="section-5.3.1" href="#section-5.3.1">5.3.1</a>. Dual-Ended MPPTP Synchronization Message Exchange</span>
531
532 The dual-ended MPPTP message exchange procedure is as follows.
533
534 o Every clock has N IP addresses but uses a single clockIdentity.
535
536 o The BMC algorithm at each clock determines the master. The master
537 is identified by its clockIdentity, allowing other clocks to know
538 the multiple IP addresses it uses.
539
540 o When a clock sends an Announce message, it sends it from each of
541 its IP addresses with its clockIdentity.
542
543 o A dual-ended MPPTP port that is in the 'slave' state uses unicast
544 negotiation to request the master to transmit unicast messages to
545 some or all of its N_s IP addresses. This negotiation is done
546 individually between a slave IP address and the corresponding
547 master IP address with which the slave desires a connection. The
548 slave port periodically sends Signaling messages to the master,
549 using some or all of its N_s IP addresses as the source, to the
550 corresponding master's N_m IP addresses. The Signaling message
551 includes the REQUEST_UNICAST_TRANSMISSION TLV [<a href="#ref-IEEE1588" title="&quot;IEEE Standard for a Precision Clock Synchronization Protocol for Networked Measurement and Control Systems&quot;">IEEE1588</a>].
552
553
554
555
556 <span class="grey">Shpiner, et al. Experimental [Page 10]</span></pre>
557 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-11" ></span>
558 <span class="grey"><a href="./rfc8039">RFC 8039</a> Multipath Time Synchronization December 2016</span>
559
560
561 ('N_s' and 'N_m' indicate the number of IP addresses of the slave
562 and master, respectively.)
563
564 o The master periodically sends unicast Sync messages from each of
565 its IP addresses to the corresponding slave IP addresses for which
566 a unicast connection was negotiated.
567
568 o The slave, upon receiving a Sync message, identifies its path
569 according to the {source IP, destination IP} addresses. The slave
570 sends a Delay_Req unicast message, swapping the source and
571 destination IP addresses from the Sync message. Note that the
572 rate of Delay_Req messages may be lower than the Sync message
573 rate, and thus a Sync message is not necessarily followed by a
574 Delay_Req.
575
576 o The master, in response to a Delay_Req message from the slave,
577 responds with a Delay_Resp message using the sourcePortIdentity
578 from the Delay_Req message and swapping the IP addresses from the
579 Delay_Req.
580
581 o Upon receiving the Delay_Resp message, the slave identifies the
582 path using the {source IP, destination IP} address pair. The
583 slave can then compute the corresponding path delay and the offset
584 from the master.
585
586 o The slave combines the information from all negotiated paths.
587
588 <span class="h4"><a class="selflink" id="section-5.3.2" href="#section-5.3.2">5.3.2</a>. Dual-Ended MPNTP Synchronization Message Exchange</span>
589
590 The MPNTP message exchange procedure is as follows.
591
592 o Each NTP clock has a set of N IP addresses. The assumption is
593 that the server information, including its multiple IP addresses,
594 is known to the NTP clients.
595
596 o The MPNTP client chooses N_svr server IP addresses and N_c client
597 IP addresses and initiates the N_svr*N_c instances of the
598 protocol, one for each {server IP, client IP} address pair,
599 allowing the client to combine the information from the N_s*N_c
600 paths.
601
602 ('N_svr' and 'N_c' indicate the number of IP addresses of the
603 server and client, respectively, with which a client chooses to
604 connect.)
605
606 o The client sends NTP messages to the master using each of the
607 source-destination address combinations.
608
609
610
611
612 <span class="grey">Shpiner, et al. Experimental [Page 11]</span></pre>
613 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-12" ></span>
614 <span class="grey"><a href="./rfc8039">RFC 8039</a> Multipath Time Synchronization December 2016</span>
615
616
617 o The server responds to the client's NTP messages using the IP
618 address combination from the received NTP packet.
619
620 o Using the {source IP, destination IP} address pair in the received
621 packets, the client identifies the path and performs its
622 computations for each of the paths accordingly.
623
624 o The client combines the information from all paths.
625
626 <span class="h3"><a class="selflink" id="section-5.4" href="#section-5.4">5.4</a>. Using Traceroute for Path Discovery</span>
627
628 The approach described thus far uses multiple IP addresses in a
629 single clock to create multiple paths. However, although each
630 two-way path is defined by a different {master IP, slave IP} address
631 pair, some of the IP address pairs may traverse exactly the same
632 network path, making them redundant.
633
634 Traceroute-based path discovery can be used for filtering only the IP
635 addresses that obtain diverse paths. 'Paris traceroute' [<a href="#ref-PARIS" title="&quot;Measuring Load-balanced Paths in the Internet&quot;">PARIS</a>] and
636 'TraceFlow' [<a href="#ref-TRACEFLOW">TRACEFLOW</a>] are examples of tools that discover the paths
637 between two points in the network. It should be noted that this
638 filtering approach is effective only if the Traceroute implementation
639 uses the same IP addresses and UDP ports as the synchronization
640 protocol packets. Since some Traceroute implementations vary the UDP
641 ports, they may not be effective in identifying and filtering
642 redundant paths in synchronization protocols.
643
644 Traceroute-based filtering can be implemented by both master and
645 slave nodes, or it can be restricted to run only on slave nodes to
646 reduce the overhead on the master. For networks that guarantee that
647 the path of the timing packets in the forward and reverse directions
648 are the same, path discovery should only be performed at the slave.
649
650 Since network routes change over time, path discovery and redundant
651 path filtering should be performed periodically. Two {master IP,
652 slave IP} address pairs that produce two diverse paths may be
653 rerouted to use the same paths. Thus, the set of addresses that are
654 used by each clock should be reassessed regularly.
655
656
657
658
659
660
661
662
663
664
665
666
667
668 <span class="grey">Shpiner, et al. Experimental [Page 12]</span></pre>
669 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-13" ></span>
670 <span class="grey"><a href="./rfc8039">RFC 8039</a> Multipath Time Synchronization December 2016</span>
671
672
673 <span class="h3"><a class="selflink" id="section-5.5" href="#section-5.5">5.5</a>. Using Unicast Discovery for MPPTP</span>
674
675 As presented above, MPPTP uses Announce messages and the BMC
676 algorithm to discover the master. The unicast discovery option of
677 PTP can be used as an alternative.
678
679 When using unicast discovery, the MPPTP slave ports maintain a list
680 of the IP addresses of the master. The slave port uses unicast
681 negotiation to request unicast service from the master as follows:
682
683 o In single-ended MPPTP, the slave uses unicast negotiation from
684 each of its identities to the master's (only) identity.
685
686 o In dual-ended MPPTP, the slave uses unicast negotiation from its
687 IP addresses, each to a corresponding master IP address, to
688 request unicast synchronization messages.
689
690 Afterwards, the message exchange continues as described in
691 Sections <a href="#section-5.2.1">5.2.1</a> and <a href="#section-5.3.1">5.3.1</a>.
692
693 The unicast discovery option can be used in networks that do not
694 support multicast or in networks in which the master clocks are known
695 in advance. In particular, unicast discovery avoids multicasting
696 Announce messages.
697
698 <span class="h2"><a class="selflink" id="section-6" href="#section-6">6</a>. Combining Algorithm</span>
699
700 Previous sections discussed the methods of creating the multiple
701 paths and obtaining the time information required by the slave
702 algorithm. Once the time information is received through each of the
703 paths, the slave should use a combining algorithm, which consolidates
704 the information from the different paths into a single clock.
705 Various methods have been suggested for combining information from
706 different paths or from different clocks, e.g., [<a href="#ref-NTP" title="&quot;Network Time Protocol Version 4: Protocol and Algorithms Specification&quot;">NTP</a>] [<a href="#ref-SLAVEDIV" title="&quot;Slave Diversity: Using Multiple Paths to Improve the Accuracy of Clock Synchronization Protocols&quot;">SLAVEDIV</a>]
707 [<a href="#ref-HIGH-AVAI">HIGH-AVAI</a>] [<a href="#ref-KALMAN" title="&quot;Kalman filtering for multi-path network synchronization&quot;">KALMAN</a>]. The choice of the combining algorithm is local
708 to the slave and does not affect interoperability. Hence, this
709 document does not define a specific method to be used by the slave.
710 The combining algorithm should be chosen carefully based on the
711 system properties, as different combining algorithms provide
712 different advantages. For example, some combining algorithms (e.g.,
713 [<a href="#ref-NTP" title="&quot;Network Time Protocol Version 4: Protocol and Algorithms Specification&quot;">NTP</a>] [<a href="#ref-DELAY-ATT">DELAY-ATT</a>]) are intended to be robust in the face of security
714 attacks, while other combining algorithms (e.g., [<a href="#ref-KALMAN" title="&quot;Kalman filtering for multi-path network synchronization&quot;">KALMAN</a>]) are more
715 resilient to random delay variation.
716
717
718
719
720
721
722
723
724 <span class="grey">Shpiner, et al. Experimental [Page 13]</span></pre>
725 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-14" ></span>
726 <span class="grey"><a href="./rfc8039">RFC 8039</a> Multipath Time Synchronization December 2016</span>
727
728
729 <span class="h2"><a class="selflink" id="section-7" href="#section-7">7</a>. Security Considerations</span>
730
731 The security aspects of time synchronization protocols are discussed
732 in detail in [<a href="./rfc7384" title="&quot;Security Requirements of Time Protocols in Packet Switched Networks&quot;">RFC7384</a>]. The methods described in this document
733 propose to run a time synchronization protocol through redundant
734 paths and thus allow the detection and mitigation of
735 man-in-the-middle attacks, as described in [<a href="#ref-DELAY-ATT">DELAY-ATT</a>].
736 Specifically, multipath synchronization can mitigate the following
737 threats (as per [<a href="./rfc7384" title="&quot;Security Requirements of Time Protocols in Packet Switched Networks&quot;">RFC7384</a>]):
738
739 o Packet manipulation (<a href="./rfc7384#section-3.2.1">Section&nbsp;3.2.1 of [RFC7384]</a>).
740
741 o Packet interception and removal (<a href="./rfc7384#section-3.2.5">Section&nbsp;3.2.5 of [RFC7384]</a>).
742
743 o Packet delay manipulation (<a href="./rfc7384#section-3.2.6">Section&nbsp;3.2.6 of [RFC7384]</a>).
744
745 It should be noted that when using multiple paths, these paths may
746 partially overlap, and thus an attack that takes place in a common
747 segment of these paths is not mitigated by the redundancy. Moreover,
748 an on-path attacker may in some cases have access to more than one
749 router or may be able to migrate from one router to another.
750 Therefore, when using multiple paths, it is important for the paths
751 to be as diverse and as independent as possible, making the
752 redundancy scheme more tolerant to on-path attacks.
753
754 It should be noted that the multipath approach requires the master
755 (or NTP server) to dedicate more resources to each slave (client)
756 than the conventional single-path approach. Hence, well-known
757 Distributed Denial-of-Service (DDoS) attacks may potentially be
758 amplified when the multipath approach is enabled.
759
760 <span class="h2"><a class="selflink" id="section-8" href="#section-8">8</a>. Scope of the Experiment</span>
761
762 This memo is published as an Experimental RFC. The purpose of the
763 experimental period is to allow the community to analyze and to
764 verify the methods defined in this document. An experimental
765 evaluation of some of these methods has been published in [<a href="#ref-MULTI" title="&quot;Multi-path Time Protocols&quot;">MULTI</a>].
766 It is expected that the experimental period will allow the methods to
767 be further investigated and verified by the community. The duration
768 of the experiment is expected to be no less than two years from the
769 publication of this document.
770
771
772
773
774
775
776
777
778
779
780 <span class="grey">Shpiner, et al. Experimental [Page 14]</span></pre>
781 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-15" ></span>
782 <span class="grey"><a href="./rfc8039">RFC 8039</a> Multipath Time Synchronization December 2016</span>
783
784
785 <span class="h2"><a class="selflink" id="section-9" href="#section-9">9</a>. References</span>
786
787 <span class="h3"><a class="selflink" id="section-9.1" href="#section-9.1">9.1</a>. Normative References</span>
788
789 [<a id="ref-IEEE1588">IEEE1588</a>] IEEE Instrumentation and Measurement Society, "IEEE
790 Standard for a Precision Clock Synchronization Protocol
791 for Networked Measurement and Control Systems", IEEE
792 Std 1588-2008, DOI 10.1109/IEEESTD.2008.4579760.
793
794 [<a id="ref-NTP">NTP</a>] Mills, D., Martin, J., Ed., Burbank, J., and W. Kasch,
795 "Network Time Protocol Version 4: Protocol and Algorithms
796 Specification", <a href="./rfc5905">RFC 5905</a>, DOI 10.17487/RFC5905, June 2010,
797 &lt;<a href="http://www.rfc-editor.org/info/rfc5905">http://www.rfc-editor.org/info/rfc5905</a>&gt;.
798
799 <span class="h3"><a class="selflink" id="section-9.2" href="#section-9.2">9.2</a>. Informative References</span>
800
801 [<a id="ref-ASYMMETRY">ASYMMETRY</a>]
802 He, Y., Faloutsos, M., Krishnamurthy, S., and B. Huffaker,
803 "On routing asymmetry in the Internet", IEEE GLOBECOM,
804 DOI 10.1109/GLOCOM.2005.1577769, December 2005.
805
806 [<a id="ref-ASYMMETRY2">ASYMMETRY2</a>]
807 Pathak, A., Pucha, H., Zhang, Y., Hu, C., and Z. Mao, "A
808 measurement study of internet delay asymmetry",
809 International Conference on Passive and Active Network
810 Measurement 2008, DOI 10.1007/978-3-540-79232-1_19,
811 April 2008.
812
813 [<a id="ref-DELAY-ATT">DELAY-ATT</a>]
814 Mizrahi, T., "A Game Theoretic Analysis of Delay Attacks
815 against Time Synchronization Protocols", IEEE
816 International Symposium on Precision Clock Synchronization
817 for Measurement, Control and Communication (ISPCS),
818 DOI 10.1109/ISPCS.2012.6336612, September 2012.
819
820 [<a id="ref-HIGH-AVAI">HIGH-AVAI</a>]
821 Ferrari, P., Flammini, A., Rinaldi, S., and G. Prytz,
822 "High availability IEEE 1588 nodes over IEEE 802.1 aq
823 Shortest Path Bridging networks", IEEE International
824 Symposium on Precision Clock Synchronization for
825 Measurement, Control and Communication (ISPCS),
826 DOI 10.1109/ISPCS.2013.6644760, September 2013.
827
828 [<a id="ref-KALMAN">KALMAN</a>] Giorgi, G. and C. Narduzzi, "Kalman filtering for
829 multi-path network synchronization", IEEE International
830 Symposium on Precision Clock Synchronization for
831 Measurement, Control and Communication (ISPCS),
832 DOI 10.1109/ISPCS.2014.6948693, September 2014.
833
834
835
836 <span class="grey">Shpiner, et al. Experimental [Page 15]</span></pre>
837 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-16" ></span>
838 <span class="grey"><a href="./rfc8039">RFC 8039</a> Multipath Time Synchronization December 2016</span>
839
840
841 [<a id="ref-MIF">MIF</a>] Blanchet, M. and P. Seite, "Multiple Interfaces and
842 Provisioning Domains Problem Statement", <a href="./rfc6418">RFC 6418</a>,
843 DOI 10.17487/RFC6418, November 2011,
844 &lt;<a href="http://www.rfc-editor.org/info/rfc6418">http://www.rfc-editor.org/info/rfc6418</a>&gt;.
845
846 [<a id="ref-MPTCP">MPTCP</a>] Ford, A., Raiciu, C., Handley, M., and O. Bonaventure,
847 "TCP Extensions for Multipath Operation with Multiple
848 Addresses", <a href="./rfc6824">RFC 6824</a>, DOI 10.17487/RFC6824, January 2013,
849 &lt;<a href="http://www.rfc-editor.org/info/rfc6824">http://www.rfc-editor.org/info/rfc6824</a>&gt;.
850
851 [<a id="ref-MULTI">MULTI</a>] Shpiner, A., Revah, Y., and T. Mizrahi, "Multi-path Time
852 Protocols", IEEE International Symposium on Precision
853 Clock Synchronization for Measurement, Control and
854 Communication (ISPCS), DOI 10.1109/ISPCS.2013.6644754,
855 September 2013.
856
857 [<a id="ref-PARIS">PARIS</a>] Augustin, B., Friedman, T., and R. Teixeira, "Measuring
858 Load-balanced Paths in the Internet", 7th ACM SIGCOMM
859 conference on Internet measurement (IMC '07),
860 DOI 10.1145/1298306.1298329, October 2007.
861
862 [<a id="ref-PARIS2">PARIS2</a>] Augustin, B., Friedman, T., and R. Teixeira, "Measuring
863 Multipath Routing in the Internet", IEEE/ACM Transactions
864 on Networking, 19(3), pp. 830-840,
865 DOI 10.1109/TNET.2010.2096232, June 2011.
866
867 [<a id="ref-RFC7384">RFC7384</a>] Mizrahi, T., "Security Requirements of Time Protocols in
868 Packet Switched Networks", <a href="./rfc7384">RFC 7384</a>, DOI 10.17487/RFC7384,
869 October 2014, &lt;<a href="http://www.rfc-editor.org/info/rfc7384">http://www.rfc-editor.org/info/rfc7384</a>&gt;.
870
871 [<a id="ref-SLAVEDIV">SLAVEDIV</a>] Mizrahi, T., "Slave Diversity: Using Multiple Paths to
872 Improve the Accuracy of Clock Synchronization Protocols",
873 IEEE International Symposium on Precision Clock
874 Synchronization for Measurement, Control and Communication
875 (ISPCS), DOI 10.1109/ISPCS.2012.6336621, September 2012.
876
877 [<a id="ref-TRACEFLOW">TRACEFLOW</a>]
878 Narasimhan, J., Venkataswami, B., Groves, R., and P.
879 Hoose, "Traceflow", Work in Progress,
880 <a href="./draft-janapath-intarea-traceflow-00">draft-janapath-intarea-traceflow-00</a>, January 2012.
881
882
883
884
885
886
887
888
889
890
891
892 <span class="grey">Shpiner, et al. Experimental [Page 16]</span></pre>
893 <hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-17" ></span>
894 <span class="grey"><a href="./rfc8039">RFC 8039</a> Multipath Time Synchronization December 2016</span>
895
896
897 Acknowledgments
898
899 The authors would like to thank Yoram Revah for his contribution to
900 this work. The authors also gratefully acknowledge the useful
901 comments provided by Peter Meyer, Doug Arnold, Joe Abley, Zhen Cao,
902 Watson Ladd, and Mirja Kuehlewind, as well as other comments received
903 from the TICTOC working group participants.
904
905 Authors' Addresses
906
907 Alex Shpiner
908 Mellanox Technologies, Ltd.
909 Hakidma 26
910 Ofer Industrial Park
911 Yokneam, 2069200
912 Israel
913
914 Email: [email protected]
915
916
917 Richard Tse
918 Microsemi
919 8555 Baxter Place
920 Burnaby, BC V5A 4V7
921 Canada
922
923 Email: [email protected]
924
925
926 Craig Schelp
927 Oracle
928
929 Email: [email protected]
930
931
932 Tal Mizrahi
933 Marvell
934 6 Hamada St.
935 Yokneam, 2066721
936 Israel
937
938 Email: [email protected]
939
940
941
942
943
944
945
946
947
948 Shpiner, et al. Experimental [Page 17]
949 </pre>