| 1 |
<pre>Network Working Group S. Bradner |
| 2 |
Request for Comments: 1944 Harvard University |
| 3 |
Category: Informational J. McQuaid |
| 4 |
Bay Networks |
| 5 |
May 1996 |
| 6 |
|
| 7 |
|
| 8 |
<span class="h1">Benchmarking Methodology for Network Interconnect Devices</span> |
| 9 |
|
| 10 |
Status of This Memo |
| 11 |
|
| 12 |
This memo provides information for the Internet community. This memo |
| 13 |
does not specify an Internet standard of any kind. Distribution of |
| 14 |
this memo is unlimited. |
| 15 |
|
| 16 |
Abstract |
| 17 |
|
| 18 |
This document discusses and defines a number of tests that may be |
| 19 |
used to describe the performance characteristics of a network |
| 20 |
interconnecting device. In addition to defining the tests this |
| 21 |
document also describes specific formats for reporting the results of |
| 22 |
the tests. <a href="#appendix-A">Appendix A</a> lists the tests and conditions that we believe |
| 23 |
should be included for specific cases and gives additional |
| 24 |
information about testing practices. <a href="#appendix-B">Appendix B</a> is a reference |
| 25 |
listing of maximum frame rates to be used with specific frame sizes |
| 26 |
on various media and <a href="#appendix-C">Appendix C</a> gives some examples of frame formats |
| 27 |
to be used in testing. |
| 28 |
|
| 29 |
<span class="h2"><a class="selflink" id="section-1" href="#section-1">1</a>. Introduction</span> |
| 30 |
|
| 31 |
Vendors often engage in "specsmanship" in an attempt to give their |
| 32 |
products a better position in the marketplace. This often involves |
| 33 |
"smoke & mirrors" to confuse the potential users of the products. |
| 34 |
|
| 35 |
This document defines a specific set of tests that vendors can use to |
| 36 |
measure and report the performance characteristics of network |
| 37 |
devices. The results of these tests will provide the user comparable |
| 38 |
data from different vendors with which to evaluate these devices. |
| 39 |
|
| 40 |
A previous document, "Benchmarking Terminology for Network |
| 41 |
Interconnect Devices" (<a href="./rfc1242">RFC 1242</a>), defined many of the terms that are |
| 42 |
used in this document. The terminology document should be consulted |
| 43 |
before attempting to make use of this document. |
| 44 |
|
| 45 |
|
| 46 |
|
| 47 |
|
| 48 |
|
| 49 |
|
| 50 |
|
| 51 |
|
| 52 |
<span class="grey">Bradner & McQuaid Informational [Page 1]</span></pre> |
| 53 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-2" ></span> |
| 54 |
<span class="grey"><a href="./rfc1944">RFC 1944</a> Benchmarking Methodology May 1996</span> |
| 55 |
|
| 56 |
|
| 57 |
<span class="h2"><a class="selflink" id="section-2" href="#section-2">2</a>. Real world</span> |
| 58 |
|
| 59 |
In producing this document the authors attempted to keep in mind the |
| 60 |
requirement that apparatus to perform the described tests must |
| 61 |
actually be built. We do not know of "off the shelf" equipment |
| 62 |
available to implement all of the tests but it is our opinion that |
| 63 |
such equipment can be constructed. |
| 64 |
|
| 65 |
<span class="h2"><a class="selflink" id="section-3" href="#section-3">3</a>. Tests to be run</span> |
| 66 |
|
| 67 |
There are a number of tests described in this document. Not all of |
| 68 |
the tests apply to all types of devices under test (DUTs). Vendors |
| 69 |
should perform all of the tests that can be supported by a specific |
| 70 |
type of product. The authors understand that it will take a |
| 71 |
considerable period of time to perform all of the recommended tests |
| 72 |
nder all of the recommended conditions. We believe that the results |
| 73 |
are worth the effort. <a href="#appendix-A">Appendix A</a> lists some of the tests and |
| 74 |
conditions that we believe should be included for specific cases. |
| 75 |
|
| 76 |
<span class="h2"><a class="selflink" id="section-4" href="#section-4">4</a>. Evaluating the results</span> |
| 77 |
|
| 78 |
Performing all of the recommended tests will result in a great deal |
| 79 |
of data. Much of this data will not apply to the evaluation of the |
| 80 |
devices under each circumstance. For example, the rate at which a |
| 81 |
router forwards IPX frames will be of little use in selecting a |
| 82 |
router for an environment that does not (and will not) support that |
| 83 |
protocol. Evaluating even that data which is relevant to a |
| 84 |
particular network installation will require experience which may not |
| 85 |
be readily available. Furthermore, selection of the tests to be run |
| 86 |
and evaluation of the test data must be done with an understanding of |
| 87 |
generally accepted testing practices regarding repeatability, |
| 88 |
variance and statistical significance of small numbers of trials. |
| 89 |
|
| 90 |
<span class="h2"><a class="selflink" id="section-5" href="#section-5">5</a>. Requirements</span> |
| 91 |
|
| 92 |
In this document, the words that are used to define the significance |
| 93 |
of each particular requirement are capitalized. These words are: |
| 94 |
|
| 95 |
* "MUST" This word, or the words "REQUIRED" and "SHALL" mean that |
| 96 |
the item is an absolute requirement of the specification. |
| 97 |
|
| 98 |
* "SHOULD" This word or the adjective "RECOMMENDED" means that there |
| 99 |
may exist valid reasons in particular circumstances to ignore this |
| 100 |
item, but the full implications should be understood and the case |
| 101 |
carefully weighed before choosing a different course. |
| 102 |
|
| 103 |
* "MAY" This word or the adjective "OPTIONAL" means that this item |
| 104 |
is truly optional. One vendor may choose to include the item because |
| 105 |
|
| 106 |
|
| 107 |
|
| 108 |
<span class="grey">Bradner & McQuaid Informational [Page 2]</span></pre> |
| 109 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-3" ></span> |
| 110 |
<span class="grey"><a href="./rfc1944">RFC 1944</a> Benchmarking Methodology May 1996</span> |
| 111 |
|
| 112 |
|
| 113 |
a particular marketplace requires it or because it enhances the |
| 114 |
product, for example; another vendor may omit the same item. |
| 115 |
|
| 116 |
An implementation is not compliant if it fails to satisfy one or more |
| 117 |
of the MUST requirements for the protocols it implements. An |
| 118 |
implementation that satisfies all the MUST and all the SHOULD |
| 119 |
requirements for its protocols is said to be "unconditionally |
| 120 |
compliant"; one that satisfies all the MUST requirements but not all |
| 121 |
the SHOULD requirements for its protocols is said to be |
| 122 |
"conditionally compliant". |
| 123 |
|
| 124 |
<span class="h2"><a class="selflink" id="section-6" href="#section-6">6</a>. Test set up</span> |
| 125 |
|
| 126 |
The ideal way to implement this series of tests is to use a tester |
| 127 |
with both transmitting and receiving ports. Connections are made |
| 128 |
from the sending ports of the tester to the receiving ports of the |
| 129 |
DUT and from the sending ports of the DUT back to the tester. (see |
| 130 |
Figure 1) Since the tester both sends the test traffic and receives |
| 131 |
it back, after the traffic has been forwarded but the DUT, the tester |
| 132 |
can easily determine if all of the transmitted packets were received |
| 133 |
and verify that the correct packets were received. The same |
| 134 |
functionality can be obtained with separate transmitting and |
| 135 |
receiving devices (see Figure 2) but unless they are remotely |
| 136 |
controlled by some computer in a way that simulates the single |
| 137 |
tester, the labor required to accurately perform some of the tests |
| 138 |
(particularly the throughput test) can be prohibitive. |
| 139 |
|
| 140 |
+------------+ |
| 141 |
| | |
| 142 |
+------------| tester |<-------------+ |
| 143 |
| | | | |
| 144 |
| +------------+ | |
| 145 |
| | |
| 146 |
| +------------+ | |
| 147 |
| | | | |
| 148 |
+----------->| DUT |--------------+ |
| 149 |
| | |
| 150 |
+------------+ |
| 151 |
Figure 1 |
| 152 |
|
| 153 |
+--------+ +------------+ +----------+ |
| 154 |
| | | | | | |
| 155 |
| sender |-------->| DUT |--------->| receiver | |
| 156 |
| | | | | | |
| 157 |
+--------+ +------------+ +----------+ |
| 158 |
Figure 2 |
| 159 |
|
| 160 |
|
| 161 |
|
| 162 |
|
| 163 |
|
| 164 |
<span class="grey">Bradner & McQuaid Informational [Page 3]</span></pre> |
| 165 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-4" ></span> |
| 166 |
<span class="grey"><a href="./rfc1944">RFC 1944</a> Benchmarking Methodology May 1996</span> |
| 167 |
|
| 168 |
|
| 169 |
<span class="h3"><a class="selflink" id="section-6.1" href="#section-6.1">6.1</a> Test set up for multiple media types</span> |
| 170 |
|
| 171 |
Two different setups could be used to test a DUT which is used in |
| 172 |
real-world networks to connect networks of differing media type, |
| 173 |
local Ethernet to a backbone FDDI ring for example. The tester could |
| 174 |
support both media types in which case the set up shown in Figure 1 |
| 175 |
would be used. |
| 176 |
|
| 177 |
Two identical DUTs are used in the other test set up. (see Figure 3) |
| 178 |
In many cases this set up may more accurately simulate the real |
| 179 |
world. For example, connecting two LANs together with a WAN link or |
| 180 |
high speed backbone. This set up would not be as good at simulating |
| 181 |
a system where clients on a Ethernet LAN were interacting with a |
| 182 |
server on an FDDI backbone. |
| 183 |
|
| 184 |
+-----------+ |
| 185 |
| | |
| 186 |
+---------------------| tester |<---------------------+ |
| 187 |
| | | | |
| 188 |
| +-----------+ | |
| 189 |
| | |
| 190 |
| +----------+ +----------+ | |
| 191 |
| | | | | | |
| 192 |
+------->| DUT 1 |-------------->| DUT 2 |---------+ |
| 193 |
| | | | |
| 194 |
+----------+ +----------+ |
| 195 |
|
| 196 |
Figure 3 |
| 197 |
|
| 198 |
<span class="h2"><a class="selflink" id="section-7" href="#section-7">7</a>. DUT set up</span> |
| 199 |
|
| 200 |
Before starting to perform the tests, the DUT to be tested MUST be |
| 201 |
configured following the instructions provided to the user. |
| 202 |
Specifically, it is expected that all of the supported protocols will |
| 203 |
be configured and enabled during this set up (See <a href="#appendix-A">Appendix A</a>). It is |
| 204 |
expected that all of the tests will be run without changing the |
| 205 |
configuration or setup of the DUT in any way other than that required |
| 206 |
to do the specific test. For example, it is not acceptable to change |
| 207 |
the size of frame handling buffers between tests of frame handling |
| 208 |
rates or to disable all but one transport protocol when testing the |
| 209 |
throughput of that protocol. It is necessary to modify the |
| 210 |
configuration when starting a test to determine the effect of filters |
| 211 |
on throughput, but the only change MUST be to enable the specific |
| 212 |
filter. The DUT set up SHOULD include the normally recommended |
| 213 |
routing update intervals and keep alive frequency. The specific |
| 214 |
version of the software and the exact DUT configuration, including |
| 215 |
what functions are disabled, used during the tests MUST be included |
| 216 |
as part of the report of the results. |
| 217 |
|
| 218 |
|
| 219 |
|
| 220 |
<span class="grey">Bradner & McQuaid Informational [Page 4]</span></pre> |
| 221 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-5" ></span> |
| 222 |
<span class="grey"><a href="./rfc1944">RFC 1944</a> Benchmarking Methodology May 1996</span> |
| 223 |
|
| 224 |
|
| 225 |
<span class="h2"><a class="selflink" id="section-8" href="#section-8">8</a>. Frame formats</span> |
| 226 |
|
| 227 |
The formats of the test frames to use for TCP/IP over Ethernet are |
| 228 |
shown in <a href="#appendix-C">Appendix C</a>: Test Frame Formats. These exact frame formats |
| 229 |
SHOULD be used in the tests described in this document for this |
| 230 |
protocol/media combination and that these frames will be used as a |
| 231 |
template for testing other protocol/media combinations. The specific |
| 232 |
formats that are used to define the test frames for a particular test |
| 233 |
series MUST be included in the report of the results. |
| 234 |
|
| 235 |
<span class="h2"><a class="selflink" id="section-9" href="#section-9">9</a>. Frame sizes</span> |
| 236 |
|
| 237 |
All of the described tests SHOULD be performed at a number of frame |
| 238 |
sizes. Specifically, the sizes SHOULD include the maximum and minimum |
| 239 |
legitimate sizes for the protocol under test on the media under test |
| 240 |
and enough sizes in between to be able to get a full characterization |
| 241 |
of the DUT performance. Except where noted, at least five frame |
| 242 |
sizes SHOULD be tested for each test condition. |
| 243 |
|
| 244 |
Theoretically the minimum size UDP Echo request frame would consist |
| 245 |
of an IP header (minimum length 20 octets), a UDP header (8 octets) |
| 246 |
and whatever MAC level header is required by the media in use. The |
| 247 |
theoretical maximum frame size is determined by the size of the |
| 248 |
length field in the IP header. In almost all cases the actual |
| 249 |
maximum and minimum sizes are determined by the limitations of the |
| 250 |
media. |
| 251 |
|
| 252 |
In theory it would be ideal to distribute the frame sizes in a way |
| 253 |
that would evenly distribute the theoretical frame rates. These |
| 254 |
recommendations incorporate this theory but specify frame sizes which |
| 255 |
are easy to understand and remember. In addition, many of the same |
| 256 |
frame sizes are specified on each of the media types to allow for |
| 257 |
easy performance comparisons. |
| 258 |
|
| 259 |
Note: The inclusion of an unrealistically small frame size on some of |
| 260 |
the media types (i.e. with little or no space for data) is to help |
| 261 |
characterize the per-frame processing overhead of the DUT. |
| 262 |
|
| 263 |
9.1 Frame sizes to be used on Ethernet |
| 264 |
|
| 265 |
64, 128, 256, 512, 1024, 1280, 1518 |
| 266 |
|
| 267 |
These sizes include the maximum and minimum frame sizes permitted |
| 268 |
by the Ethernet standard and a selection of sizes between these |
| 269 |
extremes with a finer granularity for the smaller frame sizes and |
| 270 |
higher frame rates. |
| 271 |
|
| 272 |
|
| 273 |
|
| 274 |
|
| 275 |
|
| 276 |
<span class="grey">Bradner & McQuaid Informational [Page 5]</span></pre> |
| 277 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-6" ></span> |
| 278 |
<span class="grey"><a href="./rfc1944">RFC 1944</a> Benchmarking Methodology May 1996</span> |
| 279 |
|
| 280 |
|
| 281 |
9.2 Frame sizes to be used on 4Mb and 16Mb token ring |
| 282 |
|
| 283 |
54, 64, 128, 256, 1024, 1518, 2048, 4472 |
| 284 |
|
| 285 |
The frame size recommendations for token ring assume that there is |
| 286 |
no RIF field in the frames of routed protocols. A RIF field would |
| 287 |
be present in any direct source route bridge performance test. |
| 288 |
The minimum size frame for UDP on token ring is 54 octets. The |
| 289 |
maximum size of 4472 octets is recommended for 16Mb token ring |
| 290 |
instead of the theoretical size of 17.9Kb because of the size |
| 291 |
limitations imposed by many token ring interfaces. The reminder |
| 292 |
of the sizes are selected to permit direct comparisons with other |
| 293 |
types of media. An IP (i.e. not UDP) frame may be used in |
| 294 |
addition if a higher data rate is desired, in which case the |
| 295 |
minimum frame size is 46 octets. |
| 296 |
|
| 297 |
9.3 Frame sizes to be used on FDDI |
| 298 |
|
| 299 |
54, 64, 128, 256, 1024, 1518, 2048, 4472 |
| 300 |
|
| 301 |
The minimum size frame for UDP on FDDI is 53 octets, the minimum |
| 302 |
size of 54 is recommended to allow direct comparison to token ring |
| 303 |
performance. The maximum size of 4472 is recommended instead of |
| 304 |
the theoretical maximum size of 4500 octets to permit the same |
| 305 |
type of comparison. An IP (i.e. not UDP) frame may be used in |
| 306 |
addition if a higher data rate is desired, in which case the |
| 307 |
minimum frame size is 45 octets. |
| 308 |
|
| 309 |
9.4 Frame sizes in the presence of disparate MTUs |
| 310 |
|
| 311 |
When the interconnect DUT supports connecting links with disparate |
| 312 |
MTUs, the frame sizes for the link with the *larger* MTU SHOULD be |
| 313 |
used, up to the limit of the protocol being tested. If the |
| 314 |
interconnect DUT does not support the fragmenting of frames in the |
| 315 |
presence of MTU mismatch, the forwarding rate for that frame size |
| 316 |
shall be reported as zero. |
| 317 |
|
| 318 |
For example, the test of IP forwarding with a bridge or router |
| 319 |
that joins FDDI and Ethernet should use the frame sizes of FDDI |
| 320 |
when going from the FDDI to the Ethernet link. If the bridge does |
| 321 |
not support IP fragmentation, the forwarding rate for those frames |
| 322 |
too large for Ethernet should be reported as zero. |
| 323 |
|
| 324 |
<span class="h2"><a class="selflink" id="section-10" href="#section-10">10</a>. Verifying received frames</span> |
| 325 |
|
| 326 |
The test equipment SHOULD discard any frames received during a test |
| 327 |
run that are not actual forwarded test frames. For example, keep- |
| 328 |
alive and routing update frames SHOULD NOT be included in the count |
| 329 |
|
| 330 |
|
| 331 |
|
| 332 |
<span class="grey">Bradner & McQuaid Informational [Page 6]</span></pre> |
| 333 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-7" ></span> |
| 334 |
<span class="grey"><a href="./rfc1944">RFC 1944</a> Benchmarking Methodology May 1996</span> |
| 335 |
|
| 336 |
|
| 337 |
of received frames. In any case, the test equipment SHOULD verify |
| 338 |
the length of the received frames and check that they match the |
| 339 |
expected length. |
| 340 |
|
| 341 |
Preferably, the test equipment SHOULD include sequence numbers in the |
| 342 |
transmitted frames and check for these numbers on the received |
| 343 |
frames. If this is done, the reported results SHOULD include in |
| 344 |
addition to the number of frames dropped, the number of frames that |
| 345 |
were received out of order, the number of duplicate frames received |
| 346 |
and the number of gaps in the received frame numbering sequence. |
| 347 |
This functionality is required for some of the described tests. |
| 348 |
|
| 349 |
<span class="h2"><a class="selflink" id="section-11" href="#section-11">11</a>. Modifiers</span> |
| 350 |
|
| 351 |
It might be useful to know the DUT performance under a number of |
| 352 |
conditions; some of these conditions are noted below. The reported |
| 353 |
results SHOULD include as many of these conditions as the test |
| 354 |
equipment is able to generate. The suite of tests SHOULD be first |
| 355 |
run without any modifying conditions and then repeated under each of |
| 356 |
the conditions separately. To preserve the ability to compare the |
| 357 |
results of these tests any frames that are required to generate the |
| 358 |
modifying conditions (management queries for example) will be |
| 359 |
included in the same data stream as the normal test frames in place |
| 360 |
of one of the test frames and not be supplied to the DUT on a |
| 361 |
separate network port. |
| 362 |
|
| 363 |
11.1 Broadcast frames |
| 364 |
|
| 365 |
In most router designs special processing is required when frames |
| 366 |
addressed to the hardware broadcast address are received. In |
| 367 |
bridges (or in bridge mode on routers) these broadcast frames must |
| 368 |
be flooded to a number of ports. The stream of test frames SHOULD |
| 369 |
be augmented with 1% frames addressed to the hardware broadcast |
| 370 |
address. The frames sent to the broadcast address should be of a |
| 371 |
type that the router will not need to process. The aim of this |
| 372 |
test is to determine if there is any effect on the forwarding rate |
| 373 |
of the other data in the stream. The specific frames that should |
| 374 |
be used are included in the test frame format document. The |
| 375 |
broadcast frames SHOULD be evenly distributed throughout the data |
| 376 |
stream, for example, every 100th frame. |
| 377 |
|
| 378 |
The same test SHOULD be performed on bridge-like DUTs but in this |
| 379 |
case the broadcast packets will be processed and flooded to all |
| 380 |
outputs. |
| 381 |
|
| 382 |
It is understood that a level of broadcast frames of 1% is much |
| 383 |
higher than many networks experience but, as in drug toxicity |
| 384 |
evaluations, the higher level is required to be able to gage the |
| 385 |
|
| 386 |
|
| 387 |
|
| 388 |
<span class="grey">Bradner & McQuaid Informational [Page 7]</span></pre> |
| 389 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-8" ></span> |
| 390 |
<span class="grey"><a href="./rfc1944">RFC 1944</a> Benchmarking Methodology May 1996</span> |
| 391 |
|
| 392 |
|
| 393 |
effect which would otherwise often fall within the normal |
| 394 |
variability of the system performance. Due to design factors some |
| 395 |
test equipment will not be able to generate a level of alternate |
| 396 |
frames this low. In these cases the percentage SHOULD be as small |
| 397 |
as the equipment can provide and that the actual level be |
| 398 |
described in the report of the test results. |
| 399 |
|
| 400 |
11.2 Management frames |
| 401 |
|
| 402 |
Most data networks now make use of management protocols such as |
| 403 |
SNMP. In many environments there can be a number of management |
| 404 |
stations sending queries to the same DUT at the same time. |
| 405 |
|
| 406 |
The stream of test frames SHOULD be augmented with one management |
| 407 |
query as the first frame sent each second during the duration of |
| 408 |
the trial. The result of the query must fit into one response |
| 409 |
frame. The response frame SHOULD be verified by the test |
| 410 |
equipment. One example of the specific query frame that should be |
| 411 |
used is shown in <a href="#appendix-C">Appendix C</a>. |
| 412 |
|
| 413 |
11.3 Routing update frames |
| 414 |
|
| 415 |
The processing of dynamic routing protocol updates could have a |
| 416 |
significant impact on the ability of a router to forward data |
| 417 |
frames. The stream of test frames SHOULD be augmented with one |
| 418 |
routing update frame transmitted as the first frame transmitted |
| 419 |
during the trial. Routing update frames SHOULD be sent at the |
| 420 |
rate specified in <a href="#appendix-C">Appendix C</a> for the specific routing protocol |
| 421 |
being used in the test. Two routing update frames are defined in |
| 422 |
<a href="#appendix-C">Appendix C</a> for the TCP/IP over Ethernet example. The routing |
| 423 |
frames are designed to change the routing to a number of networks |
| 424 |
that are not involved in the forwarding of the test data. The |
| 425 |
first frame sets the routing table state to "A", the second one |
| 426 |
changes the state to "B". The frames MUST be alternated during |
| 427 |
the trial. |
| 428 |
|
| 429 |
The test SHOULD verify that the routing update was processed by |
| 430 |
the DUT. |
| 431 |
|
| 432 |
11.4 Filters |
| 433 |
|
| 434 |
Filters are added to routers and bridges to selectively inhibit |
| 435 |
the forwarding of frames that would normally be forwarded. This |
| 436 |
is usually done to implement security controls on the data that is |
| 437 |
accepted between one area and another. Different products have |
| 438 |
different capabilities to implement filters. |
| 439 |
|
| 440 |
|
| 441 |
|
| 442 |
|
| 443 |
|
| 444 |
<span class="grey">Bradner & McQuaid Informational [Page 8]</span></pre> |
| 445 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-9" ></span> |
| 446 |
<span class="grey"><a href="./rfc1944">RFC 1944</a> Benchmarking Methodology May 1996</span> |
| 447 |
|
| 448 |
|
| 449 |
The DUT SHOULD be first configured to add one filter condition and |
| 450 |
the tests performed. This filter SHOULD permit the forwarding of |
| 451 |
the test data stream. In routers this filter SHOULD be of the |
| 452 |
form: |
| 453 |
|
| 454 |
forward input_protocol_address to output_protocol_address |
| 455 |
|
| 456 |
In bridges the filter SHOULD be of the form: |
| 457 |
|
| 458 |
forward destination_hardware_address |
| 459 |
|
| 460 |
The DUT SHOULD be then reconfigured to implement a total of 25 |
| 461 |
filters. The first 24 of these filters SHOULD be of the form: |
| 462 |
|
| 463 |
block input_protocol_address to output_protocol_address |
| 464 |
|
| 465 |
The 24 input and output protocol addresses SHOULD not be any that |
| 466 |
are represented in the test data stream. The last filter SHOULD |
| 467 |
permit the forwarding of the test data stream. By "first" and |
| 468 |
"last" we mean to ensure that in the second case, 25 conditions |
| 469 |
must be checked before the data frames will match the conditions |
| 470 |
that permit the forwarding of the frame. Of course, if the DUT |
| 471 |
reorders the filters or does not use a linear scan of the filter |
| 472 |
rules the effect of the sequence in which the filters are input is |
| 473 |
properly lost. |
| 474 |
|
| 475 |
The exact filters configuration command lines used SHOULD be |
| 476 |
included with the report of the results. |
| 477 |
|
| 478 |
11.4.1 Filter Addresses |
| 479 |
|
| 480 |
Two sets of filter addresses are required, one for the single |
| 481 |
filter case and one for the 25 filter case. |
| 482 |
|
| 483 |
The single filter case should permit traffic from IP address |
| 484 |
198.18.1.2 to IP address 198.19.65.2 and deny all other |
| 485 |
traffic. |
| 486 |
|
| 487 |
The 25 filter case should follow the following sequence. |
| 488 |
|
| 489 |
deny aa.ba.1.1 to aa.ba.100.1 |
| 490 |
deny aa.ba.2.2 to aa.ba.101.2 |
| 491 |
deny aa.ba.3.3 to aa.ba.103.3 |
| 492 |
... |
| 493 |
deny aa.ba.12.12 to aa.ba.112.12 |
| 494 |
allow aa.bc.1.2 to aa.bc.65.1 |
| 495 |
deny aa.ba.13.13 to aa.ba.113.13 |
| 496 |
deny aa.ba.14.14 to aa.ba.114.14 |
| 497 |
|
| 498 |
|
| 499 |
|
| 500 |
<span class="grey">Bradner & McQuaid Informational [Page 9]</span></pre> |
| 501 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-10" ></span> |
| 502 |
<span class="grey"><a href="./rfc1944">RFC 1944</a> Benchmarking Methodology May 1996</span> |
| 503 |
|
| 504 |
|
| 505 |
... |
| 506 |
deny aa.ba.24.24 to aa.ba.124.24 |
| 507 |
deny all else |
| 508 |
|
| 509 |
All previous filter conditions should be cleared from the |
| 510 |
router before this sequence is entered. The sequence is |
| 511 |
selected to test to see if the router sorts the filter |
| 512 |
conditions or accepts them in the order that they were entered. |
| 513 |
Both of these procedures will result in a greater impact on |
| 514 |
performance than will some form of hash coding. |
| 515 |
|
| 516 |
<span class="h2"><a class="selflink" id="section-12" href="#section-12">12</a>. Protocol addresses</span> |
| 517 |
|
| 518 |
It is easier to implement these tests using a single logical stream |
| 519 |
of data, with one source protocol address and one destination |
| 520 |
protocol address, and for some conditions like the filters described |
| 521 |
above, a practical requirement. Networks in the real world are not |
| 522 |
limited to single streams of data. The test suite SHOULD be first run |
| 523 |
with a single protocol (or hardware for bridge tests) source and |
| 524 |
destination address pair. The tests SHOULD then be repeated with |
| 525 |
using a random destination address. While testing routers the |
| 526 |
addresses SHOULD be random and uniformly distributed over a range of |
| 527 |
256 networks and random and uniformly distributed over the full MAC |
| 528 |
range for bridges. The specific address ranges to use for IP are |
| 529 |
shown in <a href="#appendix-C">Appendix C</a>. |
| 530 |
|
| 531 |
<span class="h2"><a class="selflink" id="section-13" href="#section-13">13</a>. Route Set Up</span> |
| 532 |
|
| 533 |
It is not reasonable that all of the routing information necessary to |
| 534 |
forward the test stream, especially in the multiple address case, |
| 535 |
will be manually set up. At the start of each trial a routing update |
| 536 |
MUST be sent to the DUT. This routing update MUST include all of the |
| 537 |
network addresses that will be required for the trial. All of the |
| 538 |
addresses SHOULD resolve to the same "next-hop". Normally this will |
| 539 |
be the address of the receiving side of the test equipment. This |
| 540 |
routing update will have to be repeated at the interval required by |
| 541 |
the routing protocol being used. An example of the format and |
| 542 |
repetition interval of the update frames is given in <a href="#appendix-C">Appendix C</a>. |
| 543 |
|
| 544 |
<span class="h2"><a class="selflink" id="section-14" href="#section-14">14</a>. Bidirectional traffic</span> |
| 545 |
|
| 546 |
Normal network activity is not all in a single direction. To test |
| 547 |
the bidirectional performance of a DUT, the test series SHOULD be run |
| 548 |
with the same data rate being offered from each direction. The sum of |
| 549 |
the data rates should not exceed the theoretical limit for the media. |
| 550 |
|
| 551 |
|
| 552 |
|
| 553 |
|
| 554 |
|
| 555 |
|
| 556 |
<span class="grey">Bradner & McQuaid Informational [Page 10]</span></pre> |
| 557 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-11" ></span> |
| 558 |
<span class="grey"><a href="./rfc1944">RFC 1944</a> Benchmarking Methodology May 1996</span> |
| 559 |
|
| 560 |
|
| 561 |
<span class="h2"><a class="selflink" id="section-15" href="#section-15">15</a>. Single stream path</span> |
| 562 |
|
| 563 |
The full suite of tests SHOULD be run along with whatever modifier |
| 564 |
conditions that are relevant using a single input and output network |
| 565 |
port on the DUT. If the internal design of the DUT has multiple |
| 566 |
distinct pathways, for example, multiple interface cards each with |
| 567 |
multiple network ports, then all possible types of pathways SHOULD be |
| 568 |
tested separately. |
| 569 |
|
| 570 |
<span class="h2"><a class="selflink" id="section-16" href="#section-16">16</a>. Multi-port</span> |
| 571 |
|
| 572 |
Many current router and bridge products provide many network ports in |
| 573 |
the same module. In performing these tests first half of the ports |
| 574 |
are designated as "input ports" and half are designated as "output |
| 575 |
ports". These ports SHOULD be evenly distributed across the DUT |
| 576 |
architecture. For example if a DUT has two interface cards each of |
| 577 |
which has four ports, two ports on each interface card are designated |
| 578 |
as input and two are designated as output. The specified tests are |
| 579 |
run using the same data rate being offered to each of the input |
| 580 |
ports. The addresses in the input data streams SHOULD be set so that |
| 581 |
a frame will be directed to each of the output ports in sequence so |
| 582 |
that all "output" ports will get an even distribution of packets from |
| 583 |
this input. The same configuration MAY be used to perform a |
| 584 |
bidirectional multi-stream test. In this case all of the ports are |
| 585 |
considered both input and output ports and each data stream MUST |
| 586 |
consist of frames addressed to all of the other ports. |
| 587 |
|
| 588 |
Consider the following 6 port DUT: |
| 589 |
|
| 590 |
-------------- |
| 591 |
---------| in A out X|-------- |
| 592 |
---------| in B out Y|-------- |
| 593 |
---------| in C out Z|-------- |
| 594 |
-------------- |
| 595 |
|
| 596 |
The addressing of the data streams for each of the inputs SHOULD be: |
| 597 |
|
| 598 |
stream sent to input A: |
| 599 |
packet to out X, packet to out Y, packet to out Z |
| 600 |
stream sent to input B: |
| 601 |
packet to out X, packet to out Y, packet to out Z |
| 602 |
stream sent to input C |
| 603 |
packet to out X, packet to out Y, packet to out Z |
| 604 |
|
| 605 |
Note that these streams each follow the same sequence so that 3 |
| 606 |
packets will arrive at output X at the same time, then 3 packets at |
| 607 |
Y, then 3 packets at Z. This procedure ensures that, as in the real |
| 608 |
world, the DUT will have to deal with multiple packets addressed to |
| 609 |
|
| 610 |
|
| 611 |
|
| 612 |
<span class="grey">Bradner & McQuaid Informational [Page 11]</span></pre> |
| 613 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-12" ></span> |
| 614 |
<span class="grey"><a href="./rfc1944">RFC 1944</a> Benchmarking Methodology May 1996</span> |
| 615 |
|
| 616 |
|
| 617 |
the same output at the same time. |
| 618 |
|
| 619 |
<span class="h2"><a class="selflink" id="section-17" href="#section-17">17</a>. Multiple protocols</span> |
| 620 |
|
| 621 |
This document does not address the issue of testing the effects of a |
| 622 |
mixed protocol environment other than to suggest that if such tests |
| 623 |
are wanted then frames SHOULD be distributed between all of the test |
| 624 |
protocols. The distribution MAY approximate the conditions on the |
| 625 |
network in which the DUT would be used. |
| 626 |
|
| 627 |
<span class="h2"><a class="selflink" id="section-18" href="#section-18">18</a>. Multiple frame sizes</span> |
| 628 |
|
| 629 |
This document does not address the issue of testing the effects of a |
| 630 |
mixed frame size environment other than to suggest that if such tests |
| 631 |
are wanted then frames SHOULD be distributed between all of the |
| 632 |
listed sizes for the protocol under test. The distribution MAY |
| 633 |
approximate the conditions on the network in which the DUT would be |
| 634 |
used. The authors do not have any idea how the results of such a test |
| 635 |
would be interpreted other than to directly compare multiple DUTs in |
| 636 |
some very specific simulated network. |
| 637 |
|
| 638 |
<span class="h2"><a class="selflink" id="section-19" href="#section-19">19</a>. Testing performance beyond a single DUT.</span> |
| 639 |
|
| 640 |
In the performance testing of a single DUT, the paradigm can be |
| 641 |
described as applying some input to a DUT and monitoring the output. |
| 642 |
The results of which can be used to form a basis of characterization |
| 643 |
of that device under those test conditions. |
| 644 |
|
| 645 |
This model is useful when the test input and output are homogenous |
| 646 |
(e.g., 64-byte IP, 802.3 frames into the DUT; 64 byte IP, 802.3 |
| 647 |
frames out), or the method of test can distinguish between dissimilar |
| 648 |
input/output. (E.g., 1518 byte IP, 802.3 frames in; 576 byte, |
| 649 |
fragmented IP, X.25 frames out.) |
| 650 |
|
| 651 |
By extending the single DUT test model, reasonable benchmarks |
| 652 |
regarding multiple DUTs or heterogeneous environments may be |
| 653 |
collected. In this extension, the single DUT is replaced by a system |
| 654 |
of interconnected network DUTs. This test methodology would support |
| 655 |
the benchmarking of a variety of device/media/service/protocol |
| 656 |
combinations. For example, a configuration for a LAN-to-WAN-to-LAN |
| 657 |
test might be: |
| 658 |
|
| 659 |
(1) 802.3-> DUT 1 -> X.25 @ 64kbps -> DUT 2 -> 802.3 |
| 660 |
|
| 661 |
Or a mixed LAN configuration might be: |
| 662 |
|
| 663 |
(2) 802.3 -> DUT 1 -> FDDI -> DUT 2 -> FDDI -> DUT 3 -> 802.3 |
| 664 |
|
| 665 |
|
| 666 |
|
| 667 |
|
| 668 |
<span class="grey">Bradner & McQuaid Informational [Page 12]</span></pre> |
| 669 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-13" ></span> |
| 670 |
<span class="grey"><a href="./rfc1944">RFC 1944</a> Benchmarking Methodology May 1996</span> |
| 671 |
|
| 672 |
|
| 673 |
In both examples 1 and 2, end-to-end benchmarks of each system could |
| 674 |
be empirically ascertained. Other behavior may be characterized |
| 675 |
through the use of intermediate devices. In example 2, the |
| 676 |
configuration may be used to give an indication of the FDDI to FDDI |
| 677 |
capability exhibited by DUT 2. |
| 678 |
|
| 679 |
Because multiple DUTs are treated as a single system, there are |
| 680 |
limitations to this methodology. For instance, this methodology may |
| 681 |
yield an aggregate benchmark for a tested system. That benchmark |
| 682 |
alone, however, may not necessarily reflect asymmetries in behavior |
| 683 |
between the DUTs, latencies introduce by other apparatus (e.g., |
| 684 |
CSUs/DSUs, switches), etc. |
| 685 |
|
| 686 |
Further, care must be used when comparing benchmarks of different |
| 687 |
systems by ensuring that the DUTs' features/configuration of the |
| 688 |
tested systems have the appropriate common denominators to allow |
| 689 |
comparison. |
| 690 |
|
| 691 |
<span class="h2"><a class="selflink" id="section-20" href="#section-20">20</a>. Maximum frame rate</span> |
| 692 |
|
| 693 |
The maximum frame rates that should be used when testing LAN |
| 694 |
connections SHOULD be the listed theoretical maximum rate for the |
| 695 |
frame size on the media. |
| 696 |
|
| 697 |
The maximum frame rate that should be used when testing WAN |
| 698 |
connections SHOULD be greater than the listed theoretical maximum |
| 699 |
rate for the frame size on that speed connection. The higher rate |
| 700 |
for WAN tests is to compensate for the fact that some vendors employ |
| 701 |
various forms of header compression. |
| 702 |
|
| 703 |
A list of maximum frame rates for LAN connections is included in |
| 704 |
<a href="#appendix-B">Appendix B</a>. |
| 705 |
|
| 706 |
<span class="h2"><a class="selflink" id="section-21" href="#section-21">21</a>. Bursty traffic</span> |
| 707 |
|
| 708 |
It is convenient to measure the DUT performance under steady state |
| 709 |
load but this is an unrealistic way to gauge the functioning of a DUT |
| 710 |
since actual network traffic normally consists of bursts of frames. |
| 711 |
Some of the tests described below SHOULD be performed with both |
| 712 |
steady state traffic and with traffic consisting of repeated bursts |
| 713 |
of frames. The frames within a burst are transmitted with the |
| 714 |
minimum legitimate inter-frame gap. |
| 715 |
|
| 716 |
The objective of the test is to determine the minimum interval |
| 717 |
between bursts which the DUT can process with no frame loss. During |
| 718 |
each test the number of frames in each burst is held constant and the |
| 719 |
inter-burst interval varied. Tests SHOULD be run with burst sizes of |
| 720 |
16, 64, 256 and 1024 frames. |
| 721 |
|
| 722 |
|
| 723 |
|
| 724 |
<span class="grey">Bradner & McQuaid Informational [Page 13]</span></pre> |
| 725 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-14" ></span> |
| 726 |
<span class="grey"><a href="./rfc1944">RFC 1944</a> Benchmarking Methodology May 1996</span> |
| 727 |
|
| 728 |
|
| 729 |
<span class="h2"><a class="selflink" id="section-22" href="#section-22">22</a>. Frames per token</span> |
| 730 |
|
| 731 |
Although it is possible to configure some token ring and FDDI |
| 732 |
interfaces to transmit more than one frame each time that the token |
| 733 |
is received, most of the network devices currently available transmit |
| 734 |
only one frame per token. These tests SHOULD first be performed |
| 735 |
while transmitting only one frame per token. |
| 736 |
|
| 737 |
Some current high-performance workstation servers do transmit more |
| 738 |
than one frame per token on FDDI to maximize throughput. Since this |
| 739 |
may be a common feature in future workstations and servers, |
| 740 |
interconnect devices with FDDI interfaces SHOULD be tested with 1, 4, |
| 741 |
8, and 16 frames per token. The reported frame rate SHOULD be the |
| 742 |
average rate of frame transmission over the total trial period. |
| 743 |
|
| 744 |
<span class="h2"><a class="selflink" id="section-23" href="#section-23">23</a>. Trial description</span> |
| 745 |
|
| 746 |
A particular test consists of multiple trials. Each trial returns |
| 747 |
one piece of information, for example the loss rate at a particular |
| 748 |
input frame rate. Each trial consists of a number of phases: |
| 749 |
|
| 750 |
a) If the DUT is a router, send the routing update to the "input" |
| 751 |
port and pause two seconds to be sure that the routing has settled. |
| 752 |
|
| 753 |
b) Send the "learning frames" to the "output" port and wait 2 |
| 754 |
seconds to be sure that the learning has settled. Bridge learning |
| 755 |
frames are frames with source addresses that are the same as the |
| 756 |
destination addresses used by the test frames. Learning frames for |
| 757 |
other protocols are used to prime the address resolution tables in |
| 758 |
the DUT. The formats of the learning frame that should be used are |
| 759 |
shown in the Test Frame Formats document. |
| 760 |
|
| 761 |
c) Run the test trial. |
| 762 |
|
| 763 |
d) Wait for two seconds for any residual frames to be received. |
| 764 |
|
| 765 |
e) Wait for at least five seconds for the DUT to restabilize. |
| 766 |
|
| 767 |
<span class="h2"><a class="selflink" id="section-24" href="#section-24">24</a>. Trial duration</span> |
| 768 |
|
| 769 |
The aim of these tests is to determine the rate continuously |
| 770 |
supportable by the DUT. The actual duration of the test trials must |
| 771 |
be a compromise between this aim and the duration of the benchmarking |
| 772 |
test suite. The duration of the test portion of each trial SHOULD be |
| 773 |
at least 60 seconds. The tests that involve some form of "binary |
| 774 |
search", for example the throughput test, to determine the exact |
| 775 |
result MAY use a shorter trial duration to minimize the length of the |
| 776 |
search procedure, but the final determination SHOULD be made with |
| 777 |
|
| 778 |
|
| 779 |
|
| 780 |
<span class="grey">Bradner & McQuaid Informational [Page 14]</span></pre> |
| 781 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-15" ></span> |
| 782 |
<span class="grey"><a href="./rfc1944">RFC 1944</a> Benchmarking Methodology May 1996</span> |
| 783 |
|
| 784 |
|
| 785 |
full length trials. |
| 786 |
|
| 787 |
<span class="h2"><a class="selflink" id="section-25" href="#section-25">25</a>. Address resolution</span> |
| 788 |
|
| 789 |
The DUT SHOULD be able to respond to address resolution requests sent |
| 790 |
by the DUT wherever the protocol requires such a process. |
| 791 |
|
| 792 |
<span class="h2"><a class="selflink" id="section-26" href="#section-26">26</a>. Benchmarking tests:</span> |
| 793 |
|
| 794 |
Note: The notation "type of data stream" refers to the above |
| 795 |
modifications to a frame stream with a constant inter-frame gap, for |
| 796 |
example, the addition of traffic filters to the configuration of the |
| 797 |
DUT. |
| 798 |
|
| 799 |
26.1 Throughput |
| 800 |
|
| 801 |
Objective: |
| 802 |
To determine the DUT throughput as defined in <a href="./rfc1242">RFC 1242</a>. |
| 803 |
|
| 804 |
Procedure: |
| 805 |
Send a specific number of frames at a specific rate through the |
| 806 |
DUT and then count the frames that are transmitted by the DUT. If |
| 807 |
the count of offered frames is equal to the count of received |
| 808 |
frames, the rate of the offered stream is raised and the test |
| 809 |
rerun. If fewer frames are received than were transmitted, the |
| 810 |
rate of the offered stream is reduced and the test is rerun. |
| 811 |
|
| 812 |
The throughput is the fastest rate at which the count of test |
| 813 |
frames transmitted by the DUT is equal to the number of test |
| 814 |
frames sent to it by the test equipment. |
| 815 |
|
| 816 |
Reporting format: |
| 817 |
The results of the throughput test SHOULD be reported in the form |
| 818 |
of a graph. If it is, the x coordinate SHOULD be the frame size, |
| 819 |
the y coordinate SHOULD be the frame rate. There SHOULD be at |
| 820 |
least two lines on the graph. There SHOULD be one line showing |
| 821 |
the theoretical frame rate for the media at the various frame |
| 822 |
sizes. The second line SHOULD be the plot of the test results. |
| 823 |
Additional lines MAY be used on the graph to report the results |
| 824 |
for each type of data stream tested. Text accompanying the graph |
| 825 |
SHOULD indicate the protocol, data stream format, and type of |
| 826 |
media used in the tests. |
| 827 |
|
| 828 |
We assume that if a single value is desired for advertising |
| 829 |
purposes the vendor will select the rate for the minimum frame |
| 830 |
size for the media. If this is done then the figure MUST be |
| 831 |
expressed in frames per second. The rate MAY also be expressed in |
| 832 |
bits (or bytes) per second if the vendor so desires. The |
| 833 |
|
| 834 |
|
| 835 |
|
| 836 |
<span class="grey">Bradner & McQuaid Informational [Page 15]</span></pre> |
| 837 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-16" ></span> |
| 838 |
<span class="grey"><a href="./rfc1944">RFC 1944</a> Benchmarking Methodology May 1996</span> |
| 839 |
|
| 840 |
|
| 841 |
statement of performance MUST include a/ the measured maximum |
| 842 |
frame rate, b/ the size of the frame used, c/ the theoretical |
| 843 |
limit of the media for that frame size, and d/ the type of |
| 844 |
protocol used in the test. Even if a single value is used as part |
| 845 |
of the advertising copy, the full table of results SHOULD be |
| 846 |
included in the product data sheet. |
| 847 |
|
| 848 |
26.2 Latency |
| 849 |
|
| 850 |
Objective: |
| 851 |
To determine the latency as defined in <a href="./rfc1242">RFC 1242</a>. |
| 852 |
|
| 853 |
Procedure: |
| 854 |
First determine the throughput for DUT at each of the listed frame |
| 855 |
sizes. Send a stream of frames at a particular frame size through |
| 856 |
the DUT at the determined throughput rate to a specific |
| 857 |
destination. The stream SHOULD be at least 120 seconds in |
| 858 |
duration. An identifying tag SHOULD be included in one frame |
| 859 |
after 60 seconds with the type of tag being implementation |
| 860 |
dependent. The time at which this frame is fully transmitted is |
| 861 |
recorded (timestamp A). The receiver logic in the test equipment |
| 862 |
MUST recognize the tag information in the frame stream and record |
| 863 |
the time at which the tagged frame was received (timestamp B). |
| 864 |
|
| 865 |
The latency is timestamp B minus timestamp A as per the relevant |
| 866 |
definition frm <a href="./rfc1242">RFC 1242</a>, namely latency as defined for store and |
| 867 |
forward devices or latency as defined for bit forwarding devices. |
| 868 |
|
| 869 |
The test MUST be repeated at least 20 times with the reported |
| 870 |
value being the average of the recorded values. |
| 871 |
|
| 872 |
This test SHOULD be performed with the test frame addressed to the |
| 873 |
same destination as the rest of the data stream and also with each |
| 874 |
of the test frames addressed to a new destination network. |
| 875 |
|
| 876 |
Reporting format: |
| 877 |
The report MUST state which definition of latency (from <a href="./rfc1242">RFC 1242</a>) |
| 878 |
was used for this test. The latency results SHOULD be reported |
| 879 |
in the format of a table with a row for each of the tested frame |
| 880 |
sizes. There SHOULD be columns for the frame size, the rate at |
| 881 |
which the latency test was run for that frame size, for the media |
| 882 |
types tested, and for the resultant latency values for each |
| 883 |
type of data stream tested. |
| 884 |
|
| 885 |
|
| 886 |
|
| 887 |
|
| 888 |
|
| 889 |
|
| 890 |
|
| 891 |
|
| 892 |
<span class="grey">Bradner & McQuaid Informational [Page 16]</span></pre> |
| 893 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-17" ></span> |
| 894 |
<span class="grey"><a href="./rfc1944">RFC 1944</a> Benchmarking Methodology May 1996</span> |
| 895 |
|
| 896 |
|
| 897 |
26.3 Frame loss rate |
| 898 |
|
| 899 |
Objective: |
| 900 |
To determine the frame loss rate, as defined in <a href="./rfc1242">RFC 1242</a>, of a DUT |
| 901 |
throughout the entire range of input data rates and frame sizes. |
| 902 |
|
| 903 |
Procedure: |
| 904 |
Send a specific number of frames at a specific rate through the |
| 905 |
DUT to be tested and count the frames that are transmitted by the |
| 906 |
DUT. The frame loss rate at each point is calculated using the |
| 907 |
following equation: |
| 908 |
|
| 909 |
( ( input_count - output_count ) * 100 ) / input_count |
| 910 |
|
| 911 |
The first trial SHOULD be run for the frame rate that corresponds |
| 912 |
to 100% of the maximum rate for the frame size on the input media. |
| 913 |
Repeat the procedure for the rate that corresponds to 90% of the |
| 914 |
maximum rate used and then for 80% of this rate. This sequence |
| 915 |
SHOULD be continued (at reducing 10% intervals) until there are |
| 916 |
two successive trials in which no frames are lost. The maximum |
| 917 |
granularity of the trials MUST be 10% of the maximum rate, a finer |
| 918 |
granularity is encouraged. |
| 919 |
|
| 920 |
Reporting format: |
| 921 |
The results of the frame loss rate test SHOULD be plotted as a |
| 922 |
graph. If this is done then the X axis MUST be the input frame |
| 923 |
rate as a percent of the theoretical rate for the media at the |
| 924 |
specific frame size. The Y axis MUST be the percent loss at the |
| 925 |
particular input rate. The left end of the X axis and the bottom |
| 926 |
of the Y axis MUST be 0 percent; the right end of the X axis and |
| 927 |
the top of the Y axis MUST be 100 percent. Multiple lines on the |
| 928 |
graph MAY used to report the frame loss rate for different frame |
| 929 |
sizes, protocols, and types of data streams. |
| 930 |
|
| 931 |
Note: See <a href="#section-18">section 18</a> for the maximum frame rates that SHOULD be |
| 932 |
used. |
| 933 |
|
| 934 |
26.4 Back-to-back frames |
| 935 |
|
| 936 |
Objective: |
| 937 |
To characterize the ability of a DUT to process back-to-back |
| 938 |
frames as defined in <a href="./rfc1242">RFC 1242</a>. |
| 939 |
|
| 940 |
Procedure: |
| 941 |
Send a burst of frames with minimum inter-frame gaps to the DUT |
| 942 |
and count the number of frames forwarded by the DUT. If the count |
| 943 |
of transmitted frames is equal to the number of frames forwarded |
| 944 |
the length of the burst is increased and the test is rerun. If |
| 945 |
|
| 946 |
|
| 947 |
|
| 948 |
<span class="grey">Bradner & McQuaid Informational [Page 17]</span></pre> |
| 949 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-18" ></span> |
| 950 |
<span class="grey"><a href="./rfc1944">RFC 1944</a> Benchmarking Methodology May 1996</span> |
| 951 |
|
| 952 |
|
| 953 |
the number of forwarded frames is less than the number |
| 954 |
transmitted, the length of the burst is reduced and the test is |
| 955 |
rerun. |
| 956 |
|
| 957 |
The back-to-back value is the number of frames in the longest |
| 958 |
burst that the DUT will handle without the loss of any frames. |
| 959 |
The trial length MUST be at least 2 seconds and SHOULD be |
| 960 |
repeated at least 50 times with the average of the recorded values |
| 961 |
being reported. |
| 962 |
|
| 963 |
Reporting format: |
| 964 |
The back-to-back results SHOULD be reported in the format of a |
| 965 |
table with a row for each of the tested frame sizes. There SHOULD |
| 966 |
be columns for the frame size and for the resultant average frame |
| 967 |
count for each type of data stream tested. The standard deviation |
| 968 |
for each measurement MAY also be reported. |
| 969 |
|
| 970 |
26.5 System recovery |
| 971 |
|
| 972 |
Objective: |
| 973 |
To characterize the speed at which a DUT recovers from an overload |
| 974 |
condition. |
| 975 |
|
| 976 |
Procedure: |
| 977 |
First determine the throughput for a DUT at each of the listed |
| 978 |
frame sizes. |
| 979 |
|
| 980 |
Send a stream of frames at a rate 110% of the recorded throughput |
| 981 |
rate or the maximum rate for the media, whichever is lower, for at |
| 982 |
least 60 seconds. At Timestamp A reduce the frame rate to 50% of |
| 983 |
the above rate and record the time of the last frame lost |
| 984 |
(Timestamp B). The system recovery time is determined by |
| 985 |
subtracting Timestamp B from Timestamp A. The test SHOULD be |
| 986 |
repeated a number of times and the average of the recorded values |
| 987 |
being reported. |
| 988 |
|
| 989 |
Reporting format: |
| 990 |
The system recovery results SHOULD be reported in the format of a |
| 991 |
table with a row for each of the tested frame sizes. There SHOULD |
| 992 |
be columns for the frame size, the frame rate used as the |
| 993 |
throughput rate for each type of data stream tested, and for the |
| 994 |
measured recovery time for each type of data stream tested. |
| 995 |
|
| 996 |
26.6 Reset |
| 997 |
|
| 998 |
Objective: |
| 999 |
To characterize the speed at which a DUT recovers from a device or |
| 1000 |
software reset. |
| 1001 |
|
| 1002 |
|
| 1003 |
|
| 1004 |
<span class="grey">Bradner & McQuaid Informational [Page 18]</span></pre> |
| 1005 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-19" ></span> |
| 1006 |
<span class="grey"><a href="./rfc1944">RFC 1944</a> Benchmarking Methodology May 1996</span> |
| 1007 |
|
| 1008 |
|
| 1009 |
Procedure: |
| 1010 |
First determine the throughput for the DUT for the minimum frame |
| 1011 |
size on the media used in the testing. |
| 1012 |
|
| 1013 |
Send a continuous stream of frames at the determined throughput |
| 1014 |
rate for the minimum sized frames. Cause a reset in the DUT. |
| 1015 |
Monitor the output until frames begin to be forwarded and record |
| 1016 |
the time that the last frame (Timestamp A) of the initial stream |
| 1017 |
and the first frame of the new stream (Timestamp B) are received. |
| 1018 |
A power interruption reset test is performed as above except that |
| 1019 |
the power to the DUT should be interrupted for 10 seconds in place |
| 1020 |
of causing a reset. |
| 1021 |
|
| 1022 |
This test SHOULD only be run using frames addressed to networks |
| 1023 |
directly connected to the DUT so that there is no requirement to |
| 1024 |
delay until a routing update is received. |
| 1025 |
|
| 1026 |
The reset value is obtained by subtracting Timestamp A from |
| 1027 |
Timestamp B. |
| 1028 |
|
| 1029 |
Hardware and software resets, as well as a power interruption |
| 1030 |
SHOULD be tested. |
| 1031 |
|
| 1032 |
Reporting format: |
| 1033 |
The reset value SHOULD be reported in a simple set of statements, |
| 1034 |
one for each reset type. |
| 1035 |
|
| 1036 |
<span class="h2"><a class="selflink" id="section-27" href="#section-27">27</a>. Security Considerations</span> |
| 1037 |
|
| 1038 |
Security issues are not addressed in this document. |
| 1039 |
|
| 1040 |
|
| 1041 |
|
| 1042 |
|
| 1043 |
|
| 1044 |
|
| 1045 |
|
| 1046 |
|
| 1047 |
|
| 1048 |
|
| 1049 |
|
| 1050 |
|
| 1051 |
|
| 1052 |
|
| 1053 |
|
| 1054 |
|
| 1055 |
|
| 1056 |
|
| 1057 |
|
| 1058 |
|
| 1059 |
|
| 1060 |
<span class="grey">Bradner & McQuaid Informational [Page 19]</span></pre> |
| 1061 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-20" ></span> |
| 1062 |
<span class="grey"><a href="./rfc1944">RFC 1944</a> Benchmarking Methodology May 1996</span> |
| 1063 |
|
| 1064 |
|
| 1065 |
<span class="h2"><a class="selflink" id="section-28" href="#section-28">28</a>. Editors' Addresses</span> |
| 1066 |
|
| 1067 |
Scott Bradner |
| 1068 |
Harvard University |
| 1069 |
1350 Mass. Ave, room 813 |
| 1070 |
Cambridge, MA 02138 |
| 1071 |
|
| 1072 |
Phone +1 617 495-3864 |
| 1073 |
Fax +1 617 496-8500 |
| 1074 |
EMail: [email protected] |
| 1075 |
|
| 1076 |
|
| 1077 |
Jim McQuaid |
| 1078 |
Bay Networks |
| 1079 |
3 Federal Street |
| 1080 |
Billerica, MA 01821 |
| 1081 |
|
| 1082 |
Phone +1 508 436-3915 |
| 1083 |
Fax: +1 508 670-8145 |
| 1084 |
EMail: [email protected] |
| 1085 |
|
| 1086 |
|
| 1087 |
|
| 1088 |
|
| 1089 |
|
| 1090 |
|
| 1091 |
|
| 1092 |
|
| 1093 |
|
| 1094 |
|
| 1095 |
|
| 1096 |
|
| 1097 |
|
| 1098 |
|
| 1099 |
|
| 1100 |
|
| 1101 |
|
| 1102 |
|
| 1103 |
|
| 1104 |
|
| 1105 |
|
| 1106 |
|
| 1107 |
|
| 1108 |
|
| 1109 |
|
| 1110 |
|
| 1111 |
|
| 1112 |
|
| 1113 |
|
| 1114 |
|
| 1115 |
|
| 1116 |
<span class="grey">Bradner & McQuaid Informational [Page 20]</span></pre> |
| 1117 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-21" ></span> |
| 1118 |
<span class="grey"><a href="./rfc1944">RFC 1944</a> Benchmarking Methodology May 1996</span> |
| 1119 |
|
| 1120 |
|
| 1121 |
Appendix A: Testing Considerations |
| 1122 |
|
| 1123 |
<span class="h3"><a class="selflink" id="appendix-A.1" href="#appendix-A.1">A.1</a> Scope Of This Appendix</span> |
| 1124 |
|
| 1125 |
This appendix discusses certain issues in the benchmarking |
| 1126 |
methodology where experience or judgment may play a role in the tests |
| 1127 |
selected to be run or in the approach to constructing the test with a |
| 1128 |
particular DUT. As such, this <a href="#appendix-M">appendix M</a>UST not be read as an |
| 1129 |
amendment to the methodology described in the body of this document |
| 1130 |
but as a guide to testing practice. |
| 1131 |
|
| 1132 |
1. Typical testing practice has been to enable all protocols to be |
| 1133 |
tested and conduct all testing with no further configuration of |
| 1134 |
protocols, even though a given set of trials may exercise only one |
| 1135 |
protocol at a time. This minimizes the opportunities to "tune" a |
| 1136 |
DUT for a single protocol. |
| 1137 |
|
| 1138 |
2. The least common denominator of the available filter functions |
| 1139 |
should be used to ensure that there is a basis for comparison |
| 1140 |
between vendors. Because of product differences, those conducting |
| 1141 |
and evaluating tests must make a judgment about this issue. |
| 1142 |
|
| 1143 |
3. Architectural considerations may need to be considered. For |
| 1144 |
example, first perform the tests with the stream going between |
| 1145 |
ports on the same interface card and the repeat the tests with the |
| 1146 |
stream going into a port on one interface card and out of a port |
| 1147 |
on a second interface card. There will almost always be a best |
| 1148 |
case and worst case configuration for a given DUT architecture. |
| 1149 |
|
| 1150 |
4. Testing done using traffic streams consisting of mixed protocols |
| 1151 |
has not shown much difference between testing with individual |
| 1152 |
protocols. That is, if protocol A testing and protocol B testing |
| 1153 |
give two different performance results, mixed protocol testing |
| 1154 |
appears to give a result which is the average of the two. |
| 1155 |
|
| 1156 |
5. Wide Area Network (WAN) performance may be tested by setting up |
| 1157 |
two identical devices connected by the appropriate short- haul |
| 1158 |
versions of the WAN modems. Performance is then measured between |
| 1159 |
a LAN interface on one DUT to a LAN interface on the other DUT. |
| 1160 |
|
| 1161 |
The maximum frame rate to be used for LAN-WAN-LAN configurations is a |
| 1162 |
judgment that can be based on known characteristics of the overall |
| 1163 |
system including compression effects, fragmentation, and gross link |
| 1164 |
speeds. Practice suggests that the rate should be at least 110% of |
| 1165 |
the slowest link speed. Substantive issues of testing compression |
| 1166 |
itself are beyond the scope of this document. |
| 1167 |
|
| 1168 |
|
| 1169 |
|
| 1170 |
|
| 1171 |
|
| 1172 |
<span class="grey">Bradner & McQuaid Informational [Page 21]</span></pre> |
| 1173 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-22" ></span> |
| 1174 |
<span class="grey"><a href="./rfc1944">RFC 1944</a> Benchmarking Methodology May 1996</span> |
| 1175 |
|
| 1176 |
|
| 1177 |
Appendix B: Maximum frame rates reference |
| 1178 |
|
| 1179 |
(Provided by Roger Beeman, Cisco Systems) |
| 1180 |
|
| 1181 |
Size Ethernet 16Mb Token Ring FDDI |
| 1182 |
(bytes) (pps) (pps) (pps) |
| 1183 |
|
| 1184 |
64 14880 24691 152439 |
| 1185 |
128 8445 13793 85616 |
| 1186 |
256 4528 7326 45620 |
| 1187 |
512 2349 3780 23585 |
| 1188 |
768 1586 2547 15903 |
| 1189 |
1024 1197 1921 11996 |
| 1190 |
1280 961 1542 9630 |
| 1191 |
1518 812 1302 8138 |
| 1192 |
|
| 1193 |
Ethernet size |
| 1194 |
Preamble 64 bits |
| 1195 |
Frame 8 x N bits |
| 1196 |
Gap 96 bits |
| 1197 |
|
| 1198 |
16Mb Token Ring size |
| 1199 |
SD 8 bits |
| 1200 |
AC 8 bits |
| 1201 |
FC 8 bits |
| 1202 |
DA 48 bits |
| 1203 |
SA 48 bits |
| 1204 |
RI 48 bits ( 06 30 00 12 00 30 ) |
| 1205 |
SNAP |
| 1206 |
DSAP 8 bits |
| 1207 |
SSAP 8 bits |
| 1208 |
Control 8 bits |
| 1209 |
Vendor 24 bits |
| 1210 |
Type 16 bits |
| 1211 |
Data 8 x ( N - 18) bits |
| 1212 |
FCS 32 bits |
| 1213 |
ED 8 bits |
| 1214 |
FS 8 bits |
| 1215 |
|
| 1216 |
Tokens or idles between packets are not included |
| 1217 |
|
| 1218 |
FDDI size |
| 1219 |
Preamble 64 bits |
| 1220 |
SD 8 bits |
| 1221 |
FC 8 bits |
| 1222 |
DA 48 bits |
| 1223 |
SA 48 bits |
| 1224 |
SNAP |
| 1225 |
|
| 1226 |
|
| 1227 |
|
| 1228 |
<span class="grey">Bradner & McQuaid Informational [Page 22]</span></pre> |
| 1229 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-23" ></span> |
| 1230 |
<span class="grey"><a href="./rfc1944">RFC 1944</a> Benchmarking Methodology May 1996</span> |
| 1231 |
|
| 1232 |
|
| 1233 |
DSAP 8 bits |
| 1234 |
SSAP 8 bits |
| 1235 |
Control 8 bits |
| 1236 |
Vendor 24 bits |
| 1237 |
Type 16 bits |
| 1238 |
Data 8 x ( N - 18) bits |
| 1239 |
FCS 32 bits |
| 1240 |
ED 4 bits |
| 1241 |
FS 12 bits |
| 1242 |
|
| 1243 |
Appendix C: Test Frame Formats |
| 1244 |
|
| 1245 |
This appendix defines the frame formats that may be used with these |
| 1246 |
tests. It also includes protocol specific parameters for TCP/IP over |
| 1247 |
Ethernet to be used with the tests as an example. |
| 1248 |
|
| 1249 |
<span class="h3"><a class="selflink" id="appendix-C.1" href="#appendix-C.1">C.1</a>. Introduction</span> |
| 1250 |
|
| 1251 |
The general logic used in the selection of the parameters and the |
| 1252 |
design of the frame formats is explained for each case within the |
| 1253 |
TCP/IP section. The same logic has been used in the other sections. |
| 1254 |
Comments are used in these sections only if there is a protocol |
| 1255 |
specific feature to be explained. Parameters and frame formats for |
| 1256 |
additional protocols can be defined by the reader by using the same |
| 1257 |
logic. |
| 1258 |
|
| 1259 |
<span class="h3"><a class="selflink" id="appendix-C.2" href="#appendix-C.2">C.2</a>. TCP/IP Information</span> |
| 1260 |
|
| 1261 |
The following section deals with the TCP/IP protocol suite. |
| 1262 |
|
| 1263 |
C.2.1 Frame Type. |
| 1264 |
An application level datagram echo request is used for the test |
| 1265 |
data frame in the protocols that support such a function. A |
| 1266 |
datagram protocol is used to minimize the chance that a router |
| 1267 |
might expect a specific session initialization sequence, as might |
| 1268 |
be the case for a reliable stream protocol. A specific defined |
| 1269 |
protocol is used because some routers verify the protocol field |
| 1270 |
and refuse to forward unknown protocols. |
| 1271 |
|
| 1272 |
For TCP/IP a UDP Echo Request is used. |
| 1273 |
|
| 1274 |
C.2.2 Protocol Addresses |
| 1275 |
Two sets of addresses must be defined: first the addresses |
| 1276 |
assigned to the router ports, and second the address that are to |
| 1277 |
be used in the frames themselves and in the routing updates. |
| 1278 |
|
| 1279 |
The network addresses 192.18.0.0 through 192.19.255.255 are have |
| 1280 |
been assigned to the BMWG by the IANA for this purpose. This |
| 1281 |
|
| 1282 |
|
| 1283 |
|
| 1284 |
<span class="grey">Bradner & McQuaid Informational [Page 23]</span></pre> |
| 1285 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-24" ></span> |
| 1286 |
<span class="grey"><a href="./rfc1944">RFC 1944</a> Benchmarking Methodology May 1996</span> |
| 1287 |
|
| 1288 |
|
| 1289 |
assignment was made to minimize the chance of conflict in case a |
| 1290 |
testing device were to be accidentally connected to part of the |
| 1291 |
Internet. The specific use of the addresses is detailed below. |
| 1292 |
|
| 1293 |
C.2.2.1 Router port protocol addresses |
| 1294 |
Half of the ports on a multi-port router are referred to as |
| 1295 |
"input" ports and the other half as "output" ports even though |
| 1296 |
some of the tests use all ports both as input and output. A |
| 1297 |
contiguous series of IP Class C network addresses from |
| 1298 |
198.18.1.0 to 198.18.64.0 have been assigned for use on the |
| 1299 |
"input" ports. A second series from 198.19.1.0 to 198.19.64.0 |
| 1300 |
have been assigned for use on the "output" ports. In all cases |
| 1301 |
the router port is node 1 on the appropriate network. For |
| 1302 |
example, a two port DUT would have an IP address of 198.18.1.1 |
| 1303 |
on one port and 198.19.1.1 on the other port. |
| 1304 |
|
| 1305 |
Some of the tests described in the methodology memo make use of |
| 1306 |
an SNMP management connection to the DUT. The management |
| 1307 |
access address for the DUT is assumed to be the first of the |
| 1308 |
"input" ports (198.18.1.1). |
| 1309 |
|
| 1310 |
C.2.2.2 Frame addresses |
| 1311 |
Some of the described tests assume adjacent network routing |
| 1312 |
(the reboot time test for example). The IP address used in the |
| 1313 |
test frame is that of node 2 on the appropriate Class C |
| 1314 |
network. (198.19.1.2 for example) |
| 1315 |
|
| 1316 |
If the test involves non-adjacent network routing the phantom |
| 1317 |
routers are located at node 10 of each of the appropriate Class |
| 1318 |
C networks. A series of Class C network addresses from |
| 1319 |
198.18.65.0 to 198.18.254.0 has been assigned for use as the |
| 1320 |
networks accessible through the phantom routers on the "input" |
| 1321 |
side of DUT. The series of Class C networks from 198.19.65.0 |
| 1322 |
to 198.19.254.0 have been assigned to be used as the networks |
| 1323 |
visible through the phantom routers on the "output" side of the |
| 1324 |
DUT. |
| 1325 |
|
| 1326 |
C.2.3 Routing Update Frequency |
| 1327 |
|
| 1328 |
The update interval for each routing protocol is may have to be |
| 1329 |
determined by the specifications of the individual protocol. For |
| 1330 |
IP RIP, Cisco IGRP and for OSPF a routing update frame or frames |
| 1331 |
should precede each stream of test frames by 5 seconds. This |
| 1332 |
frequency is sufficient for trial durations of up to 60 seconds. |
| 1333 |
Routing updates must be mixed with the stream of test frames if |
| 1334 |
longer trial periods are selected. The frequency of updates |
| 1335 |
should be taken from the following table. |
| 1336 |
|
| 1337 |
|
| 1338 |
|
| 1339 |
|
| 1340 |
<span class="grey">Bradner & McQuaid Informational [Page 24]</span></pre> |
| 1341 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-25" ></span> |
| 1342 |
<span class="grey"><a href="./rfc1944">RFC 1944</a> Benchmarking Methodology May 1996</span> |
| 1343 |
|
| 1344 |
|
| 1345 |
IP-RIP 30 sec |
| 1346 |
IGRP 90 sec |
| 1347 |
OSPF 90 sec |
| 1348 |
|
| 1349 |
C.2.4 Frame Formats - detailed discussion |
| 1350 |
|
| 1351 |
C.2.4.1 Learning Frame |
| 1352 |
In most protocols a procedure is used to determine the mapping |
| 1353 |
between the protocol node address and the MAC address. The |
| 1354 |
Address Resolution Protocol (ARP) is used to perform this |
| 1355 |
function in TCP/IP. No such procedure is required in XNS or |
| 1356 |
IPX because the MAC address is used as the protocol node |
| 1357 |
address. |
| 1358 |
|
| 1359 |
In the ideal case the tester would be able to respond to ARP |
| 1360 |
requests from the DUT. In cases where this is not possible an |
| 1361 |
ARP request should be sent to the router's "output" port. This |
| 1362 |
request should be seen as coming from the immediate destination |
| 1363 |
of the test frame stream. (i.e. the phantom router (Figure 2) |
| 1364 |
or the end node if adjacent network routing is being used.) It |
| 1365 |
is assumed that the router will cache the MAC address of the |
| 1366 |
requesting device. The ARP request should be sent 5 seconds |
| 1367 |
before the test frame stream starts in each trial. Trial |
| 1368 |
lengths of longer than 50 seconds may require that the router |
| 1369 |
be configured for an extended ARP timeout. |
| 1370 |
|
| 1371 |
+--------+ +------------+ |
| 1372 |
| | | phantom |------ P LAN |
| 1373 |
A |
| 1374 |
IN A------| DUT |------------| |------ P LAN |
| 1375 |
B |
| 1376 |
| | OUT A | router |------ P LAN |
| 1377 |
C |
| 1378 |
+--------+ +------------+ |
| 1379 |
|
| 1380 |
Figure 2 |
| 1381 |
|
| 1382 |
In the case where full routing is being used |
| 1383 |
|
| 1384 |
C.2.4.2 Routing Update Frame |
| 1385 |
If the test does not involve adjacent net routing the tester |
| 1386 |
must supply proper routing information using a routing update. |
| 1387 |
A single routing update is used before each trial on each |
| 1388 |
"destination" port (see section C.24). This update includes |
| 1389 |
the network addresses that are reachable through a phantom |
| 1390 |
router on the network attached to the port. For a full mesh |
| 1391 |
test, one destination network address is present in the routing |
| 1392 |
update for each of the "input" ports. The test stream on each |
| 1393 |
|
| 1394 |
|
| 1395 |
|
| 1396 |
<span class="grey">Bradner & McQuaid Informational [Page 25]</span></pre> |
| 1397 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-26" ></span> |
| 1398 |
<span class="grey"><a href="./rfc1944">RFC 1944</a> Benchmarking Methodology May 1996</span> |
| 1399 |
|
| 1400 |
|
| 1401 |
"input" port consists of a repeating sequence of frames, one to |
| 1402 |
each of the "output" ports. |
| 1403 |
|
| 1404 |
C.2.4.3 Management Query Frame |
| 1405 |
The management overhead test uses SNMP to query a set of |
| 1406 |
variables that should be present in all DUTs that support SNMP. |
| 1407 |
The variables for a single interface only are read by an NMS |
| 1408 |
at the appropriate intervals. The list of variables to |
| 1409 |
retrieve follow: |
| 1410 |
|
| 1411 |
sysUpTime |
| 1412 |
ifInOctets |
| 1413 |
ifOutOctets |
| 1414 |
ifInUcastPkts |
| 1415 |
ifOutUcastPkts |
| 1416 |
|
| 1417 |
C.2.4.4 Test Frames |
| 1418 |
The test frame is an UDP Echo Request with enough data to fill |
| 1419 |
out the required frame size. The data should not be all bits |
| 1420 |
off or all bits on since these patters can cause a "bit |
| 1421 |
stuffing" process to be used to maintain clock synchronization |
| 1422 |
on WAN links. This process will result in a longer frame than |
| 1423 |
was intended. |
| 1424 |
|
| 1425 |
C.2.4.5 Frame Formats - TCP/IP on Ethernet |
| 1426 |
Each of the frames below are described for the 1st pair of DUT |
| 1427 |
ports, i.e. "input" port #1 and "output" port #1. Addresses |
| 1428 |
must be changed if the frame is to be used for other ports. |
| 1429 |
|
| 1430 |
C.2.6.1 Learning Frame |
| 1431 |
|
| 1432 |
ARP Request on Ethernet |
| 1433 |
|
| 1434 |
-- DATAGRAM HEADER |
| 1435 |
offset data (hex) description |
| 1436 |
00 FF FF FF FF FF FF dest MAC address send to |
| 1437 |
broadcast address |
| 1438 |
06 xx xx xx xx xx xx set to source MAC address |
| 1439 |
12 08 06 ARP type |
| 1440 |
14 00 01 hardware type Ethernet = 1 |
| 1441 |
16 08 00 protocol type IP = 800 |
| 1442 |
18 06 hardware address length 48 bits |
| 1443 |
on Ethernet |
| 1444 |
19 04 protocol address length 4 octets |
| 1445 |
for IP |
| 1446 |
20 00 01 opcode request = 1 |
| 1447 |
22 xx xx xx xx xx xx source MAC address |
| 1448 |
28 xx xx xx xx source IP address |
| 1449 |
|
| 1450 |
|
| 1451 |
|
| 1452 |
<span class="grey">Bradner & McQuaid Informational [Page 26]</span></pre> |
| 1453 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-27" ></span> |
| 1454 |
<span class="grey"><a href="./rfc1944">RFC 1944</a> Benchmarking Methodology May 1996</span> |
| 1455 |
|
| 1456 |
|
| 1457 |
32 FF FF FF FF FF FF requesting DUT's MAC address |
| 1458 |
38 xx xx xx xx DUT's IP address |
| 1459 |
|
| 1460 |
C.2.6.2 Routing Update Frame |
| 1461 |
|
| 1462 |
-- DATAGRAM HEADER |
| 1463 |
offset data (hex) description |
| 1464 |
00 FF FF FF FF FF FF dest MAC address is broadcast |
| 1465 |
06 xx xx xx xx xx xx source hardware address |
| 1466 |
12 08 00 type |
| 1467 |
|
| 1468 |
-- IP HEADER |
| 1469 |
14 45 IP version - 4, header length (4 |
| 1470 |
byte units) - 5 |
| 1471 |
15 00 service field |
| 1472 |
16 00 EE total length |
| 1473 |
18 00 00 ID |
| 1474 |
20 40 00 flags (3 bits) 4 (do not |
| 1475 |
fragment), |
| 1476 |
fragment offset-0 |
| 1477 |
22 0A TTL |
| 1478 |
23 11 protocol - 17 (UDP) |
| 1479 |
24 C4 8D header checksum |
| 1480 |
26 xx xx xx xx source IP address |
| 1481 |
30 xx xx xx destination IP address |
| 1482 |
33 FF host part = FF for broadcast |
| 1483 |
|
| 1484 |
-- UDP HEADER |
| 1485 |
34 02 08 source port 208 = RIP |
| 1486 |
36 02 08 destination port 208 = RIP |
| 1487 |
38 00 DA UDP message length |
| 1488 |
40 00 00 UDP checksum |
| 1489 |
|
| 1490 |
-- RIP packet |
| 1491 |
42 02 command = response |
| 1492 |
43 01 version = 1 |
| 1493 |
44 00 00 0 |
| 1494 |
|
| 1495 |
-- net 1 |
| 1496 |
46 00 02 family = IP |
| 1497 |
48 00 00 0 |
| 1498 |
50 xx xx xx net 1 IP address |
| 1499 |
53 00 net not node |
| 1500 |
54 00 00 00 00 0 |
| 1501 |
58 00 00 00 00 0 |
| 1502 |
62 00 00 00 07 metric 7 |
| 1503 |
|
| 1504 |
-- net 2 |
| 1505 |
|
| 1506 |
|
| 1507 |
|
| 1508 |
<span class="grey">Bradner & McQuaid Informational [Page 27]</span></pre> |
| 1509 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-28" ></span> |
| 1510 |
<span class="grey"><a href="./rfc1944">RFC 1944</a> Benchmarking Methodology May 1996</span> |
| 1511 |
|
| 1512 |
|
| 1513 |
66 00 02 family = IP |
| 1514 |
68 00 00 0 |
| 1515 |
70 xx xx xx net 2 IP address |
| 1516 |
73 00 net not node |
| 1517 |
74 00 00 00 00 0 |
| 1518 |
78 00 00 00 00 0 |
| 1519 |
82 00 00 00 07 metric 7 |
| 1520 |
|
| 1521 |
-- net 3 |
| 1522 |
86 00 02 family = IP |
| 1523 |
88 00 00 0 |
| 1524 |
90 xx xx xx net 3 IP address |
| 1525 |
93 00 net not node |
| 1526 |
94 00 00 00 00 0 |
| 1527 |
98 00 00 00 00 0 |
| 1528 |
102 00 00 00 07 metric 7 |
| 1529 |
|
| 1530 |
-- net 4 |
| 1531 |
106 00 02 family = IP |
| 1532 |
108 00 00 0 |
| 1533 |
110 xx xx xx net 4 IP address |
| 1534 |
113 00 net not node |
| 1535 |
114 00 00 00 00 0 |
| 1536 |
118 00 00 00 00 0 |
| 1537 |
122 00 00 00 07 metric 7 |
| 1538 |
|
| 1539 |
-- net 5 |
| 1540 |
126 00 02 family = IP |
| 1541 |
128 00 00 0 |
| 1542 |
130 00 net 5 IP address |
| 1543 |
133 00 net not node |
| 1544 |
134 00 00 00 00 0 |
| 1545 |
138 00 00 00 00 0 |
| 1546 |
142 00 00 00 07 metric 7 |
| 1547 |
|
| 1548 |
-- net 6 |
| 1549 |
146 00 02 family = IP |
| 1550 |
148 00 00 0 |
| 1551 |
150 xx xx xx net 6 IP address |
| 1552 |
153 00 net not node |
| 1553 |
154 00 00 00 00 0 |
| 1554 |
158 00 00 00 00 0 |
| 1555 |
162 00 00 00 07 metric 7 |
| 1556 |
|
| 1557 |
C.2.4.6 Management Query Frame |
| 1558 |
|
| 1559 |
To be defined. |
| 1560 |
|
| 1561 |
|
| 1562 |
|
| 1563 |
|
| 1564 |
<span class="grey">Bradner & McQuaid Informational [Page 28]</span></pre> |
| 1565 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-29" ></span> |
| 1566 |
<span class="grey"><a href="./rfc1944">RFC 1944</a> Benchmarking Methodology May 1996</span> |
| 1567 |
|
| 1568 |
|
| 1569 |
C.2.6.4 Test Frames |
| 1570 |
UDP echo request on Ethernet |
| 1571 |
|
| 1572 |
-- DATAGRAM HEADER |
| 1573 |
offset data (hex) description |
| 1574 |
00 xx xx xx xx xx xx set to dest MAC address |
| 1575 |
06 xx xx xx xx xx xx set to source MAC address |
| 1576 |
12 08 00 type |
| 1577 |
|
| 1578 |
-- IP HEADER |
| 1579 |
14 45 IP version - 4 header length 5 4 |
| 1580 |
byte units |
| 1581 |
15 00 TOS |
| 1582 |
16 00 2E total length* |
| 1583 |
18 00 00 ID |
| 1584 |
20 00 00 flags (3 bits) - 0 fragment |
| 1585 |
offset-0 |
| 1586 |
22 0A TTL |
| 1587 |
23 11 protocol - 17 (UDP) |
| 1588 |
24 C4 8D header checksum* |
| 1589 |
26 xx xx xx xx set to source IP address** |
| 1590 |
30 xx xx xx xx set to destination IP address** |
| 1591 |
|
| 1592 |
-- UDP HEADER |
| 1593 |
34 C0 20 source port |
| 1594 |
36 00 07 destination port 07 = Echo |
| 1595 |
38 00 1A UDP message length* |
| 1596 |
40 00 00 UDP checksum |
| 1597 |
|
| 1598 |
-- UDP DATA |
| 1599 |
42 00 01 02 03 04 05 06 07 some data*** |
| 1600 |
50 08 09 0A 0B 0C 0D 0E 0F |
| 1601 |
|
| 1602 |
* - change for different length frames |
| 1603 |
|
| 1604 |
** - change for different logical streams |
| 1605 |
|
| 1606 |
*** - fill remainder of frame with incrementing octets, |
| 1607 |
repeated if required by frame length |
| 1608 |
|
| 1609 |
|
| 1610 |
|
| 1611 |
|
| 1612 |
|
| 1613 |
|
| 1614 |
|
| 1615 |
|
| 1616 |
|
| 1617 |
|
| 1618 |
|
| 1619 |
|
| 1620 |
<span class="grey">Bradner & McQuaid Informational [Page 29]</span></pre> |
| 1621 |
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-30" ></span> |
| 1622 |
<span class="grey"><a href="./rfc1944">RFC 1944</a> Benchmarking Methodology May 1996</span> |
| 1623 |
|
| 1624 |
|
| 1625 |
Values to be used in Total Length and UDP message length fields: |
| 1626 |
|
| 1627 |
frame size total length UDP message length |
| 1628 |
64 00 2E 00 1A |
| 1629 |
128 00 6E 00 5A |
| 1630 |
256 00 EE 00 9A |
| 1631 |
512 01 EE 01 9A |
| 1632 |
768 02 EE 02 9A |
| 1633 |
1024 03 EE 03 9A |
| 1634 |
1280 04 EE 04 9A |
| 1635 |
1518 05 DC 05 C8 |
| 1636 |
|
| 1637 |
|
| 1638 |
|
| 1639 |
|
| 1640 |
|
| 1641 |
|
| 1642 |
|
| 1643 |
|
| 1644 |
|
| 1645 |
|
| 1646 |
|
| 1647 |
|
| 1648 |
|
| 1649 |
|
| 1650 |
|
| 1651 |
|
| 1652 |
|
| 1653 |
|
| 1654 |
|
| 1655 |
|
| 1656 |
|
| 1657 |
|
| 1658 |
|
| 1659 |
|
| 1660 |
|
| 1661 |
|
| 1662 |
|
| 1663 |
|
| 1664 |
|
| 1665 |
|
| 1666 |
|
| 1667 |
|
| 1668 |
|
| 1669 |
|
| 1670 |
|
| 1671 |
|
| 1672 |
|
| 1673 |
|
| 1674 |
|
| 1675 |
|
| 1676 |
Bradner & McQuaid Informational [Page 30] |
| 1677 |
</pre> |