Debian Codegraph @

doc/tests.rst

raw
1 Supplied Tests
2 ==============
3
4 Test are supplied as Python files and can specify commands to run etc.
5 For a full list of the tests supported by flent, see the
6 :option:`--list-tests` option.
7
8 The Realtime Response Under Load (RRUL) test
9 --------------------------------------------
10
11 This test exists in a couple of variants and is a partial implementation of the
12 RRUL specification as written by Dave Taht (see
13 https://www.bufferbloat.net/projects/bloat/wiki/RRUL_Spec/). It works
14 by running RTT measurement using ICMP ping and UDP roundtrip time measurement,
15 while loading up the link with eight TCP streams (four downloads, four uploads).
16 This quite reliably saturates the measured link (wherever the bottleneck might
17 be), and thus exposes bufferbloat when it is present.
18
19 Simple TCP flow tests
20 ---------------------
21
22 These tests combine a TCP flow (either in one direction, or both) with an ICMP
23 ping measurement. It’s a simpler test than RRUL, but in some cases the single
24 TCP flow can be sufficient to saturate the link.
25
26 UDP flood test
27 --------------
28
29 This test runs *iperf* configured to emit 100Mbps of UDP packets targeted at the
30 test host, while measuring RTT using ICMP ping. It is useful for observing
31 latency in the face of a completely unresponsive packet stream.
32
33 Test parameters
34 ---------------
35
36 Some test parameters (set with :option:`--test-parameter`) affect the way tests
37 behave. These are:
38
39
40 .. envvar:: upload_streams
41 .. envvar:: download_streams
42 .. envvar:: bidir_streams
43
44 These set the number of streams for the tests that are configurable. The
45 ``tcp_nup``, ``tcp_ndown`` and ``rrul_be_nflows`` tests understand
46 ``upload_streams`` and ``download_streams``, while the ``rrul_var`` test
47 understands ``bidir_streams``. If any of these parameter is set to the
48 special value ``num_cpus`` the number of streams will be set to the number of
49 CPUs on the system (if this information is available).
50
51 .. envvar:: tcp_cong_control
52
53 Set the congestion control used for TCP flows, for platforms that supports
54 setting it. This can be specified as a simple string to set the same value
55 for upstream and downstream, or two comma-separated values to set it
56 separately for the upstream and downstream directions. On Linux, any value in
57 the sysctl ``net.ipv4.tcp_allowed_congestion_control`` can be used.
58
59 If a congestion control is specified that is not available on the system
60 running the test, setting it will simply fail. In addition, some tests
61 override the congestion control for one or more flows. The actual congestion
62 control used is stored in the ``CONG_CONTROL`` per-test metadata field.
63
64 .. envvar:: udp_bandwidth
65 .. envvar:: udp_bandwidths
66 .. envvar:: udp_pktsize
67 .. envvar:: udp_pktsizes
68
69 This sets the bandwidth and packet size of each UDP stream in the ``udp_*``
70 tests. The option is passed to ``iperf`` so can be in any syntax the iperf
71 understands (e.g. ``20M`` for 20 Mbps).
72
73 When running multiple UDP streams use the plural versions of the options
74 (``udp_bandwidths`` and ``udp_pktsizes``) to specify individual per-stream
75 values (comma-separated per stream), or the singular versions to specify the
76 same value for all streams.
77
78 .. envvar:: burst_length
79 .. envvar:: burst_ports
80 .. envvar:: burst_psize
81 .. envvar:: burst_tos
82
83 These set the length, number of ports to use, packet size and TOS value for
84 the packet bursts generated in the ``burst*`` tests.
85
86 .. envvar:: cpu_stats_hosts
87 .. envvar:: netstat_hosts
88 .. envvar:: qdisc_stats_hosts
89 .. envvar:: wifi_stats_hosts
90
91 These set hostnames to gather statistics from from during the test. The
92 hostnames are passed to SSH, so can be anything understood by SSH (including
93 using ``username@host`` syntax, or using hosts defined in ``~/.ssh/config``).
94 This will attempt to run remote commands on these hosts to gather the
95 required statistics, so passwordless login has to be enabled for. Multiple
96 hostnames can be specified, separated by commas.
97
98 CPU stats and netstat output is global to the machine being connected to. The
99 qdisc and WiFi stats need extra parameters to work. These are
100 ``qdisc_stats_interfaces``, ``wifi_stats_interfaces`` and
101 ``wifi_stats_stations``. The two former specify which interfaces to gather
102 statistics from. These are paired with the hostnames, and so must contain the
103 same number of elements (also comma-separated) as the ``_hosts`` variables.
104 To specify multiple interfaces on the same host, duplicate the hostname. The
105 ``wifi_stats_stations`` parameter specifies MAC addresses of stations to
106 gather statistics for. This list is the same for all hosts, but only stations
107 present in debugfs on each host are actually captured.
108
109 The qdisc stats gather statistics output from ``tc -s``, while the WiFi stats
110 gather statistics from debugfs. These are gathered by looping in a shell
111 script; however, for better performance, the ``tc_iterate`` and
112 ``wifistats_iterate`` programmes available in the ``misc/`` directory of the
113 source code tarball can be installed. On low-powered systems this can be
114 critical to get correct statistics. The helper programmes are packaged for
115 LEDE/OpenWrt in the ``flent-tools`` package.
116
117 .. envvar:: ping_hosts
118 .. envvar:: ping_local_binds
119 .. envvar:: ping_labels
120
121 These are used to define one or more extra host names that will receive a
122 ping flow while a test is run. The ``ping_hosts`` variable simply specifies
123 hostnames to ping (several can be specified by separating them with commas).
124 The ``ping_local_binds`` variable sets local IP address(es) to bind to for
125 the extra ping flows. If specified, it must contain the same number of local
126 addresses as the number of ping hosts. The same local address can be
127 specified multiple times, however. The ``ping_labels`` variable is used to
128 label each of the ping flows; if not specified, Flent will create a default
129 label based on the target hostname for each flow.
130
131 .. envvar:: voip_host
132 .. envvar:: voip_local_bind
133 .. envvar:: voip_control_host
134 .. envvar:: voip_marking
135
136 Similar to the ping variants above, these parameters specify a hostname that
137 will receive a VoIP test. However, unlike the ping parameters, only one
138 hostname can be specified for VoIP tests, and that end-host needs to have
139 either D-ITG (and the control server) or the IRTT server running. The marking
140 setting controls which DiffServ marking is applied to the VoIP flow and
141 defaults to no marking being set.
142
143 .. envvar:: control_hosts
144
145 Hostnames to use for the control connections for the ``rtt_fair*`` tests.
146 Comma-separated. If specified, it must contain as many hostnames as the
147 number of target hostnames specified for the test.
148
149 .. envvar:: markings
150 .. envvar:: labels
151
152 Flow markings to use for each of the flows in the ``rtt_fair*`` tests.
153 Comma-separated values of markings understood by Netperf (such as "CS0").
154 Only supports setting the same marking on both the upstream and downstream
155 packets of each flow (so no "CS0,CS0" setting as can be used for Netperf). If
156 not set, defaults to CS0 (best effort). If set, each value corresponds to a
157 flow, and any extra flows will be set to CS0. By default each flow will be
158 labelled according to its marking; to override these labels, use the
159 ``labels`` parameter.
160
161 .. envvar:: stream_delays
162
163 Specify a per-stream delay (in seconds) for the different streams started up
164 by a test. Use commas to separate values for the different streams. This can
165 be used to create tests with staggered start times, for example to test TCP
166 flow startup convergence times. What exactly constitutes a stream depends on
167 the test. For example, the rtt_fair* tests considers each hostname a stream,
168 whether or not there is one or two flows going to that host.