| 1 |
Supplied Tests |
| 2 |
============== |
| 3 |
|
| 4 |
Test are supplied as Python files and can specify commands to run etc. |
| 5 |
For a full list of the tests supported by flent, see the |
| 6 |
:option:`--list-tests` option. |
| 7 |
|
| 8 |
The Realtime Response Under Load (RRUL) test |
| 9 |
-------------------------------------------- |
| 10 |
|
| 11 |
This test exists in a couple of variants and is a partial implementation of the |
| 12 |
RRUL specification as written by Dave Taht (see |
| 13 |
https://www.bufferbloat.net/projects/bloat/wiki/RRUL_Spec/). It works |
| 14 |
by running RTT measurement using ICMP ping and UDP roundtrip time measurement, |
| 15 |
while loading up the link with eight TCP streams (four downloads, four uploads). |
| 16 |
This quite reliably saturates the measured link (wherever the bottleneck might |
| 17 |
be), and thus exposes bufferbloat when it is present. |
| 18 |
|
| 19 |
Simple TCP flow tests |
| 20 |
--------------------- |
| 21 |
|
| 22 |
These tests combine a TCP flow (either in one direction, or both) with an ICMP |
| 23 |
ping measurement. It’s a simpler test than RRUL, but in some cases the single |
| 24 |
TCP flow can be sufficient to saturate the link. |
| 25 |
|
| 26 |
UDP flood test |
| 27 |
-------------- |
| 28 |
|
| 29 |
This test runs *iperf* configured to emit 100Mbps of UDP packets targeted at the |
| 30 |
test host, while measuring RTT using ICMP ping. It is useful for observing |
| 31 |
latency in the face of a completely unresponsive packet stream. |
| 32 |
|
| 33 |
Test parameters |
| 34 |
--------------- |
| 35 |
|
| 36 |
Some test parameters (set with :option:`--test-parameter`) affect the way tests |
| 37 |
behave. These are: |
| 38 |
|
| 39 |
|
| 40 |
.. envvar:: upload_streams |
| 41 |
.. envvar:: download_streams |
| 42 |
.. envvar:: bidir_streams |
| 43 |
|
| 44 |
These set the number of streams for the tests that are configurable. The |
| 45 |
``tcp_nup``, ``tcp_ndown`` and ``rrul_be_nflows`` tests understand |
| 46 |
``upload_streams`` and ``download_streams``, while the ``rrul_var`` test |
| 47 |
understands ``bidir_streams``. If any of these parameter is set to the |
| 48 |
special value ``num_cpus`` the number of streams will be set to the number of |
| 49 |
CPUs on the system (if this information is available). |
| 50 |
|
| 51 |
.. envvar:: tcp_cong_control |
| 52 |
|
| 53 |
Set the congestion control used for TCP flows, for platforms that supports |
| 54 |
setting it. This can be specified as a simple string to set the same value |
| 55 |
for upstream and downstream, or two comma-separated values to set it |
| 56 |
separately for the upstream and downstream directions. On Linux, any value in |
| 57 |
the sysctl ``net.ipv4.tcp_allowed_congestion_control`` can be used. |
| 58 |
|
| 59 |
If a congestion control is specified that is not available on the system |
| 60 |
running the test, setting it will simply fail. In addition, some tests |
| 61 |
override the congestion control for one or more flows. The actual congestion |
| 62 |
control used is stored in the ``CONG_CONTROL`` per-test metadata field. |
| 63 |
|
| 64 |
.. envvar:: udp_bandwidth |
| 65 |
.. envvar:: udp_bandwidths |
| 66 |
.. envvar:: udp_pktsize |
| 67 |
.. envvar:: udp_pktsizes |
| 68 |
|
| 69 |
This sets the bandwidth and packet size of each UDP stream in the ``udp_*`` |
| 70 |
tests. The option is passed to ``iperf`` so can be in any syntax the iperf |
| 71 |
understands (e.g. ``20M`` for 20 Mbps). |
| 72 |
|
| 73 |
When running multiple UDP streams use the plural versions of the options |
| 74 |
(``udp_bandwidths`` and ``udp_pktsizes``) to specify individual per-stream |
| 75 |
values (comma-separated per stream), or the singular versions to specify the |
| 76 |
same value for all streams. |
| 77 |
|
| 78 |
.. envvar:: burst_length |
| 79 |
.. envvar:: burst_ports |
| 80 |
.. envvar:: burst_psize |
| 81 |
.. envvar:: burst_tos |
| 82 |
|
| 83 |
These set the length, number of ports to use, packet size and TOS value for |
| 84 |
the packet bursts generated in the ``burst*`` tests. |
| 85 |
|
| 86 |
.. envvar:: cpu_stats_hosts |
| 87 |
.. envvar:: netstat_hosts |
| 88 |
.. envvar:: qdisc_stats_hosts |
| 89 |
.. envvar:: wifi_stats_hosts |
| 90 |
|
| 91 |
These set hostnames to gather statistics from from during the test. The |
| 92 |
hostnames are passed to SSH, so can be anything understood by SSH (including |
| 93 |
using ``username@host`` syntax, or using hosts defined in ``~/.ssh/config``). |
| 94 |
This will attempt to run remote commands on these hosts to gather the |
| 95 |
required statistics, so passwordless login has to be enabled for. Multiple |
| 96 |
hostnames can be specified, separated by commas. |
| 97 |
|
| 98 |
CPU stats and netstat output is global to the machine being connected to. The |
| 99 |
qdisc and WiFi stats need extra parameters to work. These are |
| 100 |
``qdisc_stats_interfaces``, ``wifi_stats_interfaces`` and |
| 101 |
``wifi_stats_stations``. The two former specify which interfaces to gather |
| 102 |
statistics from. These are paired with the hostnames, and so must contain the |
| 103 |
same number of elements (also comma-separated) as the ``_hosts`` variables. |
| 104 |
To specify multiple interfaces on the same host, duplicate the hostname. The |
| 105 |
``wifi_stats_stations`` parameter specifies MAC addresses of stations to |
| 106 |
gather statistics for. This list is the same for all hosts, but only stations |
| 107 |
present in debugfs on each host are actually captured. |
| 108 |
|
| 109 |
The qdisc stats gather statistics output from ``tc -s``, while the WiFi stats |
| 110 |
gather statistics from debugfs. These are gathered by looping in a shell |
| 111 |
script; however, for better performance, the ``tc_iterate`` and |
| 112 |
``wifistats_iterate`` programmes available in the ``misc/`` directory of the |
| 113 |
source code tarball can be installed. On low-powered systems this can be |
| 114 |
critical to get correct statistics. The helper programmes are packaged for |
| 115 |
LEDE/OpenWrt in the ``flent-tools`` package. |
| 116 |
|
| 117 |
.. envvar:: ping_hosts |
| 118 |
.. envvar:: ping_local_binds |
| 119 |
.. envvar:: ping_labels |
| 120 |
|
| 121 |
These are used to define one or more extra host names that will receive a |
| 122 |
ping flow while a test is run. The ``ping_hosts`` variable simply specifies |
| 123 |
hostnames to ping (several can be specified by separating them with commas). |
| 124 |
The ``ping_local_binds`` variable sets local IP address(es) to bind to for |
| 125 |
the extra ping flows. If specified, it must contain the same number of local |
| 126 |
addresses as the number of ping hosts. The same local address can be |
| 127 |
specified multiple times, however. The ``ping_labels`` variable is used to |
| 128 |
label each of the ping flows; if not specified, Flent will create a default |
| 129 |
label based on the target hostname for each flow. |
| 130 |
|
| 131 |
.. envvar:: voip_host |
| 132 |
.. envvar:: voip_local_bind |
| 133 |
.. envvar:: voip_control_host |
| 134 |
.. envvar:: voip_marking |
| 135 |
|
| 136 |
Similar to the ping variants above, these parameters specify a hostname that |
| 137 |
will receive a VoIP test. However, unlike the ping parameters, only one |
| 138 |
hostname can be specified for VoIP tests, and that end-host needs to have |
| 139 |
either D-ITG (and the control server) or the IRTT server running. The marking |
| 140 |
setting controls which DiffServ marking is applied to the VoIP flow and |
| 141 |
defaults to no marking being set. |
| 142 |
|
| 143 |
.. envvar:: control_hosts |
| 144 |
|
| 145 |
Hostnames to use for the control connections for the ``rtt_fair*`` tests. |
| 146 |
Comma-separated. If specified, it must contain as many hostnames as the |
| 147 |
number of target hostnames specified for the test. |
| 148 |
|
| 149 |
.. envvar:: markings |
| 150 |
.. envvar:: labels |
| 151 |
|
| 152 |
Flow markings to use for each of the flows in the ``rtt_fair*`` tests. |
| 153 |
Comma-separated values of markings understood by Netperf (such as "CS0"). |
| 154 |
Only supports setting the same marking on both the upstream and downstream |
| 155 |
packets of each flow (so no "CS0,CS0" setting as can be used for Netperf). If |
| 156 |
not set, defaults to CS0 (best effort). If set, each value corresponds to a |
| 157 |
flow, and any extra flows will be set to CS0. By default each flow will be |
| 158 |
labelled according to its marking; to override these labels, use the |
| 159 |
``labels`` parameter. |
| 160 |
|
| 161 |
.. envvar:: stream_delays |
| 162 |
|
| 163 |
Specify a per-stream delay (in seconds) for the different streams started up |
| 164 |
by a test. Use commas to separate values for the different streams. This can |
| 165 |
be used to create tests with staggered start times, for example to test TCP |
| 166 |
flow startup convergence times. What exactly constitutes a stream depends on |
| 167 |
the test. For example, the rtt_fair* tests considers each hostname a stream, |
| 168 |
whether or not there is one or two flows going to that host. |