Debian Codegraph @

doc/data-format.rst

raw
1 The Data File Format
2 ====================
3 The aggregated test data is saved in a file called
4 :file:`<test_name>-<date>.<title>.flent.gz` (the title part is omitted if no title is
5 specified by the :option:`-t` parameter). This file contains the data points
6 generated during the test, as well as some metadata.
7
8
9 The top-level object keys
10 -------------------------
11
12 .. envvar:: version
13
14 The file format version as an integer.
15
16 .. envvar:: x_values
17
18 An array of the x values for the test data (typically the time
19 values for timeseries data).
20
21 .. envvar:: results
22
23 A JSON object containing the result data series. The keys are the
24 data series names; the value for each key is an array of y values
25 for that data series. The data array has the same length as the
26 :envvar:`x_values` array, but there may be missing data points
27 (signified by null values).
28
29 .. envvar:: metadata
30
31 An object containing various data points about the test run. The
32 metadata values are read in as configuration parameters when the
33 data set is loaded in for further processing. Not all tests use all
34 the parameters, but they are saved anyway.
35
36 .. envvar:: raw_values
37
38 An array of objects for each data series. Each element of the array
39 contains the raw values as parsed from the test tool corresponding
40 to that data series.
41
42
43 Metadata keys
44 -------------
45
46 .. envvar:: NAME
47
48 The test name.
49
50 .. envvar:: TITLE
51
52 Any extra title specified by the :option:`--title-extra` parameter
53 when the test was run.
54
55 .. envvar:: HOSTS
56
57 List of the server hostnames connected to during the test.
58
59 .. envvar:: LOCAL_HOST
60
61 The hostname of the machine that ran the test.
62
63 .. envvar:: LENGTH
64
65 Test length in seconds, as specified by the :option:`--length` parameter.
66
67 .. envvar:: TOTAL_LENGTH
68
69 Actual data series length, after the test has added time to the
70 :envvar:`LENGTH`.
71
72 .. envvar:: STEP_SIZE
73
74 Time step size granularity.
75
76 .. envvar:: TIME
77
78 ISO timestamp of the time the test was initiated.
79
80 .. envvar:: NOTE
81
82 Arbitrary text as entered with the :option:`--note` switch when the
83 test was run.
84
85 .. envvar:: FLENT_VERSION
86
87 Version of Flent that generated the data file.
88
89 .. envvar:: IP_VERSION
90
91 IP version used to run test (as specified by command line
92 parameters, or auto-detected from :c:func:`getaddrinfo()` if unspecified).
93
94 .. envvar:: KERNEL_NAME
95
96 The kernel name as reported by :command:`uname -s`.
97
98 .. envvar:: KERNEL_RELEASE
99
100 The kernel release as reported by :command:`uname -r`.
101
102 .. envvar:: MODULE_VERSIONS
103
104 The sha1sum of certain interesting Linux kernel modules, if
105 available. Can be used to match test data to specific code versions,
106 if the kernel build is instrumented to, e.g., set the build ID to a
107 git revision.
108
109 .. envvar:: SYSCTLS
110
111 The values of several networking-related sysctls on the host (if
112 available; Linux only).
113
114 .. envvar:: EGRESS_INFO
115
116 Interface name, qdisc, offload, driver and BQL configuration of the
117 interface used to reach the test target. This requires that the
118 :command:`ip` binary is present on Linux, but can be extracted from
119 :command:`route` on BSD. Qdisc information requires the
120 :command:`tc` binary to be present, and offload information requires
121 :command:`ethtool`.
122
123 If the :option:`--remote-metadata` is used, the extended metadata
124 info is gathered for each of the hostnames specified. This is
125 gathered under the :envvar:`REMOTE_METADATA` key in the metadata
126 object, keyed by the hostname values passed to
127 :option:`--remote-metadata`. Additionally, the
128 :envvar:`REMOTE_METADATA` object will contain an object called
129 :envvar:`INGRESS_INFO` which is a duplicate of
130 :envvar:`EGRESS_INFO`, but with the destination IP exchanged for the
131 source address of the host running flent. The assumption here is
132 that :option:`--remote-metadata` is used to capture metadata of a
133 router known to be in the test path, in which case
134 :envvar:`INGRESS_INFO` will contain information about the reverse
135 path from the router (which is ingress from the point of view of the
136 host running flent). If the host being queried for remote metadata
137 is off the path, the contents of :envvar:`INGRESS_INFO` will
138 probably be the same as that of :envvar:`EGRESS_INFO` .
139
140 Extended metadata
141 -----------------
142
143 If the :option:`--extended-metadata` switch is turned on, the following
144 additional values are collected and stored (to the extent they are available
145 from the platform):
146
147 .. envvar:: IP_ADDRS
148
149 IP addresses assigned to the machine running flent.
150
151 .. envvar:: GATEWAYS
152
153 IP addresses of all detected default gateways on the system, and the
154 interfaces they are reachable through. Only available if the
155 :command:`netstat` binary is present on the system.
156
157 .. envvar:: EGRESS_INFO
158
159 In the :envvar:`EGRESS_INFO` key, the IP address of the next-hop
160 router and the interface MAC address are added if extended metadata
161 is enabled.