| 1 |
<?xml version="1.0" standalone="no"?> |
| 2 |
<!DOCTYPE article PUBLIC "-//OASIS//DTD DocBook XML V4.4//EN" |
| 3 |
"http://www.oasis-open.org/docbook/xml/4.4/docbookx.dtd" |
| 4 |
[ |
| 5 |
]> |
| 6 |
|
| 7 |
<article id="index"> |
| 8 |
<articleinfo> |
| 9 |
<title>D-Bus Test Plan</title> |
| 10 |
<date>14 February 2003</date> |
| 11 |
<authorgroup> |
| 12 |
<author> |
| 13 |
<firstname>Anders</firstname> |
| 14 |
<surname>Carlsson</surname> |
| 15 |
<affiliation> |
| 16 |
<orgname>CodeFactory AB</orgname> |
| 17 |
<address><email>[email protected]</email></address> |
| 18 |
</affiliation> |
| 19 |
</author> |
| 20 |
</authorgroup> |
| 21 |
</articleinfo> |
| 22 |
<sect1 id="introduction"> |
| 23 |
<title>Introduction</title> |
| 24 |
<para> |
| 25 |
This document tries to explain the details of the test plan for D-Bus |
| 26 |
</para> |
| 27 |
<sect2 id="importance-of-testing"> |
| 28 |
<title>The importance of testing</title> |
| 29 |
<para> |
| 30 |
As with any big library or program, testing is important. It |
| 31 |
can help find bugs and regressions and make the code better |
| 32 |
overall. |
| 33 |
</para> |
| 34 |
<para> |
| 35 |
D-Bus is a large and complex piece of software (about 25,000 |
| 36 |
lines of code for the client library, and 2,500 lines of code |
| 37 |
for the bus daemon) and it's therefore important to try to make sure |
| 38 |
that all parts of the software is functioning correctly. |
| 39 |
</para> |
| 40 |
<para> |
| 41 |
D-Bus can be built with support for testing by passing |
| 42 |
<literal>--enable-tests</literal>. to the configure script. It |
| 43 |
is recommended that production systems build without testing |
| 44 |
since that reduces the D-Bus client library size. |
| 45 |
</para> |
| 46 |
</sect2> |
| 47 |
</sect1> |
| 48 |
<sect1 id="client-library"> |
| 49 |
<title>Testing the D-Bus client library</title> |
| 50 |
<para> |
| 51 |
The tests for the client library consist of the test-dbus |
| 52 |
program which is a unit test for all aspects of the client |
| 53 |
library. Whenever a bug in the client library is found and |
| 54 |
fixed, a test is added to make sure that the bug won't occur again. |
| 55 |
</para> |
| 56 |
<sect2 id="data-structures"> |
| 57 |
<title>Data Structures</title> |
| 58 |
<para> |
| 59 |
The D-Bus client library consists of some data structures that |
| 60 |
are used internally; a linked list class, a hashtable class and |
| 61 |
a string class. All aspects of those are tested by test-dbus. |
| 62 |
</para> |
| 63 |
</sect2> |
| 64 |
<sect2 id="message-loader"> |
| 65 |
<title>Message loader</title> |
| 66 |
<para> |
| 67 |
The message loader is the part of D-Bus that takes messages in |
| 68 |
raw character form and parses them, turning them into DBusMessages. |
| 69 |
</para> |
| 70 |
<para> |
| 71 |
This is one of the parts of D-Bus that |
| 72 |
<emphasis>must</emphasis> be absolutely bug-free and |
| 73 |
robust. The message loader should be able to handle invalid |
| 74 |
and incomplete messages without crashing. Not doing so is a |
| 75 |
serious issue and can easily result in D-Bus being exploitable |
| 76 |
to DoS attacks. |
| 77 |
</para> |
| 78 |
<para> |
| 79 |
To solve these problems, there is a testing feature called the |
| 80 |
Message Builder. The message builder can take a serialized |
| 81 |
message in string-form and convert it into a raw character |
| 82 |
string which can then be loaded by the message loader. |
| 83 |
</para> |
| 84 |
<figure> |
| 85 |
<title>Example of a message in string form</title> |
| 86 |
<programlisting> |
| 87 |
# Standard org.freedesktop.DBus.Hello message |
| 88 |
|
| 89 |
VALID_HEADER |
| 90 |
FIELD_NAME name |
| 91 |
TYPE STRING |
| 92 |
STRING 'org.freedesktop.DBus.Hello' |
| 93 |
FIELD_NAME srvc |
| 94 |
TYPE STRING |
| 95 |
STRING 'org.freedesktop.DBus' |
| 96 |
ALIGN 8 |
| 97 |
END_LENGTH Header |
| 98 |
START_LENGTH Body |
| 99 |
END_LENGTH Body |
| 100 |
</programlisting> |
| 101 |
</figure> |
| 102 |
<para> |
| 103 |
The file format of messages in string form is documented in |
| 104 |
the D-Bus Reference Manual. |
| 105 |
</para> |
| 106 |
<para> |
| 107 |
The message test part of test-dbus is using the message |
| 108 |
builder to build different kinds of messages, both valid, |
| 109 |
invalid, and invalid ones, to make sure that the loader won't |
| 110 |
crash or leak memory of any of those, and that the loader |
| 111 |
knows if a message is valid or not. |
| 112 |
</para> |
| 113 |
<para> |
| 114 |
There is also a test program called |
| 115 |
<literal>break-loader</literal> that loads a message in |
| 116 |
string-form into raw character form using the message |
| 117 |
builder. It then randomly changes the message, it can for |
| 118 |
example replace single bytes of data or modify the length of |
| 119 |
the message. This is to simulate network errors. The |
| 120 |
break-loader program saves all the messages leading to errors |
| 121 |
so it can easily be run for a long period of time. |
| 122 |
</para> |
| 123 |
</sect2> |
| 124 |
<sect2 id="authentication"> |
| 125 |
<title>Authentication</title> |
| 126 |
<para> |
| 127 |
For testing authentication, there is a testing feature that |
| 128 |
can read authentication sequences from a file and play them |
| 129 |
back to a dummy server and client to make sure that |
| 130 |
authentication is working according to the specification. |
| 131 |
</para> |
| 132 |
<figure> |
| 133 |
<title>Example of an authentication script</title> |
| 134 |
<programlisting> |
| 135 |
## this tests a successful auth of type EXTERNAL |
| 136 |
|
| 137 |
SERVER |
| 138 |
SEND 'AUTH EXTERNAL USERNAME_HEX' |
| 139 |
EXPECT_COMMAND OK |
| 140 |
EXPECT_STATE WAITING_FOR_INPUT |
| 141 |
SEND 'BEGIN' |
| 142 |
EXPECT_STATE AUTHENTICATED |
| 143 |
</programlisting> |
| 144 |
</figure> |
| 145 |
</sect2> |
| 146 |
</sect1> |
| 147 |
<sect1 id="daemon"> |
| 148 |
<title>Testing the D-Bus bus daemon</title> |
| 149 |
<para> |
| 150 |
Since the D-Bus bus daemon is using the D-Bus client library it |
| 151 |
will benefit from all tests done on the client library, but |
| 152 |
there is still the issue of testing client-server communication. |
| 153 |
This is more complicated since it it may require another process |
| 154 |
running. |
| 155 |
</para> |
| 156 |
<sect2 id="debug-transport"> |
| 157 |
<title>The debug transport</title> |
| 158 |
<para> |
| 159 |
In D-Bus, a <emphasis>transport</emphasis> is a class that |
| 160 |
handles sending and receiving raw data over a certain |
| 161 |
medium. The transport that is used most in D-Bus is the UNIX |
| 162 |
transport with sends and recevies data over a UNIX socket. A |
| 163 |
transport that tunnels data through X11 client messages is |
| 164 |
also under development. |
| 165 |
</para> |
| 166 |
<para> |
| 167 |
The D-Bus debug transport is a specialized transport that |
| 168 |
works in-process. This means that a client and server that |
| 169 |
exists in the same process can talk to eachother without using |
| 170 |
a socket. |
| 171 |
</para> |
| 172 |
</sect2> |
| 173 |
<sect2 id="test-bus"> |
| 174 |
<title>The test-bus program</title> |
| 175 |
<para> |
| 176 |
The test-bus program is a program that is used to test various |
| 177 |
parts of the D-Bus bus daemon; robustness and that it conforms |
| 178 |
to the specifications. |
| 179 |
</para> |
| 180 |
<para> |
| 181 |
The test program has the necessary code from the bus daemon |
| 182 |
linked in, and it uses the debug transport for |
| 183 |
communication. This means that the bus daemon code can be |
| 184 |
tested without the real bus actually running, which makes |
| 185 |
testing easier. |
| 186 |
</para> |
| 187 |
<para> |
| 188 |
The test-bus program should test all major features of the |
| 189 |
bus, such as service registration, notification when things |
| 190 |
occurs and message matching. |
| 191 |
</para> |
| 192 |
</sect2> |
| 193 |
</sect1> |
| 194 |
<sect1 id="other-tests"> |
| 195 |
<title>Other tests</title> |
| 196 |
|
| 197 |
<sect2 id="oom-robustness"> |
| 198 |
<title>Out-Of-Memory robustness</title> |
| 199 |
<para> |
| 200 |
Since D-Bus should be able to be used in embedded devices, and |
| 201 |
also as a system service, it should be able to cope with |
| 202 |
low-memory situations without exiting or crashing. |
| 203 |
</para> |
| 204 |
<para> |
| 205 |
In practice, this means that both the client and server code |
| 206 |
must be able to handle dbus_malloc returning NULL. |
| 207 |
</para> |
| 208 |
<para> |
| 209 |
To test this, two environment variables |
| 210 |
exist. <literal>DBUS_MALLOC_FAIL_NTH</literal> will make every |
| 211 |
nth call to dbus_malloc return NULL, and |
| 212 |
<literal>DBUS_MALLOC_FAIL_GREATER_THAN</literal> will make any |
| 213 |
dbus_malloc call with a request for more than the specified |
| 214 |
number of bytes fail. |
| 215 |
</para> |
| 216 |
</sect2> |
| 217 |
|
| 218 |
<sect2 id="leaks-and-other-stuff"> |
| 219 |
<title>Memory leaks and code robustness</title> |
| 220 |
<para> |
| 221 |
Naturally there are some things that tests can't be written |
| 222 |
for, for example things like memory leaks and out-of-bounds |
| 223 |
memory reading or writing. |
| 224 |
</para> |
| 225 |
<para> |
| 226 |
Luckily there exists good tools for catching such errors. One |
| 227 |
free good tool is <ulink url="http://devel-home.kde.org/~sewardj/">Valgrind</ulink>, which runs the program in a |
| 228 |
virtual CPU which makes catching errors easy. All test programs can be run under Valgrind, |
| 229 |
</para> |
| 230 |
</sect2> |
| 231 |
</sect1> |
| 232 |
</article> |