Emeric Vigier | 2f62582 | 2012-08-06 11:09:52 -0400 | [diff] [blame] | 1 | <?xml version="1.0" standalone="no"?> |
| 2 | <!DOCTYPE article PUBLIC "-//OASIS//DTD DocBook XML V4.1.2//EN" |
| 3 | "http://www.oasis-open.org/docbook/xml/4.1.2/docbookx.dtd" |
| 4 | [ |
| 5 | ]> |
| 6 | |
| 7 | <article id="index"> |
| 8 | <articleinfo> |
| 9 | <title>D-Bus Test Plan</title> |
| 10 | <date>14 February 2003</date> |
| 11 | <authorgroup> |
| 12 | <author> |
| 13 | <firstname>Anders</firstname> |
| 14 | <surname>Carlsson</surname> |
| 15 | <affiliation> |
| 16 | <orgname>CodeFactory AB</orgname> |
| 17 | <address><email>andersca@codefactory.se</email></address> |
| 18 | </affiliation> |
| 19 | </author> |
| 20 | </authorgroup> |
| 21 | </articleinfo> |
| 22 | <sect1 id="introduction"> |
| 23 | <title>Introduction</title> |
| 24 | <para> |
| 25 | This document tries to explain the details of the test plan for D-Bus |
| 26 | </para> |
| 27 | <sect2 id="importance-of-testing"> |
| 28 | <title>The importance of testing</title> |
| 29 | <para> |
| 30 | As with any big library or program, testing is important. It |
| 31 | can help find bugs and regressions and make the code better |
| 32 | overall. |
| 33 | </para> |
| 34 | <para> |
| 35 | D-Bus is a large and complex piece of software (about 25,000 |
| 36 | lines of code for the client library, and 2,500 lines of code |
| 37 | for the bus daemon) and it's therefore important to try to make sure |
| 38 | that all parts of the software is functioning correctly. |
| 39 | </para> |
| 40 | <para> |
| 41 | D-Bus can be built with support for testing by passing |
| 42 | <literal>--enable-tests</literal>. to the configure script. It |
| 43 | is recommended that production systems build without testing |
| 44 | since that reduces the D-Bus client library size. |
| 45 | </para> |
| 46 | </sect2> |
| 47 | </sect1> |
| 48 | <sect1 id="client-library"> |
| 49 | <title>Testing the D-Bus client library</title> |
| 50 | <para> |
| 51 | The tests for the client library consist of the dbus-test |
| 52 | program which is a unit test for all aspects of the client |
| 53 | library. Whenever a bug in the client library is found and |
| 54 | fixed, a test is added to make sure that the bug won't occur again. |
| 55 | </para> |
| 56 | <sect2 id="data-structures"> |
| 57 | <title>Data Structures</title> |
| 58 | <para> |
| 59 | The D-Bus client library consists of some data structures that |
| 60 | are used internally; a linked list class, a hashtable class and |
| 61 | a string class. All aspects of those are tested by dbus-test. |
| 62 | </para> |
| 63 | </sect2> |
| 64 | <sect2 id="message-loader"> |
| 65 | <title>Message loader</title> |
| 66 | <para> |
| 67 | The message loader is the part of D-Bus that takes messages in |
| 68 | raw character form and parses them, turning them into DBusMessages. |
| 69 | </para> |
| 70 | <para> |
| 71 | This is one of the parts of D-Bus that |
| 72 | <emphasis>must</emphasis> be absolutely bug-free and |
| 73 | robust. The message loader should be able to handle invalid |
| 74 | and incomplete messages without crashing. Not doing so is a |
| 75 | serious issue and can easily result in D-Bus being exploitable |
| 76 | to DoS attacks. |
| 77 | </para> |
| 78 | <para> |
| 79 | To solve these problems, there is a testing feature called the |
| 80 | Message Builder. The message builder can take a serialized |
| 81 | message in string-form and convert it into a raw character |
| 82 | string which can then be loaded by the message loader. |
| 83 | </para> |
| 84 | <figure> |
| 85 | <title>Example of a message in string form</title> |
| 86 | <programlisting> |
| 87 | # Standard org.freedesktop.DBus.Hello message |
| 88 | |
| 89 | VALID_HEADER |
| 90 | FIELD_NAME name |
| 91 | TYPE STRING |
| 92 | STRING 'org.freedesktop.DBus.Hello' |
| 93 | FIELD_NAME srvc |
| 94 | TYPE STRING |
| 95 | STRING 'org.freedesktop.DBus' |
| 96 | ALIGN 8 |
| 97 | END_LENGTH Header |
| 98 | START_LENGTH Body |
| 99 | END_LENGTH Body |
| 100 | </programlisting> |
| 101 | </figure> |
| 102 | <para> |
| 103 | The file format of messages in string form is documented in |
| 104 | the D-Bus Reference Manual. |
| 105 | </para> |
| 106 | <para> |
| 107 | The message test part of dbus-test is using the message |
| 108 | builder to build different kinds of messages, both valid, |
| 109 | invalid, and invalid ones, to make sure that the loader won't |
| 110 | crash or leak memory of any of those, and that the loader |
| 111 | knows if a message is valid or not. |
| 112 | </para> |
| 113 | <para> |
| 114 | There is also a test program called |
| 115 | <literal>break-loader</literal> that loads a message in |
| 116 | string-form into raw character form using the message |
| 117 | builder. It then randomly changes the message, it can for |
| 118 | example replace single bytes of data or modify the length of |
| 119 | the message. This is to simulate network errors. The |
| 120 | break-loader program saves all the messages leading to errors |
| 121 | so it can easily be run for a long period of time. |
| 122 | </para> |
| 123 | </sect2> |
| 124 | <sect2 id="authentication"> |
| 125 | <title>Authentication</title> |
| 126 | <para> |
| 127 | For testing authentication, there is a testing feature that |
| 128 | can read authentication sequences from a file and play them |
| 129 | back to a dummy server and client to make sure that |
| 130 | authentication is working according to the specification. |
| 131 | </para> |
| 132 | <figure> |
| 133 | <title>Example of an authentication script</title> |
| 134 | <programlisting> |
| 135 | ## this tests a successful auth of type EXTERNAL |
| 136 | |
| 137 | SERVER |
| 138 | SEND 'AUTH EXTERNAL USERNAME_HEX' |
| 139 | EXPECT_COMMAND OK |
| 140 | EXPECT_STATE WAITING_FOR_INPUT |
| 141 | SEND 'BEGIN' |
| 142 | EXPECT_STATE AUTHENTICATED |
| 143 | </programlisting> |
| 144 | </figure> |
| 145 | </sect2> |
| 146 | </sect1> |
| 147 | <sect1 id="daemon"> |
| 148 | <title>Testing the D-Bus bus daemon</title> |
| 149 | <para> |
| 150 | Since the D-Bus bus daemon is using the D-Bus client library it |
| 151 | will benefit from all tests done on the client library, but |
| 152 | there is still the issue of testing client-server communication. |
| 153 | This is more complicated since it it may require another process |
| 154 | running. |
| 155 | </para> |
| 156 | <sect2 id="debug-transport"> |
| 157 | <title>The debug transport</title> |
| 158 | <para> |
| 159 | In D-Bus, a <emphasis>transport</emphasis> is a class that |
| 160 | handles sending and receiving raw data over a certain |
| 161 | medium. The transport that is used most in D-Bus is the UNIX |
| 162 | transport with sends and recevies data over a UNIX socket. A |
| 163 | transport that tunnels data through X11 client messages is |
| 164 | also under development. |
| 165 | </para> |
| 166 | <para> |
| 167 | The D-Bus debug transport is a specialized transport that |
| 168 | works in-process. This means that a client and server that |
| 169 | exists in the same process can talk to eachother without using |
| 170 | a socket. |
| 171 | </para> |
| 172 | </sect2> |
| 173 | <sect2 id="bus-test"> |
| 174 | <title>The bus-test program</title> |
| 175 | <para> |
| 176 | The bus-test program is a program that is used to test various |
| 177 | parts of the D-Bus bus daemon; robustness and that it conforms |
| 178 | to the specifications. |
| 179 | </para> |
| 180 | <para> |
| 181 | The test program has the necessary code from the bus daemon |
| 182 | linked in, and it uses the debug transport for |
| 183 | communication. This means that the bus daemon code can be |
| 184 | tested without the real bus actually running, which makes |
| 185 | testing easier. |
| 186 | </para> |
| 187 | <para> |
| 188 | The bus-test program should test all major features of the |
| 189 | bus, such as service registration, notification when things |
| 190 | occurs and message matching. |
| 191 | </para> |
| 192 | </sect2> |
| 193 | </sect1> |
| 194 | <sect1 id="other-tests"> |
| 195 | <title>Other tests</title> |
| 196 | |
| 197 | <sect2 id="oom-robustness"> |
| 198 | <title>Out-Of-Memory robustness</title> |
| 199 | <para> |
| 200 | Since D-Bus should be able to be used in embedded devices, and |
| 201 | also as a system service, it should be able to cope with |
| 202 | low-memory situations without exiting or crashing. |
| 203 | </para> |
| 204 | <para> |
| 205 | In practice, this means that both the client and server code |
| 206 | must be able to handle dbus_malloc returning NULL. |
| 207 | </para> |
| 208 | <para> |
| 209 | To test this, two environment variables |
| 210 | exist. <literal>DBUS_MALLOC_FAIL_NTH</literal> will make every |
| 211 | nth call to dbus_malloc return NULL, and |
| 212 | <literal>DBUS_MALLOC_FAIL_GREATER_THAN</literal> will make any |
| 213 | dbus_malloc call with a request for more than the specified |
| 214 | number of bytes fail. |
| 215 | </para> |
| 216 | </sect2> |
| 217 | |
| 218 | <sect2 id="leaks-and-other-stuff"> |
| 219 | <title>Memory leaks and code robustness</title> |
| 220 | <para> |
| 221 | Naturally there are some things that tests can't be written |
| 222 | for, for example things like memory leaks and out-of-bounds |
| 223 | memory reading or writing. |
| 224 | </para> |
| 225 | <para> |
| 226 | Luckily there exists good tools for catching such errors. One |
| 227 | free good tool is <ulink url="http://devel-home.kde.org/~sewardj/">Valgrind</ulink>, which runs the program in a |
| 228 | virtual CPU which makes catching errors easy. All test programs can be run under Valgrind, |
| 229 | </para> |
| 230 | </sect2> |
| 231 | </sect1> |
| 232 | </article> |