| 1 |
This README file describes the files and directories related -*- rst -*- |
| 2 |
to the Python test suite under the current 'test' directory. |
| 3 |
|
| 4 |
- dotest.py |
| 5 |
|
| 6 |
Provides the test driver for the test suite. To invoke it, cd to the 'test' |
| 7 |
directory and issue the './dotest.py' command or './dotest.py -v' for more |
| 8 |
verbose output. '.dotest.py -h' prints out the help messge. |
| 9 |
|
| 10 |
A specific naming pattern is followed by the .py script under the 'test' |
| 11 |
directory in order to be recognized by 'dotest.py' test driver as a module |
| 12 |
which implements a test case, namely, Test*.py. |
| 13 |
|
| 14 |
Some example usages: |
| 15 |
|
| 16 |
1. ./dotest.py -v . 2> ~/Developer/Log/lldbtest.log0 |
| 17 |
This runs the test suite and directs the run log to a file. |
| 18 |
|
| 19 |
2. LLDB_LOG=/tmp/lldb.log GDB_REMOTE_LOG=/tmp/gdb-remote.log ./dotest.py -v . 2> ~/Developer/Log/lldbtest.log |
| 20 |
This runs the test suite, with logging turned on for the lldb as well as |
| 21 |
the process.gdb-remote channels and directs the run log to a file. |
| 22 |
|
| 23 |
- lldbtest.py |
| 24 |
|
| 25 |
Provides an abstract base class of lldb test case named 'TestBase', which in |
| 26 |
turn inherits from Python's unittest.TestCase. The concrete subclass can |
| 27 |
override lldbtest.TestBase in order to inherit the common behavior for |
| 28 |
unittest.TestCase.setUp/tearDown implemented in this file. |
| 29 |
|
| 30 |
To provide a test case, the concrete subclass provides methods whose names |
| 31 |
start with the letters test. For more details about the Python's unittest |
| 32 |
framework, go to http://docs.python.org/library/unittest.html. |
| 33 |
|
| 34 |
./command_source/TestCommandSource.py provides a simple example of test case |
| 35 |
which overrides lldbtest.TestBase to exercise the lldb's 'command source' |
| 36 |
command. |
| 37 |
|
| 38 |
The doc string provides more details about the setup required for running a |
| 39 |
test case on its own. To run the whole test suite, 'dotest.py' is all you |
| 40 |
need to do. |
| 41 |
|
| 42 |
- subdirectories of 'test' |
| 43 |
|
| 44 |
Most of them predate the introduction of the python test suite and contain |
| 45 |
example C/C++/ObjC source files which get compiled into executables which are |
| 46 |
to be exercised by the debugger. |
| 47 |
|
| 48 |
For such subdirectory which has an associated Test*.py file, it was added as |
| 49 |
part of the Python-based test suite to test lldb functionality. |
| 50 |
|
| 51 |
Some of the subdirectories, for example, the 'help' subdirectory, do not have |
| 52 |
C/C++/ObjC source files; they were created to house the Python test case which |
| 53 |
does not involve lldb reading in an executable file at all. |
| 54 |
|
| 55 |
The sample_test directory contains examples of both a full and an "inline" |
| 56 |
testcase that run a process to a breakpoint and check a local variable. These |
| 57 |
are convenient starting points for adding new tests. |
| 58 |
|
| 59 |
- make directory |
| 60 |
|
| 61 |
Contains Makefile.rules, which can be utilized by test cases to write Makefile |
| 62 |
based rules to build binaries for the inferiors. |
| 63 |
|
| 64 |
By default, the built executable name is a.out, which can be overwritten by |
| 65 |
specifying your EXE make variable, via the Makefile under the specific test |
| 66 |
directory or via supplying a Python dictionary to the build method in your |
| 67 |
Python test script. An example of the latter can be found in |
| 68 |
test/lang/objc/radar-9691614/TestObjCMethodReturningBOOL.py, where: |
| 69 |
|
| 70 |
def test_method_ret_BOOL(self): |
| 71 |
"""Test that objective-c method returning BOOL works correctly.""" |
| 72 |
d = {'EXE': self.exe_name} |
| 73 |
self.build(dictionary=d) |
| 74 |
self.setTearDownCleanup(dictionary=d) |
| 75 |
... |
| 76 |
|
| 77 |
def setUp(self): |
| 78 |
# Call super's setUp(). |
| 79 |
TestBase.setUp(self) |
| 80 |
# We'll use the test method name as the exe_name. |
| 81 |
self.exe_name = self.testMethodName |
| 82 |
# Find the line number to break inside main(). |
| 83 |
self.main_source = "main.m" |
| 84 |
self.line = line_number(self.main_source, '// Set breakpoint here.') |
| 85 |
|
| 86 |
The exe names for the two test methods are equal to the test method names and |
| 87 |
are therefore guaranteed different. |
| 88 |
|
| 89 |
- plugins directory |
| 90 |
|
| 91 |
Contains platform specific plugin to build binaries with dsym/dwarf debugging |
| 92 |
info. Other platform specific functionalities may be added in the future. |
| 93 |
|
| 94 |
- Profiling dotest.py runs |
| 95 |
|
| 96 |
I used the following command line thingy to do the profiling on a SnowLeopard |
| 97 |
machine: |
| 98 |
|
| 99 |
$ DOTEST_PROFILE=YES DOTEST_SCRIPT_DIR=/Volumes/data/lldb/svn/trunk/test /System/Library/Frameworks/Python.framework/Versions/Current/lib/python2.6/cProfile.py -o my.profile ./dotest.py -v -w 2> ~/Developer/Log/lldbtest.log |
| 100 |
|
| 101 |
After that, I used the pstats.py module to browse the statistics: |
| 102 |
|
| 103 |
$ python /System/Library/Frameworks/Python.framework/Versions/Current/lib/python2.6/pstats.py my.profile |
| 104 |
|
| 105 |
- Writing test cases: |
| 106 |
|
| 107 |
We strongly prefer writing test cases using the SB API's rather than |
| 108 |
the runCmd & expect. Unless you are actually testing some feature |
| 109 |
of the command line, please don't write command based tests. For |
| 110 |
historical reasons there are plenty of examples of tests in the test |
| 111 |
suite that use runCmd where they shouldn't, but don't copy them, |
| 112 |
copy the plenty that do use the SB API's instead. |
| 113 |
|
| 114 |
The reason for this is that our policy is that we will maintain |
| 115 |
compatibility with the SB API's. But we don't make any similar |
| 116 |
guarantee about the details of command result format. If your test |
| 117 |
is using the command line, it is going to have to check against the |
| 118 |
command result text, and you either end up writing your check |
| 119 |
pattern by checking as little as possible so you won't be exposed to |
| 120 |
random changes in the text; in which case you can end up missing |
| 121 |
some failure, or you test too much and it means irrelevant changes |
| 122 |
break your tests. |
| 123 |
|
| 124 |
However, if you use the Python API's it is possible to check all the |
| 125 |
results you want to check in a very explicit way, which makes the |
| 126 |
tests much more robust. |
| 127 |
|
| 128 |
Even if you are testing that a command-line command does some |
| 129 |
specific thing, it is still better in general to use the SB API's to |
| 130 |
drive to the point where you want to run the test, then use |
| 131 |
SBInterpreter::HandleCommand to run the command. You get the full |
| 132 |
result text from the command in the command return object, and all |
| 133 |
the part where you are driving the debugger to the point you want to |
| 134 |
test will be more robust. |
| 135 |
|
| 136 |
The sample_test directory contains a standard and an "inline" test |
| 137 |
that are good starting points for writing a new test. |
| 138 |
|
| 139 |
- Attaching in test cases: |
| 140 |
|
| 141 |
If you need to attach to inferiors in your tests, you must make sure |
| 142 |
the inferior calls lldb_enable_attach(), before the debugger |
| 143 |
attempts to attach. This function performs any platform-specific |
| 144 |
processing needed to enable attaching to this process (e.g., on |
| 145 |
Linux, we execute prctl(PR_SET_TRACER) syscall to disable |
| 146 |
protections present in some Linux systems). |