Debian Codegraph @

deps/curl/tests/runtests.1

raw
1 .\" **************************************************************************
2 .\" * _ _ ____ _
3 .\" * Project ___| | | | _ \| |
4 .\" * / __| | | | |_) | |
5 .\" * | (__| |_| | _ <| |___
6 .\" * \___|\___/|_| \_\_____|
7 .\" *
8 .\" * Copyright (C) 1998 - 2019, Daniel Stenberg, <[email protected]>, et al.
9 .\" *
10 .\" * This software is licensed as described in the file COPYING, which
11 .\" * you should have received as part of this distribution. The terms
12 .\" * are also available at https://curl.haxx.se/docs/copyright.html.
13 .\" *
14 .\" * You may opt to use, copy, modify, merge, publish, distribute and/or sell
15 .\" * copies of the Software, and permit persons to whom the Software is
16 .\" * furnished to do so, under the terms of the COPYING file.
17 .\" *
18 .\" * This software is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY
19 .\" * KIND, either express or implied.
20 .\" *
21 .\" **************************************************************************
22 .\"
23 .TH runtests.pl 1 "2 Feb 2010" "Curl 7.20.0" "runtests"
24 .SH NAME
25 runtests.pl \- run one or more test cases
26 .SH SYNOPSIS
27 .B runtests.pl [options] [test number] [!test number] [key word] [!key word]
28 .SH DESCRIPTION
29 \fIruntests.pl\fP runs one, several or all the existing test cases in curl's
30 test suite. It is often called from the root Makefile of the curl package with
31 \&'make test'.
32 .SH "TEST NUMBER"
33 If no test case number is given, all existing tests that the script can find
34 will be considered for running. You can specify single test cases to run,
35 space-separated, like "1 3 5 7 11", and you can specify a range like "45 to
36 67". You can also specify only the tests you don't want to run by listing
37 the numbers with a leading exclamation point, like "!66".
38 .P
39 It is also possible to specify tests to skip based on a key word describing
40 the test. These are specified with a leading exclamation point and the
41 key word or phrase, like "!HTTP NTLM auth". Likewise, tests to run can
42 be specified simply by specifying the unadorned key words, like "FTPS".
43 Remember that the exclamation marks and spaces will need to be quoted somehow
44 when entered at many command shells.
45 .SH OPTIONS
46 .IP "-a"
47 Continue running the rest of the test cases even if one test fails. By
48 default, the test script stops as soon as an error is detected.
49 .IP "-bN"
50 Use N as the base TCP/UDP port number on which to start the test servers.
51 .IP "-c <curl>"
52 Provide a path to a custom curl binary to run the tests with. Default is the
53 curl executable in the build tree.
54 .IP "-d"
55 Enable protocol debug: have the servers display protocol output.
56 .IP "-e"
57 Run the test event-based (if possible). This will make runtests invoke curl
58 with --test-event option. This option only works if both curl and libcurl were
59 built debug-enabled.
60 .IP "-g"
61 Run the given test(s) with gdb. This is best used on a single test case and
62 curl built --disable-shared. This then fires up gdb with command line set to
63 run the specified test case. Simply (set a break-point and) type 'run' to
64 start.
65 .IP "-h"
66 Displays a help text about this program's command line options.
67 .IP "-k"
68 Keep output and log files in log/ after a test run, even if no error was
69 detected. Useful for debugging.
70 .IP "-l"
71 Lists all test case names.
72 .IP "-n"
73 Disable the check for and use of valgrind.
74 .IP "-p"
75 Prints out all files in "log/" to stdout when a test case fails. Very
76 practical when used in the automated and distributed tests since then the
77 people checking the failures and the reasons for them might not have physical
78 access to the machine and logs.
79 .IP "-R"
80 Run the tests in a scrambled, or randomized, order instead of sequentially.
81
82 The random seed initially set for this is fixed per month and can be set with
83 \fI--seed\fP.
84 .IP "-r"
85 Display run time statistics. (Requires Perl Time::HiRes module)
86 .IP "-rf"
87 Display full run time statistics. (Requires Perl Time::HiRes module)
88 .IP "--repeat=[num]"
89 This will repeat the given set of test numbers this many times. If no test
90 numbers are given, it will repeat ALL tests this many times. It iteratively
91 adds the new sequence at the end of the initially given one.
92
93 If \fB-R\fP is also used, the scrambling is done after the repeats have
94 extended the test sequence.
95 .IP "-s"
96 Shorter output. Speaks less than default.
97 .IP "--seed=[num]"
98 When using \fI--shallow\fP or \fI-R\rP that random certain aspects of the
99 behavior, this option can set the initial seed. If not set, the random seed
100 will be set based on the currently set local year and month and the first line
101 of the "curl -V" output.
102 .IP "--shallow=[num]"
103 Used together with \fB-t\fP. This limits the number of tests to fail in
104 torture mode to no more than 'num' per test case. If this reduces the amount,
105 the script will randomly discard entries to fail until the amount is 'num'.
106
107 The random seed initially set for this is fixed per month and can be set with
108 \fI--seed\fP.
109 .IP "-t[num]"
110 Selects a \fBtorture\fP test for the given tests. This makes runtests.pl first
111 run the tests once and count the number of memory allocations made. It then
112 reruns the test that number of times, each time forcing one of the allocations
113 to fail until all allocs have been tested. By setting \fInum\fP you can force
114 the allocation with that number to be set to fail at once instead of looping
115 through everyone, which is very handy when debugging and then often in
116 combination with \fI-g\fP.
117 .IP "-v"
118 Enable verbose output. Speaks more than default.
119 .IP "-vc <curl>"
120 Provide a path to a custom curl binary to run when verifying that the servers
121 running are indeed our test servers. Default is the curl executable in the
122 build tree.
123 .SH "RUNNING TESTS"
124 Many tests have conditions that must be met before the test case can run
125 fine. They could depend on built-in features in libcurl or features present in
126 the operating system or even in third-party libraries that curl may or may not
127 use.
128 .P
129 The test script checks most of these by itself to determine when it is
130 safe to attempt to run each test. Those which cannot be run due to
131 failed requirements will simply be skipped and listed at the completion
132 of all test cases. In some unusual configurations, the test script
133 cannot make the correct determination for all tests. In these cases,
134 the problematic tests can be skipped using the "!keyword" skip feature
135 documented earlier.
136 .SH "WRITING TESTS"
137 The simplest way to write test cases is to start with a similar existing test,
138 save it with a new number and then adjust it to fit. There's an attempt to
139 document the test case file format in the tests/FILEFORMAT.