Debian Codegraph @

tests/README

raw
1 file: florist/tests/README [$Revision$]
2
3 THIS SOFTWARE AND THE ACCOMPANYING DOCUMENTATION ARE DISTRIBUTED
4 IN THE HOPE THAT IT WILL BE USEFUL, BUT WITHOUT ANY WARRANTY;
5 WITHOUT EVEN THE IMPLIED WARRANTY OF MERCHANTABILITY OR FITNESS
6 FOR A PARTICULAR PURPOSE -- NOTWITHSTANDING ANY STATEMENTS MADE
7 BELOW IN THIS FILE.
8
9 POSIX.5B VALIDATION TESTS:
10
11 This directory contains working drafts of validation tests for
12 IEEE Standard 1003.5b. These test programs, which are derived
13 from programs originally written as tests of Florist, have been
14 developed under contract to Logicon Eagle Technologies for the
15 U.S. Department of Defense, Defense Information Systems Agency,
16 Center for Standards.
17
18 PREREQUISITES:
19
20 The tests are supposed to be 100% portable, but the procedures
21 for compiling them and running them will depend on the environment
22 in which you are working.
23
24 The Makefile here has been used successfully in the following
25 environment:
26
27 make utility : GNU Make version 3.74
28 Ada compiler : GNATMAKE 3.13a
29 POSIX.5b implementation : Florist 1.4, 31 Dec 1998
30 OS : Solaris 2.7
31 architecture : SPARCstation 20 HS14
32
33 The Makefile depends on the "make" utility, how to call the Ada
34 compiler, and where the sources and library files associated with
35 the implementation of POSIX.5b are installed.
36
37 The Makefile in this directory is designed to work with the Gnu
38 "make" utility. (It will probably work with other "make"
39 utilities, except for the pattern-matching rules, which use the
40 extended Gnu syntax.)
41
42 The Makefile is set up to work with the Florist implementation of
43 POSIX.5/.5b, and the GNAT Ada compiler. It assumes you have
44 compiled Florist, that the sources and object library are in the
45 directory ./floristlib.
46
47 With this Makefile you can compile all the tests using the command
48 "make tests". You can run all the tests by typing "make
49 run_tests". The tests will be run in a subdirectory called
50 "test.dir", and the results will be logged to the file
51 "test.dir/run_tests_1.log". It will take quite a long time to run
52 all the tests, even when they all pass. In particular, the "p99*"
53 series of tests normally take a long time. There is also high
54 probability that one of the tests may crash the test run, or
55 "hang". In the latter case, you will need to kill the test run,
56 and run the remaining tests in smaller groups. Try commenting the
57 bad tests out of the script "run_tests_1", and running them
58 individually.
59
60 VERBOSITY CONTROLS:
61
62 You can modify how much output the test produces, using the
63 following command line options.
64
65 -t = Terse output. This eliminates all but the first and last
66 lines of output
67
68 -v = Verbose output. This provides additional output, that
69 may be helpful in narrowing down the nature of a test
70 failure.
71
72 The output of the default verbosity level is described below.
73
74 NORMAL TEST OUTPUT:
75
76 Each test will produce an initial line giving the name of the test
77 and some version information, e.g.
78
79 ,.,. p020400 POSIX Ada Validation Tests, Version 1.2
80
81 By default, the test will then produce a series of lines to
82 indicate its progress through various internal subdivisions, like
83 the following:
84
85 ---- *-Subtest: package POSIX
86 ---- *-Subtest: version query functions
87 ---- *-Subtest: optional facility subtypes
88 ---- *-Subtest: bytes and I/O counts
89 ---- *-Subtest: Blocking_Behavior type
90 ---- *-Subtest: Signal_Masking type
91 ---- *-Subtest: POSIX_Character
92 ---- *-Subtest: POSIX_String type
93 ---- *-Subtest: POSIX_String_List type
94 ---- *-Subtest: Option_Set type
95 ---- *-Subtest: Error_Code type
96 ---- *-Subtest: uname-derived functions
97 ---- *-Subtest: Timespec type
98
99 If the test passes it will then terminate and produce the message:
100
101 ==== Test Completed Successfully.
102
103 ORDINARY FAILURES:
104
105 Any messages of the following form indicate a failure at some
106 point within the test:
107
108 !!TEST FAILED: ...
109
110 If the test fails and successfully terminates, it will produce a
111 message of the form:
112
113 ==== Failed ...some number... tests.
114
115 EXTREME FAILURES:
116
117 Some tests may hang, or terminate without completing. We have
118 tried to design the tests so that they can tolerate some
119 anticipated kinds of failures and still run to completion, but the
120 range of possible failures is practically infinite. In these
121 cases, it may help to use the "-v" option to see more precisely
122 how far the test got before the problem occurred.
123
124 UNSUPPORTED FEATURES:
125
126 Some tests will produce "Nonsupport of ......" message after the
127 "Completed successfully" message. This indicates that the feature
128 you are testing is not provided by the POSIX Ada binding
129 implementation. This should only happen if it the corresponding
130 capablity is not provided by the underlying operating system.
131 This is not a failure, unless the implementation actually claims
132 to support the feature, or there are other tests where the feature
133 appears to be supported.
134
135 You can use the "-v" command-line argument option when you run the
136 test to see messages indicating each of the points where the
137 test found a feature unsupported.
138
139 PRIVILEGE FAILURES:
140
141 Some tests attempt to perform operations that may require special
142 privilege. These tests passed when we ran them on Solaris 2.6
143 with "root" privilege, but running strange programs with root
144 permission is always a risky thing to do. You will need to review
145 these tests and determine what needs to be done on your system to
146 give them the appropriate privilege, and arrange to test them in
147 an environment where they will do no serious harm if the run amok.
148
149 TAILORABLE FEATURES:
150
151 The POSIX standards provide considerable leeway for variation
152 among implementations. For example, an implementation may impose
153 arbitrary restrictions on the form of the name of a message queue.
154 In order to accomodate these variations, we have attempted to
155 parameterize the tests, and to isolate the implementation dependent
156 constants (such as message queue names) in a single package, named
157 Test_Parameters. You may need to edit this package to fit the
158 implementation being tested.
159
160 OTHER SETUP:
161
162 Some tests require other external setup, before the test program
163 is executed. For example, some require setting an environment
164 variable, and some require putting an executable file in a known
165 location. See the script "run_tests_1" for examples.
166
167 STATUS:
168
169 These are the final versions of the tests delivered to the
170 Government under the POSIX.5b Test Suite project.