Debian Codegraph @

Lib/idlelib/idle_test/README.txt

raw
1 README FOR IDLE TESTS IN IDLELIB.IDLE_TEST
2
3 0. Quick Start
4
5 Automated unit tests were added in 3.3 for Python 3.x.
6 To run the tests from a command line:
7
8 python -m test.test_idle
9
10 Human-mediated tests were added later in 3.4.
11
12 python -m idlelib.idle_test.htest
13
14
15 1. Test Files
16
17 The idle directory, idlelib, has over 60 xyz.py files. The idle_test
18 subdirectory contains test_xyz.py for each implementation file xyz.py.
19 To add a test for abc.py, open idle_test/template.py and immediately
20 Save As test_abc.py. Insert 'abc' on the first line, and replace
21 'zzdummy' with 'abc.
22
23 Remove the imports of requires and tkinter if not needed. Otherwise,
24 add to the tkinter imports as needed.
25
26 Add a prefix to 'Test' for the initial test class. The template class
27 contains code needed or possibly needed for gui tests. See the next
28 section if doing gui tests. If not, and not needed for further classes,
29 this code can be removed.
30
31 Add the following at the end of abc.py. If an htest was added first,
32 insert the import and main lines before the htest lines.
33
34 if __name__ == "__main__":
35 from unittest import main
36 main('idlelib.idle_test.test_abc', verbosity=2, exit=False)
37
38 The ', exit=False' is only needed if an htest follows.
39
40
41
42 2. GUI Tests
43
44 When run as part of the Python test suite, Idle GUI tests need to run
45 test.support.requires('gui'). A test is a GUI test if it creates a
46 tkinter.Tk root or master object either directly or indirectly by
47 instantiating a tkinter or idle class. GUI tests cannot run in test
48 processes that either have no graphical environment available or are not
49 allowed to use it.
50
51 To guard a module consisting entirely of GUI tests, start with
52
53 from test.support import requires
54 requires('gui')
55
56 To guard a test class, put "requires('gui')" in its setUpClass function.
57 The template.py file does this.
58
59 To avoid interfering with other GUI tests, all GUI objects must be
60 destroyed and deleted by the end of the test. The Tk root created in a
61 setUpX function should be destroyed in the corresponding tearDownX and
62 the module or class attribute deleted. Others widgets should descend
63 from the single root and the attributes deleted BEFORE root is
64 destroyed. See https://bugs.python.org/issue20567.
65
66 @classmethod
67 def setUpClass(cls):
68 requires('gui')
69 cls.root = tk.Tk()
70 cls.text = tk.Text(root)
71
72 @classmethod
73 def tearDownClass(cls):
74 del cls.text
75 cls.root.update_idletasks()
76 cls.root.destroy()
77 del cls.root
78
79 The update_idletasks call is sometimes needed to prevent the following
80 warning either when running a test alone or as part of the test suite
81 (#27196). It should not hurt if not needed.
82
83 can't invoke "event" command: application has been destroyed
84 ...
85 "ttk::ThemeChanged"
86
87 If a test creates instance 'e' of EditorWindow, call 'e._close()' before
88 or as the first part of teardown. The effect of omitting this depends
89 on the later shutdown. Then enable the after_cancel loop in the
90 template. This prevents messages like the following.
91
92 bgerror failed to handle background error.
93 Original error: invalid command name "106096696timer_event"
94 Error in bgerror: can't invoke "tk" command: application has been destroyed
95
96 Requires('gui') causes the test(s) it guards to be skipped if any of
97 these conditions are met:
98
99 - The tests are being run by regrtest.py, and it was started without
100 enabling the "gui" resource with the "-u" command line option.
101
102 - The tests are being run on Windows by a service that is not allowed
103 to interact with the graphical environment.
104
105 - The tests are being run on Linux and X Windows is not available.
106
107 - The tests are being run on Mac OSX in a process that cannot make a
108 window manager connection.
109
110 - tkinter.Tk cannot be successfully instantiated for some reason.
111
112 - test.support.use_resources has been set by something other than
113 regrtest.py and does not contain "gui".
114
115 Tests of non-GUI operations should avoid creating tk widgets. Incidental
116 uses of tk variables and messageboxes can be replaced by the mock
117 classes in idle_test/mock_tk.py. The mock text handles some uses of the
118 tk Text widget.
119
120
121 3. Running Unit Tests
122
123 Assume that xyz.py and test_xyz.py both end with a unittest.main() call.
124 Running either from an Idle editor runs all tests in the test_xyz file
125 with the version of Python running Idle. Test output appears in the
126 Shell window. The 'verbosity=2' option lists all test methods in the
127 file, which is appropriate when developing tests. The 'exit=False'
128 option is needed in xyx.py files when an htest follows.
129
130 The following command lines also run all test methods, including
131 GUI tests, in test_xyz.py. (Both '-m idlelib' and '-m idlelib.idle'
132 start Idle and so cannot run tests.)
133
134 python -m idlelib.xyz
135 python -m idlelib.idle_test.test_xyz
136
137 The following runs all idle_test/test_*.py tests interactively.
138
139 >>> import unittest
140 >>> unittest.main('idlelib.idle_test', verbosity=2)
141
142 The following run all Idle tests at a command line. Option '-v' is the
143 same as 'verbosity=2'.
144
145 python -m unittest -v idlelib.idle_test
146 python -m test -v -ugui test_idle
147 python -m test.test_idle
148
149 IDLE tests are 'discovered' by idlelib.idle_test.__init__.load_tests
150 when this is imported into test.test_idle. Normally, neither file
151 should be changed when working on individual test modules. The third
152 command runs unittest indirectly through regrtest. The same happens when
153 the entire test suite is run with 'python -m test'. So that command must
154 work for buildbots to stay green. IDLE tests must not disturb the
155 environment in a way that makes other tests fail (GH-62281).
156
157 To test subsets of modules, see idlelib.idle_test.__init__. This
158 can be used to find refleaks or possible sources of "Theme changed"
159 tcl messages (GH-71383).
160
161 To run an individual Testcase or test method, extend the dotted name
162 given to unittest on the command line or use the test -m option. The
163 latter allows use of other regrtest options. When using the latter,
164 all components of the pattern must be present, but any can be replaced
165 by '*'.
166
167 python -m unittest -v idlelib.idle_test.test_xyz.Test_case.test_meth
168 python -m test -m idlelib.idle_test.text_xyz.Test_case.test_meth test_idle
169
170 The test suite can be run in an IDLE user process from Shell.
171 >>> import test.autotest # Issue 25588, 2017/10/13, 3.6.4, 3.7.0a2.
172 There are currently failures not usually present, and this does not
173 work when run from the editor.
174
175
176 4. Human-mediated Tests
177
178 Human-mediated tests are widget tests that cannot be automated but need
179 human verification. They are contained in idlelib/idle_test/htest.py,
180 which has instructions. (Some modules need an auxiliary function,
181 identified with "# htest # on the header line.) The set is about
182 complete, though some tests need improvement. To run all htests, run the
183 htest file from an editor or from the command line with:
184
185 python -m idlelib.idle_test.htest
186
187
188 5. Test Coverage
189
190 Install the coverage package into your Python 3.6 site-packages
191 directory. (Its exact location depends on the OS).
192 > python3 -m pip install coverage
193 (On Windows, replace 'python3 with 'py -3.6' or perhaps just 'python'.)
194
195 The problem with running coverage with repository python is that
196 coverage uses absolute imports for its submodules, hence it needs to be
197 in a directory in sys.path. One solution: copy the package to the
198 directory containing the cpython repository. Call it 'dev'. Then run
199 coverage either directly or from a script in that directory so that
200 'dev' is prepended to sys.path.
201
202 Either edit or add dev/.coveragerc so it looks something like this.
203 ---
204 # .coveragerc sets coverage options.
205 [run]
206 branch = True
207
208 [report]
209 # Regexes for lines to exclude from consideration
210 exclude_lines =
211 # Don't complain if non-runnable code isn't run:
212 if 0:
213 if __name__ == .__main__.:
214
215 .*# htest #
216 if not _utest:
217 if _htest:
218 ---
219 The additions for IDLE are 'branch = True', to test coverage both ways,
220 and the last three exclude lines, to exclude things peculiar to IDLE
221 that are not executed during tests.
222
223 A script like the following cover.bat (for Windows) is very handy.
224 ---
225 @echo off
226 rem Usage: cover filename [test_ suffix] # proper case required by coverage
227 rem filename without .py, 2nd parameter if test is not test_filename
228 setlocal
229 set py=f:\dev\3x\pcbuild\win32\python_d.exe
230 set src=idlelib.%1
231 if "%2" EQU "" set tst=f:/dev/3x/Lib/idlelib/idle_test/test_%1.py
232 if "%2" NEQ "" set tst=f:/dev/ex/Lib/idlelib/idle_test/test_%2.py
233
234 %py% -m coverage run --pylib --source=%src% %tst%
235 %py% -m coverage report --show-missing
236 %py% -m coverage html
237 start htmlcov\3x_Lib_idlelib_%1_py.html
238 rem Above opens new report; htmlcov\index.html displays report index
239 ---
240 The second parameter was added for tests of module x not named test_x.
241 (There were several before modules were renamed, now only one is left.)