Debian Codegraph @

csb/test/__init__.py

raw ◀ ▶
2 Debian patches touch this file
1 """
2 This is a top level package, hosting the entire CSB test framework. It is divided
3 into several major parts:
4
5 - test cases, located under csb.test.cases
6 - test data, in C{/csb/test/data} (not a package)
7 - test console, in C{/csb/test/app.py}
8
9 This module, csb.test, contains all the glue-code functions, classes and
10 decorators you would need in order to write tests for CSB.
11
12 1. Configuration and Tree
13
14 L{Config<csb.test.Config>} is a common config object shared between CSB
15 tests. Each config instance contains properties like:
16
17 - data: the data folder, automatically discovered and loaded in
18 csb.test.Config.DATA at module import time
19 - temp: a default temp folder, which test cases can use
20
21 Each L{Config<csb.test.Config>} provides a convenient way to retrieve
22 files from C{/csb/test/data}. Be sure to check out L{Config.getTestFile}
23 and L{Config.getPickle}. In case you need a temp file, use
24 L{Config.getTempStream} or have a look at L{csb.io.TempFile} and
25 L{csb.io.TempFolder}.
26
27 All test data files should be placed in the C{data} folder. All test
28 modules must be placed in the root package: csb.test.cases. There is
29 a strict naming convention for test modules: the name of a test module
30 should be the same as the name of the CSB API package it tests. For
31 example, if you are writing tests for C{csb/bio/io/__init__.py}, the
32 test module must be C{csb/test/cases/bio/io/__init__.py}. C{csb.test.cases}
33 is the root package of all test modules in CSB.
34
35 2. Writing Tests
36
37 Writing a test is easy. All you need is to import csb.test and then
38 create your own test cases, derived from L{csb.test.Case}:
39
40 >>> import csb.test
41 >>> @csb.test.unit
42 class TestSomeClass(csb.test.Case):
43 def setUp(self):
44 super(TestSomeClass, self).setUp()
45 # do something with self.config here...
46
47 In this way your test case instance is automatically equipped with a
48 reference to the test config, so your test method can be:
49
50 >>> @csb.test.unit
51 class TestSomeClass(csb.test.Case):
52 def testSomeMethod(self):
53 myDataFile = self.config.getTestFile('some.file')
54 self.assert...
55
56 The "unit" decorator marks a test case as a collection of unit tests.
57 All possibilities are: L{csb.test.unit}, L{csb.test.functional}, L{csb.test.custom},
58 and L{csb.test.regression}.
59
60 Writing custom (a.k.a. "data", "slow", "dynamic") tests is a little bit
61 more work. Custom tests must be functions, not classes. Basically a
62 custom test is a function, which builds a unittest.TestSuite instance
63 and then returns it when called without arguments.
64
65 Regression tests are usually created in response to reported bugs. Therefore,
66 the best practice is to mark each test method with its relevant bug ID:
67
68 >>> @csb.test.regression
69 class SomeClassRegressions(csb.test.Case)
70 def testSomeFeature(self)
71 \"""
72 @see: [CSB 000XXXX]
73 \"""
74 # regression test body...
75
76 3. Style Guide:
77
78 - name test case packages as already described
79 - group tests in csb.test.Case-s and name them properly
80 - prefix test methods with "test", like "testParser" - very important
81 - use camelCase for methods and variables. This applies to all the
82 code under csb.test (including test) and does not apply to the rest
83 of the library!
84 - for functional tests it's okay to define just one test method: runTest
85 - for unit tests you should create more specific test names, for example:
86 "testParseFile" - a unit test for some method called "parse_file"
87 - use csb.test decorators to mark tests as unit, functional, regression, etc.
88 - make every test module executable::
89
90 if __name__ == '__main__':
91 csb.test.Console() # Discovers and runs all test cases in the module
92
93 4. Test Execution
94
95 Test discovery is handled by C{test builders} and a test runner
96 C{app}. Test builders are subclasses of L{AbstractTestBuilder}.
97 For every test type (unit, functional, regression, custom) there is a
98 corresponding test builder. L{AnyTestBuilder} is a special builder which
99 scans for unit, regression and functional tests at the same time.
100
101 Test builder classes inherit the following test discovery methods:
102
103 - C{loadTests} - load tests from a test namespace. Wildcard
104 namespaces are handled by C{loadAllTests}
105 - C{loadAllTests} - load tests from the given namespace, and
106 from all sub-packages (recursive)
107 - C{loadFromFile} - load tests from an absolute file name
108 - C{loadMultipleTests} - calls C{loadTests} for a list of
109 namespaces and combines all loaded tests in a single suite
110
111 Each of those return test suite objects, which can be directly executed
112 with python's unittest runner.
113
114 Much simpler way to execute a test suite is to use our test app
115 (C{csb/test/app.py}), which is simply an instance of L{csb.test.Console}::
116
117 $ python csb/test/app.py --help
118
119 The app has two main arguments:
120
121 - test type - tells the app which TestBuilder to use for test dicsovery
122 ("any" triggers L{AnyTestBuilder}, "unit" - L{UnitTestBuilder}, etc.)
123 - test namespaces - a list of "dotted" test modules, for example::
124
125 csb.test.cases.bio.io.* # io and sub-packages
126 csb.test.cases.bio.utils # only utils
127 . # current module
128
129 In addition to running the app from the command line, you can run it
130 also programmatically by instantiating L{csb.test.Console}. You can
131 construct a test console object by passing a list of test namespace(s)
132 and a test builder class to the Console's constructor.
133
134
135 5. Commit Policies
136
137 Follow these guidelines when making changes to the repository:
138
139 - B{no bugs in "trunk"}: after fixing a bug or implementing a new
140 feature, make sure at least the default test set passes by running
141 the test console without any arguments. This is equivalent to:
142 app.py -t any "csb.test.cases.*". (If no test case from this set covers
143 the affected code, create a test case first, as described in the other
144 policies)
145
146 - B{no recurrent issues}: when a bug is found, first write a regression
147 test with a proper "@see: BugID" tag in the docstring. Run the test
148 to make sure it fails. After fixing the bug, run the test again before
149 you commit, as required by the previous policy
150
151 - B{test all new features}: there should be a test case for every new feature
152 we implement. One possible approach is to write a test case first and
153 make sure it fails; when the new feature is ready, run the test again
154 to make sure it passes
155
156 @warning: for compatibility reasons do NOT import and use the unittest module
157 directly. Always import unittest from csb.test, which is guaranteed
158 to be python 2.7+ compatible.
159 """
160 import os
161 import sys
•162 import importlib
163 import types
164 import time
165 import tempfile
166 import traceback
167 import argparse
168
169 import csb.io
170 import csb.core
171
172 try:
173 from unittest import skip, skipIf
174 import unittest
175 except ImportError:
176 import unittest2 as unittest
177
178 from abc import ABCMeta, abstractproperty
179
180
181 class Attributes(object):
182
183 UNIT = '__CSBUnitTest__'
184 CUSTOM = '__CSBCustomTest__'
185 FUNCTIONAL = '__CSBFunctionalTest__'
186 REGRESSION = '__CSBRegressionTest__'
187
188 class Config(object):
189 """
190 General CSB Test Config. Config instances contain the following properties:
191
192 - data - path to the CSB Test Data directory. Default is L{Config.DATA}
193 - temp - path to the system's temp directory. Default is L{Config.TEMP}
194 - config - the L{Config} class
195 """
196
197 DATA = os.path.join(os.path.abspath(os.path.dirname(__file__)), 'data')
198 """
199 @cvar: path to the default test data directory: <install dir>/csb/test/data
200 """
201 GENERATED_DATA = DATA
202 """
203 @cvar: path to the default data directory for generated test files
204 """
205 TEMP = os.path.abspath(tempfile.gettempdir())
206 """
207 @cvar: path to the default system's temp directory
208 """
209
210 @staticmethod
211 def setDefaultDataRoot(path):
212 """
213 Override the default L{Config.DATA} with a new data root directory.
214
215 @param path: full directory path
216 @type path: str
217 """
218 if not os.path.isdir(path):
219 raise IOError('Path not found: {0}'.format(path))
220
221 Config.DATA = os.path.abspath(path)
222
223 @staticmethod
224 def setDefaultGeneratedDataRoot(path):
225 """
226 Override the default L{Config.GENERATED_DATA} with a new data root directory.
227
228 @param path: full directory path
229 @type path: str
230 """
231 if not os.path.isdir(path):
232 raise IOError('Path not found: {0}'.format(path))
233
234 Config.GENERATED_DATA = os.path.abspath(path)
235
236 @property
237 def data(self):
238 """
239 Test data directory
240 @rtype: str
241 """
242 return Config.DATA
243
244 @property
245 def generatedData(self):
246 """
247 Test data directory for generated files
248 @rtype: str
249 """
250 return Config.GENERATED_DATA
251
252 @property
253 def temp(self):
254 """
255 Test temp directory
256 @rtype: str
257 """
258 return Config.TEMP
259
260 def getTestFile(self, fileName, subDir=''):
261 """
262 Search for C{fileName} in the L{Config.DATA} directory. If not found,
263 try also L{Config.GENERATED_DATA} (if different).
264
265 @param fileName: the name of a test file to retrieve
266 @type fileName: str
267 @param subDir: scan a sub-directory of L{Config.DATA}
268 @type subDir: str
269
270 @return: full path to C{fileName}
271 @rtype: str
272
273 @raise IOError: if no such file is found
274 """
275 for data in [self.data, self.generatedData]:
276 file = os.path.join(data, subDir, fileName)
277
278 if os.path.isfile(file):
279 return file
280
281 raise IOError('Test file not found: {0}'.format(fileName))
282
283 def getPickle(self, fileName, subDir=''):
284 """
285 Same as C{self.getTestFile}, but try to unpickle the the file
286 and return the unpickled object. Pickles are usually stored in
287 L{Config.GENERATED_DATA}.
288
289 @param fileName: the name of a test file to retrieve
290 @type fileName: str
291 @param subDir: scan a sub-directory of L{Config.DATA}
292 @type subDir: str
293
294 @rtype: object
295 """
296 file = self.getTestFile(fileName, subDir)
297 return csb.io.Pickle.load(open(file, 'rb'))
298
299 def getContent(self, fileName, subDir=''):
300 """
301 Same as C{self.getTestFile}, but also read and return the contents of
302 the file.
303
304 @param fileName: the name of a test file to retrieve
305 @type fileName: str
306 @param subDir: scan a sub-directory of L{Config.DATA}
307 @type subDir: str
308
309 @rtype: str
310 """
311 with open(self.getTestFile(fileName, subDir)) as f:
312 return f.read()
313
314 def getTempStream(self, mode='t'):
315 """
316 Return a temporary file stream::
317
318 with self.getTempStream() as tmp:
319 tmp.write(something)
320 tmp.flush()
321 file_name = tmp.name
322
323 @param mode: file open mode (text, binary), default=t
324 @type mode: str
325 @rtype: file stream
326 """
327 return csb.io.TempFile(mode=mode)
328
329 def ensureDataConsistency(self):
330 """
331 Try to deserialize some pickled data files. Call L{Config.updateDataFiles}
332 if the pickles appeared incompatible with the current interpreter.
333 """
334 try:
335 self.getPickle('1nz9.model1.pickle')
336 except:
337 self.updateDataFiles()
338
339 def updateDataFiles(self):
340 """
341 Refresh the pickled structures in csb/test/data. This might be needed when
342 the internal representation of some classes has changed.
343 """
344 from csb.io import Pickle
345 from csb.bio.io.wwpdb import RegularStructureParser
346 from csb.bio.structure import Ensemble, ChemElements
347
348 parser = RegularStructureParser(self.getTestFile('1nz9.pdb'))
349 model1 = parser.parse_structure(model=1)
350 model2 = parser.parse_structure(model=2)
351
352 ensemble = Ensemble()
353 ensemble.models.append(model1)
354 ensemble.models.append(model2)
355 Pickle.dump(ensemble, open(os.path.join(self.generatedData, '1nz9.full.pickle'), 'wb'))
356
357 mse = model1.chains['A'].find(164)
358 mse.label = 'MSE'
359 mse.atoms['SD']._element = ChemElements.Se
360 mse.atoms['SD']._full_name = 'SE '
361 Pickle.dump(model1, open(os.path.join(self.generatedData, '1nz9.model1.pickle'), 'wb'))
362
363 class Case(unittest.TestCase):
364 """
365 Base class, defining a CSB Test Case. Provides a default implementation
366 of C{unittest.TestCase.setUp} which grabs a reference to a L{Config}.
367 """
368
369 @property
370 def config(self):
371 """
372 Test config instance
373 @rtype: L{Config}
374 """
375 return self.__config
376
377 def setUp(self):
378 """
379 Provide a reference to the CSB Test Config in the C{self.config} property.
380 """
381 self.__config = Config()
382 assert hasattr(self.config, 'data'), 'The CSB Test Config must contain the data directory'
383 assert self.config.data, 'The CSB Test Config must contain the data directory'
384
385 def reRaise(self, addArgs=()):
386 """
387 Re-raise the last exception with its full traceback, but modify the
388 argument list with C{addArgs} and the original stack trace.
389
390 @param addArgs: additional arguments to append to the exception
391 @type addArgs: tuple
392 """
393 klass, ex, _tb = sys.exc_info()
394 ex.args = list(ex.args) + list(addArgs) + [''.join(traceback.format_exc())]
395
396 raise klass(ex.args)
397
398 def assertAlmostEqual(self, first, second, places=None, msg=None, delta=None):
399
400 if first == second:
401 return
402 if delta is not None and places is not None:
403 raise TypeError("specify delta or places not both")
404
405 if delta is not None:
406
407 if abs(first - second) <= delta:
408 return
409
410 m = '{0} != {1} within {2} delta'.format(first, second, delta)
411 msg = self._formatMessage(msg, m)
412
413 raise self.failureException(msg)
414
415 else:
416 if places is None:
417 places = 7
418
419 return super(Case, self).assertAlmostEqual(first, second, places=places, msg=msg)
420
421 def assertFasterThan(self, duration, callable, *args, **kargs):
422 """
423 Fail if it took more than C{duration} seconds to invoke C{callable}.
424
425 @param duration: maximum amount of seconds allowed
426 @type duration: float
427 """
428
429 start = time.time()
430 callable(*args, **kargs)
431 execution = time.time() - start
432
433 if execution > duration:
434 self.fail('{0}s is slower than {1}s)'.format(execution, duration))
435
436 @classmethod
437 def execute(cls):
438 """
439 Run this test case.
440 """
441 suite = unittest.TestLoader().loadTestsFromTestCase(cls)
442 runner = unittest.TextTestRunner()
443
444 return runner.run(suite)
445
446 class AbstractTestBuilder(object):
447 """
448 This is a base class, defining a test loader which exposes the C{loadTests}
449 method.
450
451 Subclasses must override the C{labels} abstract property, which controls
452 what kind of test cases are loaded by the test builder.
453 """
454
455 __metaclass__ = ABCMeta
456
457 @abstractproperty
458 def labels(self):
459 pass
460
461 def loadFromFile(self, file):
462 """
463 Load L{csb.test.Case}s from a module file.
464
465 @param file: test module file name
466 @type file: str
467
468 @return: a C{unittest.TestSuite} ready for the test runner
469 @rtype: C{unittest.TestSuite}
470 """
471 mod = self._loadSource(file)
472 suite = unittest.TestLoader().loadTestsFromModule(mod)
473 return unittest.TestSuite(self._filter(suite))
474
475 def loadTests(self, namespace):
476 """
477 Load L{csb.test.Case}s from the given CSB C{namespace}. If the namespace
478 ends with a wildcard, tests from sub-packages will be loaded as well.
479 If the namespace is '__main__' or '.', tests are loaded from __main__.
480
481 @param namespace: test module namespace, e.g. 'csb.test.cases.bio' will
482 load tests from '/csb/test/cases/bio/__init__.py'
483 @type namespace: str
484
485 @return: a C{unittest.TestSuite} ready for the test runner
486 @rtype: C{unittest.TestSuite}
487 """
488 if namespace.strip() == '.*':
489 namespace = '__main__.*'
490 elif namespace.strip() == '.':
491 namespace = '__main__'
492
493 if namespace.endswith('.*'):
494 return self.loadAllTests(namespace[:-2])
495 else:
496 loader = unittest.TestLoader()
497 tests = loader.loadTestsFromName(namespace)
498 return unittest.TestSuite(self._filter(tests))
499
500 def loadMultipleTests(self, namespaces):
501 """
502 Load L{csb.test.Case}s from a list of given CSB C{namespaces}.
503
504 @param namespaces: a list of test module namespaces, e.g.
505 ('csb.test.cases.bio', 'csb.test.cases.bio.io') will
506 load tests from '/csb/test/cases/bio.py' and
507 '/csb/test/cases/bio/io.py'
508 @type namespaces: tuple of str
509
510 @return: a C{unittest.TestSuite} ready for the test runner
511 @rtype: C{unittest.TestSuite}
512 """
513 if not csb.core.iterable(namespaces):
514 raise TypeError(namespaces)
515
516 return unittest.TestSuite(self.loadTests(n) for n in namespaces)
517
518 def loadAllTests(self, namespace, extension='.py'):
519 """
520 Load L{csb.test.Case}s recursively from the given CSB C{namespace} and
521 all of its sub-packages. Same as::
522
523 builder.loadTests('namespace.*')
524
525 @param namespace: test module namespace, e.g. 'csb.test.cases.bio' will
526 load tests from /csb/test/cases/bio/*'
527 @type namespace: str
528
529 @return: a C{unittest.TestSuite} ready for the test runner
530 @rtype: C{unittest.TestSuite}
531 """
532 suites = []
533
534 try:
535 base = __import__(namespace, level=0, fromlist=['']).__file__
536 except ImportError:
•537 raise NameError('Namespace {0} is not importable'.format(namespace))
538
539 if os.path.splitext(os.path.basename(base))[0] != '__init__':
540 suites.append(self.loadTests(namespace))
541
542 else:
543
544 for entry in os.walk(os.path.dirname(base)):
545
546 for item in entry[2]:
547 file = os.path.join(entry[0], item)
548 if extension and item.endswith(extension):
549 suites.append(self.loadFromFile(file))
550
551 return unittest.TestSuite(suites)
552
553 def _loadSource(self, path):
554 """
555 Import and return the Python module identified by C{path}.
556
557 @note: Module objects behave as singletons. If you import two different
558 modules and give them the same name in imp.load_source(mn), this
559 counts for a redefinition of the module originally named mn, which
560 is basically the same as reload(mn). Therefore, you need to ensure
561 that for every call to imp.load_source(mn, src.py) the mn parameter
562 is a string that uniquely identifies the source file src.py.
563 """
564 name = os.path.splitext(os.path.abspath(path))[0]
565 name = name.replace('.', '-').rstrip('__init__').strip(os.path.sep)
566
•567 return importlib.machinery.SourceFileLoader(name, path).load_module()
568
569 def _recurse(self, obj):
570 """
571 Extract test cases recursively from a test C{obj} container.
572 """
573 cases = []
574 if isinstance(obj, unittest.TestSuite) or csb.core.iterable(obj):
575 for item in obj:
576 cases.extend(self._recurse(item))
577 else:
578 cases.append(obj)
579 return cases
580
581 def _filter(self, tests):
582 """
583 Filter a list of objects using C{self.labels}.
584 """
585 filtered = []
586
587 for test in self._recurse(tests):
588 for label in self.labels:
589 if hasattr(test, label) and getattr(test, label) is True:
590 filtered.append(test)
591
592 return filtered
593
594 class AnyTestBuilder(AbstractTestBuilder):
595 """
596 Build a test suite of cases, marked as either unit, functional or regression
597 tests. For detailed documentation see L{AbstractTestBuilder}.
598 """
599 @property
600 def labels(self):
601 return [Attributes.UNIT, Attributes.FUNCTIONAL, Attributes.REGRESSION]
602
603 class UnitTestBuilder(AbstractTestBuilder):
604 """
605 Build a test suite of cases, marked as unit tests.
606 For detailed documentation see L{AbstractTestBuilder}.
607 """
608 @property
609 def labels(self):
610 return [Attributes.UNIT]
611
612 class FunctionalTestBuilder(AbstractTestBuilder):
613 """
614 Build a test suite of cases, marked as functional tests.
615 For detailed documentation see L{AbstractTestBuilder}.
616 """
617 @property
618 def labels(self):
619 return [Attributes.FUNCTIONAL]
620
621 class RegressionTestBuilder(AbstractTestBuilder):
622 """
623 Build a test suite of cases, marked as regression tests.
624 For detailed documentation see L{AbstractTestBuilder}.
625 """
626 @property
627 def labels(self):
628 return [Attributes.REGRESSION]
629
630 class CustomTestBuilder(AbstractTestBuilder):
631 """
632 Build a test suite of cases, marked as custom tests. CustomTestBuilder will
633 search for functions, marked with the 'custom' test decorator, which return
634 a dynamically built C{unittest.TestSuite} object when called without
635 parameters. This is convenient when doing data-related tests, e.g.
636 instantiating a single type of a test case many times iteratively, for
637 each entry in a database.
638
639 For detailed documentation see L{AbstractTestBuilder}.
640 """
641 @property
642 def labels(self):
643 return [Attributes.CUSTOM]
644
645 def loadFromFile(self, file):
646
647 mod = self._loadSource(file)
648 suites = self._inspect(mod)
649
650 return unittest.TestSuite(suites)
651
652 def loadTests(self, namespace):
653
654 if namespace.strip() == '.*':
655 namespace = '__main__.*'
656 elif namespace.strip() == '.':
657 namespace = '__main__'
658
659 if namespace.endswith('.*'):
660 return self.loadAllTests(namespace[:-2])
661 else:
662 try:
663 mod = __import__(namespace, fromlist=[''])
664 except ImportError:
•665 raise NameError('Namespace {0} is not importable'.format(namespace))
666 suites = self._inspect(mod)
667 return unittest.TestSuite(suites)
668
669 def _inspect(self, module):
670
671 objects = map(lambda n: getattr(module, n), dir(module))
672 return self._filter(objects)
673
674 def _filter(self, factories):
675 """
676 Filter a list of objects using C{self.labels}.
677 """
678 filtered = []
679
680 for obj in factories:
681 for label in self.labels:
682 if hasattr(obj, label) and getattr(obj, label) is True:
683 suite = obj()
684 if not isinstance(suite, unittest.TestSuite):
685 raise ValueError('Custom test function {0} must return a '
686 'unittest.TestSuite, not {1}'.format(obj.__name__, type(suite)))
687 filtered.append(suite)
688
689 return filtered
690
691 def unit(klass):
692 """
693 A class decorator, used to label unit test cases.
694
695 @param klass: a C{unittest.TestCase} class type
696 @type klass: type
697 """
698 if not isinstance(klass, type):
699 raise TypeError("Can't apply class decorator on {0}".format(type(klass)))
700
701 setattr(klass, Attributes.UNIT, True)
702 return klass
703
704 def functional(klass):
705 """
706 A class decorator, used to label functional test cases.
707
708 @param klass: a C{unittest.TestCase} class type
709 @type klass: type
710 """
711 if not isinstance(klass, type):
712 raise TypeError("Can't apply class decorator on {0}".format(type(klass)))
713
714 setattr(klass, Attributes.FUNCTIONAL, True)
715 return klass
716
717 def regression(klass):
718 """
719 A class decorator, used to label regression test cases.
720
721 @param klass: a C{unittest.TestCase} class type
722 @type klass: type
723 """
724 if not isinstance(klass, type):
725 raise TypeError("Can't apply class decorator on {0}".format(type(klass)))
726
727 setattr(klass, Attributes.REGRESSION, True)
728 return klass
729
730 def custom(function):
731 """
732 A function decorator, used to mark functions which build custom (dynamic)
733 test suites when called.
734
735 @param function: a callable object, which returns a dynamically compiled
736 C{unittest.TestSuite}
737 @type function: callable
738 """
739 if isinstance(function, type):
740 raise TypeError("Can't apply function decorator on a class")
741 elif not hasattr(function, '__call__'):
742 raise TypeError("Can't apply function decorator on non-callable {0}".format(type(function)))
743
744 setattr(function, Attributes.CUSTOM, True)
745 return function
746
747 def skip(reason, condition=None):
748 """
749 Mark a test case or method for skipping.
750
751 @param reason: message
752 @type reason: str
753 @param condition: skip only if the specified condition is True
754 @type condition: bool/expression
755 """
756 if isinstance(reason, types.FunctionType):
757 raise TypeError('skip: no reason specified')
758
759 if condition is None:
760 return unittest.skip(reason)
761 else:
762 return unittest.skipIf(condition, reason)
763
764 class Console(object):
765 """
766 Build and run all tests of the specified namespace and kind.
767
768 @param namespace: a dotted name, which specifies the test module
769 (see L{csb.test.AbstractTestBuilder.loadTests})
770 @type namespace: str
771 @param builder: test builder to use
772 @type builder: any L{csb.test.AbstractTestBuilder} subclass
773 @param verbosity: verbosity level for C{unittest.TestRunner}
774 @type verbosity: int
775 @param update: if True, refresh all pickles in csb/test/data
776 @type update: bool
777 @param generated_data: where to cache generated test files (directory)
778 @type generated_data: str
779 """
780
781 BUILDERS = {'unit': UnitTestBuilder, 'functional': FunctionalTestBuilder,
782 'custom': CustomTestBuilder, 'any': AnyTestBuilder,
783 'regression': RegressionTestBuilder}
784
785
786 def __init__(self, namespace=('__main__',), builder=AnyTestBuilder, verbosity=1,
787 update=False, generated_data=Config.GENERATED_DATA, argv=None):
788
789 if not argv:
790 argv = sys.argv
791
792 self._namespace = None
793 self._builder = None
794 self._verbosity = 1
795 self._update = False
796 self._gendata = str(generated_data)
797 self._program = os.path.basename(argv[0])
798
799 self.namespace = namespace
800 self.builder = builder
801 self.verbosity = verbosity
802 self.update = update
803 self.generated_data = generated_data
804
805 self.parseArguments(argv[1:])
806 self.run()
807
808 @property
809 def namespace(self):
810 return self._namespace
811 @namespace.setter
812 def namespace(self, value):
813 if csb.core.iterable(value):
814 self._namespace = list(value)
815 else:
816 self._namespace = [value]
817
818 @property
819 def builder(self):
820 return self._builder
821 @builder.setter
822 def builder(self, value):
823 self._builder = value
824
825 @property
826 def verbosity(self):
827 return self._verbosity
828 @verbosity.setter
829 def verbosity(self, value):
830 self._verbosity = value
831
832 @property
833 def builders(self):
834 return ', '.join(Console.BUILDERS)
835
836 @property
837 def program(self):
838 return self._program
839
840 @property
841 def update(self):
842 return self._update
843 @update.setter
844 def update(self, value):
845 self._update = bool(value)
846
847 @property
848 def generated_data(self):
849 return self._gendata
850 @generated_data.setter
851 def generated_data(self, value):
852 self._gendata = os.path.abspath(value)
853
854 def run(self):
855
856 Config.setDefaultGeneratedDataRoot(self.generated_data)
857
858 if self.update:
859 Config().updateDataFiles()
860 else:
861 Config().ensureDataConsistency()
862
863 builder = self.builder()
864 suite = builder.loadMultipleTests(self.namespace)
865
866 runner = unittest.TextTestRunner(verbosity=self.verbosity)
867 runner.run(suite)
868
869 def parseArguments(self, argv):
870
871 parser = argparse.ArgumentParser(prog=self.program, description="CSB Test Runner Console.")
872
873 parser.add_argument("-t", "--type", type=str, default="any", choices=list(Console.BUILDERS),
874 help="Type of tests to load from each namespace (default=any)")
875 parser.add_argument("-v", "--verbosity", type=int, default=1,
876 help="Verbosity level passed to unittest.TextTestRunner (default=1).")
877 parser.add_argument("-u", "--update-files", default=False, action="store_true",
878 help="Force update of the test pickles in " + Config.GENERATED_DATA)
879 parser.add_argument("-g", "--generated-resources", type=str, default=Config.GENERATED_DATA,
880 help="Generate, store and load additional test resources in this directory"
881 " (default=" + Config.GENERATED_DATA + ")")
882
883 parser.add_argument("namespaces", nargs='*',
884 help="""An optional list of CSB test dotted namespaces, from which to
885 load tests. '__main__' and '.' are interpreted as the
886 current module. If a namespace ends with an asterisk
887 '.*', all sub-packages will be scanned as well.
888
889 Examples:
890 "csb.test.cases.bio.*"
891 "csb.test.cases.bio.io" "csb.test.cases.bio.utils"
892 ".")""")
893
894 args = parser.parse_args(argv)
895
896 self.builder = Console.BUILDERS[args.type]
897 self.verbosity = args.verbosity
898 self.update = args.update_files
899 self.generated_data = args.generated_resources
900
901 if args.namespaces:
902 self.namespace = args.namespaces
903
904
905 if __name__ == '__main__':
906
907 Console()