/Doc/library/unittest.rst

http://unladen-swallow.googlecode.com/ · ReStructuredText · 947 lines · 633 code · 314 blank · 0 comment · 0 complexity · 13b70ac6767890876473a26ad1c5db91 MD5 · raw file

  1. :mod:`unittest` --- Unit testing framework
  2. ==========================================
  3. .. module:: unittest
  4. :synopsis: Unit testing framework for Python.
  5. .. moduleauthor:: Steve Purcell <stephen_purcell@yahoo.com>
  6. .. sectionauthor:: Steve Purcell <stephen_purcell@yahoo.com>
  7. .. sectionauthor:: Fred L. Drake, Jr. <fdrake@acm.org>
  8. .. sectionauthor:: Raymond Hettinger <python@rcn.com>
  9. .. versionadded:: 2.1
  10. The Python unit testing framework, sometimes referred to as "PyUnit," is a
  11. Python language version of JUnit, by Kent Beck and Erich Gamma. JUnit is, in
  12. turn, a Java version of Kent's Smalltalk testing framework. Each is the de
  13. facto standard unit testing framework for its respective language.
  14. :mod:`unittest` supports test automation, sharing of setup and shutdown code for
  15. tests, aggregation of tests into collections, and independence of the tests from
  16. the reporting framework. The :mod:`unittest` module provides classes that make
  17. it easy to support these qualities for a set of tests.
  18. To achieve this, :mod:`unittest` supports some important concepts:
  19. test fixture
  20. A :dfn:`test fixture` represents the preparation needed to perform one or more
  21. tests, and any associate cleanup actions. This may involve, for example,
  22. creating temporary or proxy databases, directories, or starting a server
  23. process.
  24. test case
  25. A :dfn:`test case` is the smallest unit of testing. It checks for a specific
  26. response to a particular set of inputs. :mod:`unittest` provides a base class,
  27. :class:`TestCase`, which may be used to create new test cases.
  28. test suite
  29. A :dfn:`test suite` is a collection of test cases, test suites, or both. It is
  30. used to aggregate tests that should be executed together.
  31. test runner
  32. A :dfn:`test runner` is a component which orchestrates the execution of tests
  33. and provides the outcome to the user. The runner may use a graphical interface,
  34. a textual interface, or return a special value to indicate the results of
  35. executing the tests.
  36. The test case and test fixture concepts are supported through the
  37. :class:`TestCase` and :class:`FunctionTestCase` classes; the former should be
  38. used when creating new tests, and the latter can be used when integrating
  39. existing test code with a :mod:`unittest`\ -driven framework. When building test
  40. fixtures using :class:`TestCase`, the :meth:`setUp` and :meth:`tearDown` methods
  41. can be overridden to provide initialization and cleanup for the fixture. With
  42. :class:`FunctionTestCase`, existing functions can be passed to the constructor
  43. for these purposes. When the test is run, the fixture initialization is run
  44. first; if it succeeds, the cleanup method is run after the test has been
  45. executed, regardless of the outcome of the test. Each instance of the
  46. :class:`TestCase` will only be used to run a single test method, so a new
  47. fixture is created for each test.
  48. Test suites are implemented by the :class:`TestSuite` class. This class allows
  49. individual tests and test suites to be aggregated; when the suite is executed,
  50. all tests added directly to the suite and in "child" test suites are run.
  51. A test runner is an object that provides a single method, :meth:`run`, which
  52. accepts a :class:`TestCase` or :class:`TestSuite` object as a parameter, and
  53. returns a result object. The class :class:`TestResult` is provided for use as
  54. the result object. :mod:`unittest` provides the :class:`TextTestRunner` as an
  55. example test runner which reports test results on the standard error stream by
  56. default. Alternate runners can be implemented for other environments (such as
  57. graphical environments) without any need to derive from a specific class.
  58. .. seealso::
  59. Module :mod:`doctest`
  60. Another test-support module with a very different flavor.
  61. `Simple Smalltalk Testing: With Patterns <http://www.XProgramming.com/testfram.htm>`_
  62. Kent Beck's original paper on testing frameworks using the pattern shared by
  63. :mod:`unittest`.
  64. `Nose <http://code.google.com/p/python-nose/>`_ and `py.test <http://pytest.org>`_
  65. Third-party unittest frameworks with a lighter-weight syntax
  66. for writing tests. For example, ``assert func(10) == 42``.
  67. `python-mock <http://python-mock.sourceforge.net/>`_ and `minimock <http://blog.ianbicking.org/minimock.html>`_
  68. Tools for creating mock test objects (objects simulating external resources).
  69. .. _unittest-minimal-example:
  70. Basic example
  71. -------------
  72. The :mod:`unittest` module provides a rich set of tools for constructing and
  73. running tests. This section demonstrates that a small subset of the tools
  74. suffice to meet the needs of most users.
  75. Here is a short script to test three functions from the :mod:`random` module::
  76. import random
  77. import unittest
  78. class TestSequenceFunctions(unittest.TestCase):
  79. def setUp(self):
  80. self.seq = range(10)
  81. def testshuffle(self):
  82. # make sure the shuffled sequence does not lose any elements
  83. random.shuffle(self.seq)
  84. self.seq.sort()
  85. self.assertEqual(self.seq, range(10))
  86. def testchoice(self):
  87. element = random.choice(self.seq)
  88. self.assert_(element in self.seq)
  89. def testsample(self):
  90. self.assertRaises(ValueError, random.sample, self.seq, 20)
  91. for element in random.sample(self.seq, 5):
  92. self.assert_(element in self.seq)
  93. if __name__ == '__main__':
  94. unittest.main()
  95. A testcase is created by subclassing :class:`unittest.TestCase`. The three
  96. individual tests are defined with methods whose names start with the letters
  97. ``test``. This naming convention informs the test runner about which methods
  98. represent tests.
  99. The crux of each test is a call to :meth:`assertEqual` to check for an expected
  100. result; :meth:`assert_` to verify a condition; or :meth:`assertRaises` to verify
  101. that an expected exception gets raised. These methods are used instead of the
  102. :keyword:`assert` statement so the test runner can accumulate all test results
  103. and produce a report.
  104. When a :meth:`setUp` method is defined, the test runner will run that method
  105. prior to each test. Likewise, if a :meth:`tearDown` method is defined, the test
  106. runner will invoke that method after each test. In the example, :meth:`setUp`
  107. was used to create a fresh sequence for each test.
  108. The final block shows a simple way to run the tests. :func:`unittest.main`
  109. provides a command line interface to the test script. When run from the command
  110. line, the above script produces an output that looks like this::
  111. ...
  112. ----------------------------------------------------------------------
  113. Ran 3 tests in 0.000s
  114. OK
  115. Instead of :func:`unittest.main`, there are other ways to run the tests with a
  116. finer level of control, less terse output, and no requirement to be run from the
  117. command line. For example, the last two lines may be replaced with::
  118. suite = unittest.TestLoader().loadTestsFromTestCase(TestSequenceFunctions)
  119. unittest.TextTestRunner(verbosity=2).run(suite)
  120. Running the revised script from the interpreter or another script produces the
  121. following output::
  122. testchoice (__main__.TestSequenceFunctions) ... ok
  123. testsample (__main__.TestSequenceFunctions) ... ok
  124. testshuffle (__main__.TestSequenceFunctions) ... ok
  125. ----------------------------------------------------------------------
  126. Ran 3 tests in 0.110s
  127. OK
  128. The above examples show the most commonly used :mod:`unittest` features which
  129. are sufficient to meet many everyday testing needs. The remainder of the
  130. documentation explores the full feature set from first principles.
  131. .. _organizing-tests:
  132. Organizing test code
  133. --------------------
  134. The basic building blocks of unit testing are :dfn:`test cases` --- single
  135. scenarios that must be set up and checked for correctness. In :mod:`unittest`,
  136. test cases are represented by instances of :mod:`unittest`'s :class:`TestCase`
  137. class. To make your own test cases you must write subclasses of
  138. :class:`TestCase`, or use :class:`FunctionTestCase`.
  139. An instance of a :class:`TestCase`\ -derived class is an object that can
  140. completely run a single test method, together with optional set-up and tidy-up
  141. code.
  142. The testing code of a :class:`TestCase` instance should be entirely self
  143. contained, such that it can be run either in isolation or in arbitrary
  144. combination with any number of other test cases.
  145. The simplest :class:`TestCase` subclass will simply override the :meth:`runTest`
  146. method in order to perform specific testing code::
  147. import unittest
  148. class DefaultWidgetSizeTestCase(unittest.TestCase):
  149. def runTest(self):
  150. widget = Widget('The widget')
  151. self.assertEqual(widget.size(), (50, 50), 'incorrect default size')
  152. Note that in order to test something, we use the one of the :meth:`assert\*` or
  153. :meth:`fail\*` methods provided by the :class:`TestCase` base class. If the
  154. test fails, an exception will be raised, and :mod:`unittest` will identify the
  155. test case as a :dfn:`failure`. Any other exceptions will be treated as
  156. :dfn:`errors`. This helps you identify where the problem is: :dfn:`failures` are
  157. caused by incorrect results - a 5 where you expected a 6. :dfn:`Errors` are
  158. caused by incorrect code - e.g., a :exc:`TypeError` caused by an incorrect
  159. function call.
  160. The way to run a test case will be described later. For now, note that to
  161. construct an instance of such a test case, we call its constructor without
  162. arguments::
  163. testCase = DefaultWidgetSizeTestCase()
  164. Now, such test cases can be numerous, and their set-up can be repetitive. In
  165. the above case, constructing a :class:`Widget` in each of 100 Widget test case
  166. subclasses would mean unsightly duplication.
  167. Luckily, we can factor out such set-up code by implementing a method called
  168. :meth:`setUp`, which the testing framework will automatically call for us when
  169. we run the test::
  170. import unittest
  171. class SimpleWidgetTestCase(unittest.TestCase):
  172. def setUp(self):
  173. self.widget = Widget('The widget')
  174. class DefaultWidgetSizeTestCase(SimpleWidgetTestCase):
  175. def runTest(self):
  176. self.failUnless(self.widget.size() == (50,50),
  177. 'incorrect default size')
  178. class WidgetResizeTestCase(SimpleWidgetTestCase):
  179. def runTest(self):
  180. self.widget.resize(100,150)
  181. self.failUnless(self.widget.size() == (100,150),
  182. 'wrong size after resize')
  183. If the :meth:`setUp` method raises an exception while the test is running, the
  184. framework will consider the test to have suffered an error, and the
  185. :meth:`runTest` method will not be executed.
  186. Similarly, we can provide a :meth:`tearDown` method that tidies up after the
  187. :meth:`runTest` method has been run::
  188. import unittest
  189. class SimpleWidgetTestCase(unittest.TestCase):
  190. def setUp(self):
  191. self.widget = Widget('The widget')
  192. def tearDown(self):
  193. self.widget.dispose()
  194. self.widget = None
  195. If :meth:`setUp` succeeded, the :meth:`tearDown` method will be run whether
  196. :meth:`runTest` succeeded or not.
  197. Such a working environment for the testing code is called a :dfn:`fixture`.
  198. Often, many small test cases will use the same fixture. In this case, we would
  199. end up subclassing :class:`SimpleWidgetTestCase` into many small one-method
  200. classes such as :class:`DefaultWidgetSizeTestCase`. This is time-consuming and
  201. discouraging, so in the same vein as JUnit, :mod:`unittest` provides a simpler
  202. mechanism::
  203. import unittest
  204. class WidgetTestCase(unittest.TestCase):
  205. def setUp(self):
  206. self.widget = Widget('The widget')
  207. def tearDown(self):
  208. self.widget.dispose()
  209. self.widget = None
  210. def testDefaultSize(self):
  211. self.failUnless(self.widget.size() == (50,50),
  212. 'incorrect default size')
  213. def testResize(self):
  214. self.widget.resize(100,150)
  215. self.failUnless(self.widget.size() == (100,150),
  216. 'wrong size after resize')
  217. Here we have not provided a :meth:`runTest` method, but have instead provided
  218. two different test methods. Class instances will now each run one of the
  219. :meth:`test\*` methods, with ``self.widget`` created and destroyed separately
  220. for each instance. When creating an instance we must specify the test method it
  221. is to run. We do this by passing the method name in the constructor::
  222. defaultSizeTestCase = WidgetTestCase('testDefaultSize')
  223. resizeTestCase = WidgetTestCase('testResize')
  224. Test case instances are grouped together according to the features they test.
  225. :mod:`unittest` provides a mechanism for this: the :dfn:`test suite`,
  226. represented by :mod:`unittest`'s :class:`TestSuite` class::
  227. widgetTestSuite = unittest.TestSuite()
  228. widgetTestSuite.addTest(WidgetTestCase('testDefaultSize'))
  229. widgetTestSuite.addTest(WidgetTestCase('testResize'))
  230. For the ease of running tests, as we will see later, it is a good idea to
  231. provide in each test module a callable object that returns a pre-built test
  232. suite::
  233. def suite():
  234. suite = unittest.TestSuite()
  235. suite.addTest(WidgetTestCase('testDefaultSize'))
  236. suite.addTest(WidgetTestCase('testResize'))
  237. return suite
  238. or even::
  239. def suite():
  240. tests = ['testDefaultSize', 'testResize']
  241. return unittest.TestSuite(map(WidgetTestCase, tests))
  242. Since it is a common pattern to create a :class:`TestCase` subclass with many
  243. similarly named test functions, :mod:`unittest` provides a :class:`TestLoader`
  244. class that can be used to automate the process of creating a test suite and
  245. populating it with individual tests. For example, ::
  246. suite = unittest.TestLoader().loadTestsFromTestCase(WidgetTestCase)
  247. will create a test suite that will run ``WidgetTestCase.testDefaultSize()`` and
  248. ``WidgetTestCase.testResize``. :class:`TestLoader` uses the ``'test'`` method
  249. name prefix to identify test methods automatically.
  250. Note that the order in which the various test cases will be run is determined by
  251. sorting the test function names with the built-in :func:`cmp` function.
  252. Often it is desirable to group suites of test cases together, so as to run tests
  253. for the whole system at once. This is easy, since :class:`TestSuite` instances
  254. can be added to a :class:`TestSuite` just as :class:`TestCase` instances can be
  255. added to a :class:`TestSuite`::
  256. suite1 = module1.TheTestSuite()
  257. suite2 = module2.TheTestSuite()
  258. alltests = unittest.TestSuite([suite1, suite2])
  259. You can place the definitions of test cases and test suites in the same modules
  260. as the code they are to test (such as :file:`widget.py`), but there are several
  261. advantages to placing the test code in a separate module, such as
  262. :file:`test_widget.py`:
  263. * The test module can be run standalone from the command line.
  264. * The test code can more easily be separated from shipped code.
  265. * There is less temptation to change test code to fit the code it tests without
  266. a good reason.
  267. * Test code should be modified much less frequently than the code it tests.
  268. * Tested code can be refactored more easily.
  269. * Tests for modules written in C must be in separate modules anyway, so why not
  270. be consistent?
  271. * If the testing strategy changes, there is no need to change the source code.
  272. .. _legacy-unit-tests:
  273. Re-using old test code
  274. ----------------------
  275. Some users will find that they have existing test code that they would like to
  276. run from :mod:`unittest`, without converting every old test function to a
  277. :class:`TestCase` subclass.
  278. For this reason, :mod:`unittest` provides a :class:`FunctionTestCase` class.
  279. This subclass of :class:`TestCase` can be used to wrap an existing test
  280. function. Set-up and tear-down functions can also be provided.
  281. Given the following test function::
  282. def testSomething():
  283. something = makeSomething()
  284. assert something.name is not None
  285. # ...
  286. one can create an equivalent test case instance as follows::
  287. testcase = unittest.FunctionTestCase(testSomething)
  288. If there are additional set-up and tear-down methods that should be called as
  289. part of the test case's operation, they can also be provided like so::
  290. testcase = unittest.FunctionTestCase(testSomething,
  291. setUp=makeSomethingDB,
  292. tearDown=deleteSomethingDB)
  293. To make migrating existing test suites easier, :mod:`unittest` supports tests
  294. raising :exc:`AssertionError` to indicate test failure. However, it is
  295. recommended that you use the explicit :meth:`TestCase.fail\*` and
  296. :meth:`TestCase.assert\*` methods instead, as future versions of :mod:`unittest`
  297. may treat :exc:`AssertionError` differently.
  298. .. note::
  299. Even though :class:`FunctionTestCase` can be used to quickly convert an existing
  300. test base over to a :mod:`unittest`\ -based system, this approach is not
  301. recommended. Taking the time to set up proper :class:`TestCase` subclasses will
  302. make future test refactorings infinitely easier.
  303. .. _unittest-contents:
  304. Classes and functions
  305. ---------------------
  306. .. class:: TestCase([methodName])
  307. Instances of the :class:`TestCase` class represent the smallest testable units
  308. in the :mod:`unittest` universe. This class is intended to be used as a base
  309. class, with specific tests being implemented by concrete subclasses. This class
  310. implements the interface needed by the test runner to allow it to drive the
  311. test, and methods that the test code can use to check for and report various
  312. kinds of failure.
  313. Each instance of :class:`TestCase` will run a single test method: the method
  314. named *methodName*. If you remember, we had an earlier example that went
  315. something like this::
  316. def suite():
  317. suite = unittest.TestSuite()
  318. suite.addTest(WidgetTestCase('testDefaultSize'))
  319. suite.addTest(WidgetTestCase('testResize'))
  320. return suite
  321. Here, we create two instances of :class:`WidgetTestCase`, each of which runs a
  322. single test.
  323. *methodName* defaults to ``'runTest'``.
  324. .. class:: FunctionTestCase(testFunc[, setUp[, tearDown[, description]]])
  325. This class implements the portion of the :class:`TestCase` interface which
  326. allows the test runner to drive the test, but does not provide the methods which
  327. test code can use to check and report errors. This is used to create test cases
  328. using legacy test code, allowing it to be integrated into a :mod:`unittest`\
  329. -based test framework.
  330. .. class:: TestSuite([tests])
  331. This class represents an aggregation of individual tests cases and test suites.
  332. The class presents the interface needed by the test runner to allow it to be run
  333. as any other test case. Running a :class:`TestSuite` instance is the same as
  334. iterating over the suite, running each test individually.
  335. If *tests* is given, it must be an iterable of individual test cases or other
  336. test suites that will be used to build the suite initially. Additional methods
  337. are provided to add test cases and suites to the collection later on.
  338. .. class:: TestLoader()
  339. This class is responsible for loading tests according to various criteria and
  340. returning them wrapped in a :class:`TestSuite`. It can load all tests within a
  341. given module or :class:`TestCase` subclass.
  342. .. class:: TestResult()
  343. This class is used to compile information about which tests have succeeded and
  344. which have failed.
  345. .. data:: defaultTestLoader
  346. Instance of the :class:`TestLoader` class intended to be shared. If no
  347. customization of the :class:`TestLoader` is needed, this instance can be used
  348. instead of repeatedly creating new instances.
  349. .. class:: TextTestRunner([stream[, descriptions[, verbosity]]])
  350. A basic test runner implementation which prints results on standard error. It
  351. has a few configurable parameters, but is essentially very simple. Graphical
  352. applications which run test suites should provide alternate implementations.
  353. .. function:: main([module[, defaultTest[, argv[, testRunner[, testLoader]]]]])
  354. A command-line program that runs a set of tests; this is primarily for making
  355. test modules conveniently executable. The simplest use for this function is to
  356. include the following line at the end of a test script::
  357. if __name__ == '__main__':
  358. unittest.main()
  359. The *testRunner* argument can either be a test runner class or an already
  360. created instance of it.
  361. In some cases, the existing tests may have been written using the :mod:`doctest`
  362. module. If so, that module provides a :class:`DocTestSuite` class that can
  363. automatically build :class:`unittest.TestSuite` instances from the existing
  364. :mod:`doctest`\ -based tests.
  365. .. versionadded:: 2.3
  366. .. _testcase-objects:
  367. TestCase Objects
  368. ----------------
  369. Each :class:`TestCase` instance represents a single test, but each concrete
  370. subclass may be used to define multiple tests --- the concrete class represents
  371. a single test fixture. The fixture is created and cleaned up for each test
  372. case.
  373. :class:`TestCase` instances provide three groups of methods: one group used to
  374. run the test, another used by the test implementation to check conditions and
  375. report failures, and some inquiry methods allowing information about the test
  376. itself to be gathered.
  377. Methods in the first group (running the test) are:
  378. .. method:: TestCase.setUp()
  379. Method called to prepare the test fixture. This is called immediately before
  380. calling the test method; any exception raised by this method will be considered
  381. an error rather than a test failure. The default implementation does nothing.
  382. .. method:: TestCase.tearDown()
  383. Method called immediately after the test method has been called and the result
  384. recorded. This is called even if the test method raised an exception, so the
  385. implementation in subclasses may need to be particularly careful about checking
  386. internal state. Any exception raised by this method will be considered an error
  387. rather than a test failure. This method will only be called if the
  388. :meth:`setUp` succeeds, regardless of the outcome of the test method. The
  389. default implementation does nothing.
  390. .. method:: TestCase.run([result])
  391. Run the test, collecting the result into the test result object passed as
  392. *result*. If *result* is omitted or :const:`None`, a temporary result object is
  393. created (by calling the :meth:`defaultTestCase` method) and used; this result
  394. object is not returned to :meth:`run`'s caller.
  395. The same effect may be had by simply calling the :class:`TestCase` instance.
  396. .. method:: TestCase.debug()
  397. Run the test without collecting the result. This allows exceptions raised by
  398. the test to be propagated to the caller, and can be used to support running
  399. tests under a debugger.
  400. The test code can use any of the following methods to check for and report
  401. failures.
  402. .. method:: TestCase.assert_(expr[, msg])
  403. TestCase.failUnless(expr[, msg])
  404. TestCase.assertTrue(expr[, msg])
  405. Signal a test failure if *expr* is false; the explanation for the error will be
  406. *msg* if given, otherwise it will be :const:`None`.
  407. .. method:: TestCase.assertEqual(first, second[, msg])
  408. TestCase.failUnlessEqual(first, second[, msg])
  409. Test that *first* and *second* are equal. If the values do not compare equal,
  410. the test will fail with the explanation given by *msg*, or :const:`None`. Note
  411. that using :meth:`failUnlessEqual` improves upon doing the comparison as the
  412. first parameter to :meth:`failUnless`: the default value for *msg* can be
  413. computed to include representations of both *first* and *second*.
  414. .. method:: TestCase.assertNotEqual(first, second[, msg])
  415. TestCase.failIfEqual(first, second[, msg])
  416. Test that *first* and *second* are not equal. If the values do compare equal,
  417. the test will fail with the explanation given by *msg*, or :const:`None`. Note
  418. that using :meth:`failIfEqual` improves upon doing the comparison as the first
  419. parameter to :meth:`failUnless` is that the default value for *msg* can be
  420. computed to include representations of both *first* and *second*.
  421. .. method:: TestCase.assertAlmostEqual(first, second[, places[, msg]])
  422. TestCase.failUnlessAlmostEqual(first, second[, places[, msg]])
  423. Test that *first* and *second* are approximately equal by computing the
  424. difference, rounding to the given number of decimal *places* (default 7),
  425. and comparing to zero.
  426. Note that comparing a given number of decimal places is not the same as
  427. comparing a given number of significant digits. If the values do not compare
  428. equal, the test will fail with the explanation given by *msg*, or :const:`None`.
  429. .. method:: TestCase.assertNotAlmostEqual(first, second[, places[, msg]])
  430. TestCase.failIfAlmostEqual(first, second[, places[, msg]])
  431. Test that *first* and *second* are not approximately equal by computing the
  432. difference, rounding to the given number of decimal *places* (default 7),
  433. and comparing to zero.
  434. Note that comparing a given number of decimal places is not the same as
  435. comparing a given number of significant digits. If the values do not compare
  436. equal, the test will fail with the explanation given by *msg*, or :const:`None`.
  437. .. method:: TestCase.assertRaises(exception, callable, ...)
  438. TestCase.failUnlessRaises(exception, callable, ...)
  439. Test that an exception is raised when *callable* is called with any positional
  440. or keyword arguments that are also passed to :meth:`assertRaises`. The test
  441. passes if *exception* is raised, is an error if another exception is raised, or
  442. fails if no exception is raised. To catch any of a group of exceptions, a tuple
  443. containing the exception classes may be passed as *exception*.
  444. .. method:: TestCase.failIf(expr[, msg])
  445. TestCase.assertFalse(expr[, msg])
  446. The inverse of the :meth:`failUnless` method is the :meth:`failIf` method. This
  447. signals a test failure if *expr* is true, with *msg* or :const:`None` for the
  448. error message.
  449. .. method:: TestCase.fail([msg])
  450. Signals a test failure unconditionally, with *msg* or :const:`None` for the
  451. error message.
  452. .. attribute:: TestCase.failureException
  453. This class attribute gives the exception raised by the :meth:`test` method. If
  454. a test framework needs to use a specialized exception, possibly to carry
  455. additional information, it must subclass this exception in order to "play fair"
  456. with the framework. The initial value of this attribute is
  457. :exc:`AssertionError`.
  458. Testing frameworks can use the following methods to collect information on the
  459. test:
  460. .. method:: TestCase.countTestCases()
  461. Return the number of tests represented by this test object. For
  462. :class:`TestCase` instances, this will always be ``1``.
  463. .. method:: TestCase.defaultTestResult()
  464. Return an instance of the test result class that should be used for this test
  465. case class (if no other result instance is provided to the :meth:`run` method).
  466. For :class:`TestCase` instances, this will always be an instance of
  467. :class:`TestResult`; subclasses of :class:`TestCase` should override this as
  468. necessary.
  469. .. method:: TestCase.id()
  470. Return a string identifying the specific test case. This is usually the full
  471. name of the test method, including the module and class name.
  472. .. method:: TestCase.shortDescription()
  473. Returns a one-line description of the test, or :const:`None` if no description
  474. has been provided. The default implementation of this method returns the first
  475. line of the test method's docstring, if available, or :const:`None`.
  476. .. _testsuite-objects:
  477. TestSuite Objects
  478. -----------------
  479. :class:`TestSuite` objects behave much like :class:`TestCase` objects, except
  480. they do not actually implement a test. Instead, they are used to aggregate
  481. tests into groups of tests that should be run together. Some additional methods
  482. are available to add tests to :class:`TestSuite` instances:
  483. .. method:: TestSuite.addTest(test)
  484. Add a :class:`TestCase` or :class:`TestSuite` to the suite.
  485. .. method:: TestSuite.addTests(tests)
  486. Add all the tests from an iterable of :class:`TestCase` and :class:`TestSuite`
  487. instances to this test suite.
  488. This is equivalent to iterating over *tests*, calling :meth:`addTest` for each
  489. element.
  490. :class:`TestSuite` shares the following methods with :class:`TestCase`:
  491. .. method:: TestSuite.run(result)
  492. Run the tests associated with this suite, collecting the result into the test
  493. result object passed as *result*. Note that unlike :meth:`TestCase.run`,
  494. :meth:`TestSuite.run` requires the result object to be passed in.
  495. .. method:: TestSuite.debug()
  496. Run the tests associated with this suite without collecting the result. This
  497. allows exceptions raised by the test to be propagated to the caller and can be
  498. used to support running tests under a debugger.
  499. .. method:: TestSuite.countTestCases()
  500. Return the number of tests represented by this test object, including all
  501. individual tests and sub-suites.
  502. In the typical usage of a :class:`TestSuite` object, the :meth:`run` method is
  503. invoked by a :class:`TestRunner` rather than by the end-user test harness.
  504. .. _testresult-objects:
  505. TestResult Objects
  506. ------------------
  507. A :class:`TestResult` object stores the results of a set of tests. The
  508. :class:`TestCase` and :class:`TestSuite` classes ensure that results are
  509. properly recorded; test authors do not need to worry about recording the outcome
  510. of tests.
  511. Testing frameworks built on top of :mod:`unittest` may want access to the
  512. :class:`TestResult` object generated by running a set of tests for reporting
  513. purposes; a :class:`TestResult` instance is returned by the
  514. :meth:`TestRunner.run` method for this purpose.
  515. :class:`TestResult` instances have the following attributes that will be of
  516. interest when inspecting the results of running a set of tests:
  517. .. attribute:: TestResult.errors
  518. A list containing 2-tuples of :class:`TestCase` instances and strings holding
  519. formatted tracebacks. Each tuple represents a test which raised an unexpected
  520. exception.
  521. .. versionchanged:: 2.2
  522. Contains formatted tracebacks instead of :func:`sys.exc_info` results.
  523. .. attribute:: TestResult.failures
  524. A list containing 2-tuples of :class:`TestCase` instances and strings holding
  525. formatted tracebacks. Each tuple represents a test where a failure was
  526. explicitly signalled using the :meth:`TestCase.fail\*` or
  527. :meth:`TestCase.assert\*` methods.
  528. .. versionchanged:: 2.2
  529. Contains formatted tracebacks instead of :func:`sys.exc_info` results.
  530. .. attribute:: TestResult.testsRun
  531. The total number of tests run so far.
  532. .. method:: TestResult.wasSuccessful()
  533. Returns :const:`True` if all tests run so far have passed, otherwise returns
  534. :const:`False`.
  535. .. method:: TestResult.stop()
  536. This method can be called to signal that the set of tests being run should be
  537. aborted by setting the :class:`TestResult`'s ``shouldStop`` attribute to
  538. :const:`True`. :class:`TestRunner` objects should respect this flag and return
  539. without running any additional tests.
  540. For example, this feature is used by the :class:`TextTestRunner` class to stop
  541. the test framework when the user signals an interrupt from the keyboard.
  542. Interactive tools which provide :class:`TestRunner` implementations can use this
  543. in a similar manner.
  544. The following methods of the :class:`TestResult` class are used to maintain the
  545. internal data structures, and may be extended in subclasses to support
  546. additional reporting requirements. This is particularly useful in building
  547. tools which support interactive reporting while tests are being run.
  548. .. method:: TestResult.startTest(test)
  549. Called when the test case *test* is about to be run.
  550. The default implementation simply increments the instance's ``testsRun``
  551. counter.
  552. .. method:: TestResult.stopTest(test)
  553. Called after the test case *test* has been executed, regardless of the outcome.
  554. The default implementation does nothing.
  555. .. method:: TestResult.addError(test, err)
  556. Called when the test case *test* raises an unexpected exception *err* is a tuple
  557. of the form returned by :func:`sys.exc_info`: ``(type, value, traceback)``.
  558. The default implementation appends a tuple ``(test, formatted_err)`` to the
  559. instance's ``errors`` attribute, where *formatted_err* is a formatted
  560. traceback derived from *err*.
  561. .. method:: TestResult.addFailure(test, err)
  562. Called when the test case *test* signals a failure. *err* is a tuple of the form
  563. returned by :func:`sys.exc_info`: ``(type, value, traceback)``.
  564. The default implementation appends a tuple ``(test, formatted_err)`` to the
  565. instance's ``failures`` attribute, where *formatted_err* is a formatted
  566. traceback derived from *err*.
  567. .. method:: TestResult.addSuccess(test)
  568. Called when the test case *test* succeeds.
  569. The default implementation does nothing.
  570. .. _testloader-objects:
  571. TestLoader Objects
  572. ------------------
  573. The :class:`TestLoader` class is used to create test suites from classes and
  574. modules. Normally, there is no need to create an instance of this class; the
  575. :mod:`unittest` module provides an instance that can be shared as
  576. ``unittest.defaultTestLoader``. Using a subclass or instance, however, allows
  577. customization of some configurable properties.
  578. :class:`TestLoader` objects have the following methods:
  579. .. method:: TestLoader.loadTestsFromTestCase(testCaseClass)
  580. Return a suite of all tests cases contained in the :class:`TestCase`\ -derived
  581. :class:`testCaseClass`.
  582. .. method:: TestLoader.loadTestsFromModule(module)
  583. Return a suite of all tests cases contained in the given module. This method
  584. searches *module* for classes derived from :class:`TestCase` and creates an
  585. instance of the class for each test method defined for the class.
  586. .. warning::
  587. While using a hierarchy of :class:`TestCase`\ -derived classes can be convenient
  588. in sharing fixtures and helper functions, defining test methods on base classes
  589. that are not intended to be instantiated directly does not play well with this
  590. method. Doing so, however, can be useful when the fixtures are different and
  591. defined in subclasses.
  592. .. method:: TestLoader.loadTestsFromName(name[, module])
  593. Return a suite of all tests cases given a string specifier.
  594. The specifier *name* is a "dotted name" that may resolve either to a module, a
  595. test case class, a test method within a test case class, a :class:`TestSuite`
  596. instance, or a callable object which returns a :class:`TestCase` or
  597. :class:`TestSuite` instance. These checks are applied in the order listed here;
  598. that is, a method on a possible test case class will be picked up as "a test
  599. method within a test case class", rather than "a callable object".
  600. For example, if you have a module :mod:`SampleTests` containing a
  601. :class:`TestCase`\ -derived class :class:`SampleTestCase` with three test
  602. methods (:meth:`test_one`, :meth:`test_two`, and :meth:`test_three`), the
  603. specifier ``'SampleTests.SampleTestCase'`` would cause this method to return a
  604. suite which will run all three test methods. Using the specifier
  605. ``'SampleTests.SampleTestCase.test_two'`` would cause it to return a test suite
  606. which will run only the :meth:`test_two` test method. The specifier can refer
  607. to modules and packages which have not been imported; they will be imported as a
  608. side-effect.
  609. The method optionally resolves *name* relative to the given *module*.
  610. .. method:: TestLoader.loadTestsFromNames(names[, module])
  611. Similar to :meth:`loadTestsFromName`, but takes a sequence of names rather than
  612. a single name. The return value is a test suite which supports all the tests
  613. defined for each name.
  614. .. method:: TestLoader.getTestCaseNames(testCaseClass)
  615. Return a sorted sequence of method names found within *testCaseClass*; this
  616. should be a subclass of :class:`TestCase`.
  617. The following attributes of a :class:`TestLoader` can be configured either by
  618. subclassing or assignment on an instance:
  619. .. attribute:: TestLoader.testMethodPrefix
  620. String giving the prefix of method names which will be interpreted as test
  621. methods. The default value is ``'test'``.
  622. This affects :meth:`getTestCaseNames` and all the :meth:`loadTestsFrom\*`
  623. methods.
  624. .. attribute:: TestLoader.sortTestMethodsUsing
  625. Function to be used to compare method names when sorting them in
  626. :meth:`getTestCaseNames` and all the :meth:`loadTestsFrom\*` methods. The
  627. default value is the built-in :func:`cmp` function; the attribute can also be
  628. set to :const:`None` to disable the sort.
  629. .. attribute:: TestLoader.suiteClass
  630. Callable object that constructs a test suite from a list of tests. No methods on
  631. the resulting object are needed. The default value is the :class:`TestSuite`
  632. class.
  633. This affects all the :meth:`loadTestsFrom\*` methods.