Writing New Tests

When writing new tests, it very often helps to see how a similar test is implemented. For example, the files in the test_parsers/test_rst/ directory all look very similar. So when adding a test, you don't have to reinvent the wheel.

If there is no similar test, you can write a new test from scratch using Python's unittest module. For an example, please have a look at the following imaginary test_square.py:

#! /usr/bin/env python

# $Id: testing.txt 7463 2012-06-22 19:49:51Z milde $
# Author: Your Name <your_email_address@example.org>
# Copyright: This module has been placed in the public domain.

"""
Test module for docutils.square.
"""

import unittest
import docutils.square


class SquareTest(unittest.TestCase):

    def test_square(self):
        self.assertEqual(docutils.square.square(0), 0)
        self.assertEqual(docutils.square.square(5), 25)
        self.assertEqual(docutils.square.square(7), 49)

    def test_square_root(self):
        self.assertEqual(docutils.square.sqrt(49), 7)
        self.assertEqual(docutils.square.sqrt(0), 0)
        self.assertRaises(docutils.square.SquareRootError,
                          docutils.square.sqrt, 20)


if __name__ == '__main__':
    unittest.main()

For more details on how to write tests, please refer to the documentation of the unittest module.

Directory Structure

• functional/ The main data directory.
  • input/ The input files.
    • some_test.txt, for example.
  • output/ The actual output.
    • some_test.html, for example.
  • expected/ The expected output.
    • some_test.html, for example.
  • tests/ The config files for processing the input files.
    • some_test.py, for example.
    • _default.py, the default configuration file.

The Testing Process

When running test_functional.py, all config files in functional/tests/ are processed. (Config files whose names begin with an underscore are ignored.) The current working directory is always Docutils' main test directory (test/).

For example, functional/tests/some_test.py could read like this:

# Source and destination file names.
test_source = "some_test.txt"
test_destination = "some_test.html"

# Keyword parameters passed to publish_file.
reader_name = "standalone"
parser_name = "rst"
writer_name = "html"
settings_overrides['output-encoding'] = 'utf-8'
# Relative to main ``test/`` directory.
settings_overrides['stylesheet_path'] = '../docutils/writers/html4css1/html4css1.css'

The two variables test_source and test_destination contain the input file name (relative to functional/input/) and the output file name (relative to functional/output/ and functional/expected/). Note that the file names can be chosen arbitrarily. However, the file names in functional/output/ must match the file names in functional/expected/.

If defined, _test_more must be a function with the following signature:

def _test_more(expected_dir, output_dir, test_case, parameters):

This function is called from the test case to perform tests beyond the simple comparison of expected and actual output files.

test_source and test_destination are removed from the namespace, as are all variables whose names begin with an underscore ("_"). The remaining names are passed as keyword arguments to docutils.core.publish_file, so you can set reader, parser, writer and anything else you want to configure. Note that settings_overrides is already initialized as a dictionary before the execution of the config file.

Creating New Tests

In order to create a new test, put the input test file into functional/input/. Then create a config file in functional/tests/ which sets at least input and output file names, reader, parser and writer.

Now run test_functional.py. The test will fail, of course, because you do not have an expected output yet. However, an output file will have been generated in functional/output/. Check this output file for validity and correctness. Then copy the file to functional/expected/.

If you rerun test_functional.py now, it should pass.

If you run test_functional.py later and the actual output doesn't match the expected output anymore, the test will fail.

If this is the case and you made an intentional change, check the actual output for validity and correctness, copy it to functional/expected/ (overwriting the old expected output), and commit the change.

The Default Configuration File

The file functional/tests/_default.py contains default settings. It is executed just before the actual configuration files, which has the same effect as if the contents of _default.py were prepended to every configuration file.