How to Use Doctest to Enhance Your Python Documentation and Tests
Doctest is a Python module embedded in the python standard library that evaluates code for blocks considered interactive Python sessions. Once found, it will execute those pieces of code to validate that they behavie exactly as they are documented.
"Python has two basic modes: script and interactive. The normal mode is the mode where the scripted and finished .py files are run in the Python interpreter. Interactive mode is a command line shell which gives immediate feedback for each statement, while running previously fed statements in active memory. As new lines are fed into the interpreter, the fed program is evaluated both in part and in whole." - wikibooks.org
As per Python Docs, doctest can be used several ways:
- Run interactive sessions from the
docstringsof a module, class or method to evalute that the code and documentation are up-to-date - Execute a series of tests from a test file, an approximation to of regression tests or simply to display examples
- Enhancing the documentation of a module by describing input and output examples. This examples described in the docstrings are considered
executable documentation
Use Case 1 - Import doctest into your modules
The easiest way to start using doctest is to include the following code snippet at the end of your modules.
if __name__ == "__main__":
import doctest
doctest.testmod()
When you directly run the module (python your_module.py), doctest will verify and run all interactive examples specified in the docstrings. However, by default it will send to stdout only failed examples. To output in the terminal all the examples use python your_module.py -v.
Below is a module example, containging two methods and the proper documentation in order for doctest to recognize the interactive code and execute it.
"""
An example module that provides basic mathematical functions.
>>> add(3,5)
8
>>> subtract(2,6)
-4
"""
def add(x:int, y:int) -> int:
""" Performs addition of two integer numbers
Args:
x (int)
y (int)
Returns:
int: addition result
>>> add(30, 3)
33
>>> add("20", 3)
Traceback (most recent call last):
...
TypeError: add() expects args to be of type 'int'
"""
try:
return x + y
except TypeError:
raise TypeError("add() expects args to be of type 'int'")
def subtract(x:int, y:int) -> int:
""" Performs subtraction of two integer numbers
Args:
x (int)
y (int)
Returns:
[int]: subtraction result
>>> subtract(30, 24)
6
>>> subtract("35", 24)
Traceback (most recent call last):
...
TypeError: subtract() expects args to be of type 'int'
"""
try:
return x - y
except TypeError:
raise TypeError("subtract() expects args to be of type 'int'")
if __name__ == "__main__":
import doctest
doctest.testmod()
For doctest to recognize an interactive example from the docstrings of a module or method you need add the following syntax.
"""
>>> subtract(30, 24)
6
"""
This is what doctest looks for in the docstring of a module. When it finds >>> subtract(30, 24), it will execute it in an interactive python console session and it will assert the standard output of the function with the expected output 6.
Examples of negative test were also created, where the function will raise an error to the standard output due to invalid arguments passed.
"""
>>> subtract("35", 24)
Traceback (most recent call last):
...
TypeError: subtract() expects args to be of type 'int'
"""
Passing a string caused a TypeError to be raised. This example illustrates how to deal with standard outputs when multiple lines are printed.
Traceback (most recent call last):is the first line displayed,TypeError: subtract() expects args to be of type 'int'is the last line of the output...ignores any lines displayed in between.
Heres the full console output displayed when doctest.testmod() runs in the example module simple_math shown above.
Trying:
add(3,5)
Expecting:
8
ok
Trying:
subtract(2,6)
Expecting:
-4
ok
Trying:
add(30, 3)
Expecting:
33
ok
Trying:
add("20", 3)
Expecting:
Traceback (most recent call last):
...
TypeError: add() expects args to be of type 'int'
ok
Trying:
subtract(30, 24)
Expecting:
6
ok
Trying:
subtract("35", 24)
Expecting:
Traceback (most recent call last):
...
TypeError: subtract() expects args to be of type 'int'
ok
3 items passed all tests:
2 tests in simple_math
2 tests in simple_math.add
2 tests in simple_math.subtract
6 tests in 3 items.
6 passed and 0 failed.
Test passed.
Use Case 2 - Run testmod() from the console
If importing doctest to all your modules is a bit too much you can always just run testmod from the terminal by typing python -m doctest -v your_module.py.
Use Case 3 - Run doctest from a text file
Alternatively, you can setup your interactive tests in a separate test file, not within docstrings of a module, by using the doctest.testfile("<filename>")
Here's an example of how the example text file looks like for the simple_math module.
The ``simple_math`` module
======================
Using ``add``
----------------------
Example from text file in reStructuredText format
First import ``add`` from the ``simple_math`` module:
>>> from simple_math import add
Now execute add
>>> add(13, 17)
30
Similarly to the used cases 1 and 2, the text file can be run from either python code or the console.
Code
import doctest
doctest.testfile("simple_math_examples.txt")
Command
python -m doctest -v simple_math_examples.txt
This will be the output after running the interactive examples from the text file.
Trying:
from simple_math import add
Expecting nothing
ok
Trying:
add(13, 17)
Expecting:
30
ok
1 items passed all tests:
2 tests in simple_math_examples.txt
2 tests in 1 items.
2 passed and 0 failed.
Test passed.
Use doctest to your advantage
Doctest is great because is a mix of documentation and testing. You can quickly add examples to your code which will help your future self or other developers down the road. You can also go further and create a rich "tutorial" or "documentation" text file where anyone can run it to get a better understanding of your module and how its intended to be used.