Usage

As a quick-start, here’s how you would set up a conftest.py in the root of your project such that running pytest would check examples in your project’s source code and Sphinx source. doctest and Python code-block examples will be checked:

from doctest import ELLIPSIS
from sybil import Sybil
from sybil.parsers.capture import parse_captures
from sybil.parsers.codeblock import PythonCodeBlockParser
from sybil.parsers.doctest import DocTestParser

pytest_collect_file = Sybil(
    parsers=[
        DocTestParser(optionflags=ELLIPSIS),
        PythonCodeBlockParser(),
    ],
    patterns=['*.rst', '*.py'],
).pytest()

You’ll also want to disable pytest’s own doctest plugin by putting this in your pytest config:

addopts = -p no:doctest

An example of a documentation source file that could be checked using the above configuration is shown below:

Sample Documentation
====================

Let's put something in the Sybil document's namespace:

.. invisible-code-block: python

  remember_me = b'see how namespaces work?'

Suppose we define a function, convoluted and pointless but shows stuff nicely:

.. code-block:: python

  import sys

  def prefix_and_print(message):
      print('prefix:', message.decode('ascii'))

Now we can use a doctest REPL to show it in action:

>>> prefix_and_print(remember_me)
prefix: see how namespaces work?

The namespace carries across from example to example, no matter what parser:

>>> remember_me
b'see how namespaces work?'

Method of operation

Sybil works by discovering a series of documents as part of the test runner integration. These documents are then parsed into a set of non-overlapping regions. When the tests are run, each Region is turned into an Example that is evaluated in the document’s namespace. The examples are evaluated in the order in which they appear in the document. If an example does not evaluate as expected, a test failure occurs and Sybil continues on to evaluate the remaining examples in the Document.

Test runner integration

Sybil aims to integrate with all major Python test runners. Those currently catered for explicitly are listed below, but you may find that one of these integration methods may work as required with other test runners. If not, please file an issue on GitHub.

To show how the integration options work, the following documentation examples will be tested. They use doctests, code blocks and require a temporary directory:

Another fairly pointless function:

.. code-block:: python

  import sys

  def write_message(filename, message):
      with open(filename, 'w') as target:
          target.write(message)

Now we can use a doctest REPL to show it in action:

>>> write_message('test.txt', 'is this thing on?')
>>> with open('test.txt') as source:
...     print(source.read())
is this thing on?

pytest

To have pytest check the examples, Sybil makes use of the pytest_collect_file hook. To use this, configuration is placed in a confest.py in your documentation source directory, as shown below. pytest should be invoked from a location that has the opportunity to recurse into that directory:

from os import chdir, getcwd
from shutil import rmtree
from tempfile import mkdtemp
import pytest
from sybil import Sybil
from sybil.parsers.codeblock import PythonCodeBlockParser
from sybil.parsers.doctest import DocTestParser

@pytest.fixture(scope="module")
def tempdir():
    # there are better ways to do temp directories, but it's a simple example:
    path = mkdtemp()
    cwd = getcwd()
    try:
        chdir(path)
        yield path
    finally:
        chdir(cwd)
        rmtree(path)

pytest_collect_file = Sybil(
    parsers=[
        DocTestParser(),
        PythonCodeBlockParser(future_imports=['print_function']),
    ],
    pattern='*.rst',
    fixtures=['tempdir']
).pytest()

The file glob passed as pattern should match any documentation source files that contain examples which you would like to be checked.

As you can see, if your examples require any fixtures, these can be requested by passing their names to the fixtures argument of the Sybil class. These will be available in the Document namespace in a way that should feel natural to pytest users.

The setup and teardown parameters can still be used to pass Document setup and teardown callables.

The path parameter, however, is ignored.

Note

pytest provides its own doctest plugin, which can cause problems. It should be disabled by including the following in your pytest configuration file:

[pytest]
addopts = -p no:doctest

unittest

To have Test Discovery check the example, Sybil makes use of the load_tests protocol. As such, the following should be placed in a test module, say test_docs.py, where the unit test discovery process can find it:

from os import chdir, getcwd
from shutil import rmtree
from tempfile import mkdtemp
from sybil import Sybil
from sybil.parsers.codeblock import PythonCodeBlockParser
from sybil.parsers.doctest import DocTestParser

def sybil_setup(namespace):
    # there are better ways to do temp directories, but it's a simple example:
    namespace['path'] = path = mkdtemp()
    namespace['cwd'] = getcwd()
    chdir(path)

def sybil_teardown(namespace):
    chdir(namespace['cwd'])
    rmtree(namespace['path'])

load_tests = Sybil(
    parsers=[
        DocTestParser(),
        PythonCodeBlockParser(future_imports=['print_function']),
    ],
    path='../docs', pattern='*.rst',
    setup=sybil_setup, teardown=sybil_teardown
).unittest()

The path parameter gives the path, relative to the file containing this code, that contains the documentation source files.

The file glob passed as pattern should match any documentation source files that contain examples which you would like to be checked.

Any setup or teardown necessary for your tests can be carried out in callables passed to the setup and teardown parameters, which are both called with the Document namespace.

The fixtures parameter, is ignored.