Restructured Text Parsers¶

Sybil includes a range of parsers for extracting and checking examples from Restructured Text including the ability to capture previous blocks into a variable in the namespace, and skip the evaluation of examples where necessary.

doctest¶

Parsers are included for both classic doctest examples along with those in doctest directives. They are evaluated in the document’s namespace. The parsers can optionally be instantiated with doctest option flags.

Here are some classic doctest examples:

>>> context = 'This'
>>> context
'This'
>>> 1 + 1
2

These could be parsed with the a sybil.parsers.rest.DocTestParser in the following configuration:

from sybil import Sybil
from sybil.parsers.rest import DocTestParser
sybil = Sybil(parsers=[DocTestParser()])

If you want to ensure that only examples within a doctest directive are checked, and any other doctest examples are ignored, then you can use the sybil.parsers.rest.DocTestDirectiveParser instead:

This example will never work:

>>> 1 + 1
3

However, this one will:

.. doctest::

    >>> 1 + 1
    2

These could be checked with the following configuration:

from sybil import Sybil
from sybil.parsers.rest import DocTestDirectiveParser
sybil = Sybil(parsers=[DocTestDirectiveParser()])

Note

You will have to enable sphinx.ext.doctest in your conf.py for Sphinx to render doctest directives.

Code blocks¶

The codeblock parsers extract examples from Sphinx code-block directives and evaluate them in the document’s namespace. The boilerplate necessary for examples to successfully evaluate and be checked can hinder the quality of documentation. To help with this, these parsers also evaluate “invisible” code blocks such as this one:

.. invisible-code-block: python

  remember_me = b'see how namespaces work?'

These take advantage of Sphinx comment syntax so that the code block will not be rendered in your documentation but can be used to set up the document’s namespace or make assertions about what the evaluation of other examples has put in that namespace.

Python¶

Python code blocks can be checked using the sybil.parsers.rest.PythonCodeBlockParser.

Here’s a Python code block and an invisible Python code block that checks it:

Here's a function:

.. code-block:: python

  import sys

  def prefix(message):
      return 'prefix:'+message

This won't show up but will verify it works:

.. invisible-code-block: python

  assert prefix('foo') == 'prefix:foo'

These could be checked with the following configuration:

from sybil import Sybil
from sybil.parsers.rest import PythonCodeBlockParser
sybil = Sybil(parsers=[PythonCodeBlockParser()])

Note

You should not wrap doctest examples in a python code-block, they will render correctly without that and you should use the doctest parser to check them.

Other languages¶

Note

If your code-block examples define content, such as JSON or YAML, rather than executable code, you may find the capture parser is more useful.

sybil.parsers.rest.CodeBlockParser can be used to check examples in any language you require, either by instantiating with a specified language and evaluator, or by subclassing to create your own parser.

As an example, let’s look at evaluating bash commands in a subprocess and checking the output is as expected:

.. code-block:: bash

   $ echo hi there
   hi there

We can do this using CodeBlockParser as follows:

from subprocess import check_output
from textwrap import dedent

from sybil import Sybil
from sybil.parsers.codeblock import CodeBlockParser

def evaluate_bash(example):
    command, expected = dedent(example.parsed).strip().split('\n')
    actual = check_output(command[2:].split()).strip().decode('ascii')
    assert actual == expected, repr(actual) + ' != ' + repr(expected)

parser = CodeBlockParser(language='bash', evaluator=evaluate_bash)
sybil = Sybil(parsers=[parser])

Alternatively, we can create our own parser class and use it as follows:

from subprocess import check_output
from textwrap import dedent

from sybil import Sybil
from sybil.parsers.codeblock import CodeBlockParser

class BashCodeBlockParser(CodeBlockParser):

    language = 'bash'

    def evaluate(self, example):
        command, expected = dedent(example.parsed).strip().split('\n')
        actual = check_output(command[2:].split()).strip().decode('ascii')
        assert actual == expected, repr(actual) + ' != ' + repr(expected)

sybil = Sybil([BashCodeBlockParser()])

Capturing blocks¶

sybil.parsers.rest.CaptureParser() takes advantage of Sphinx comment syntax to introduce a special comment that takes the preceding ReST block and inserts its raw content into the document’s namespace using the name specified.

For example:

A simple example::

  root.txt
  subdir/
  subdir/file.txt
  subdir/logs/

.. -> expected_listing

This listing could be captured into the namespace using the following configuration:

from sybil import Sybil
from sybil.parsers.rest import CaptureParser
sybil = Sybil(parsers=[CaptureParser()])

The above documentation source, when parsed by this parser and then evaluated, would mean that expected_listing could be used in other examples in the document:

>>> expected_listing.split()
['root.txt', 'subdir/', 'subdir/file.txt', 'subdir/logs/']

It can also be used with code-block examples that define content rather executable code, for example:

.. code-block:: json

    {
        "a key": "value",
        "b key": 42
    }

.. -> json_source

The JSON source can now be used as follows:

>>> import json
>>> json.loads(json_source)
{'a key': 'value', 'b key': 42}

Note

It’s important that the capture directive, .. -> json_source in this case, has identical indentation to the code block above it for this to work.

Skipping examples¶

sybil.parsers.rest.SkipParser takes advantage of Sphinx comment syntax to introduce special comments that allow other examples in the document to be skipped. This can be useful if they include pseudo code or examples that can only be evaluated on a particular version of Python.

For example:

.. skip: next

This would be wrong:

>>> 1 == 2
True

If you need to skip a collection of examples, this can be done as follows:

This is pseudo-code:

.. skip: start

>>> foo = ...
>>> foo(..)

.. skip: end

You can also add conditions to either next or start as shown below:

>>> import sys

This will only work on Python 3:

.. skip: next if(sys.version_info < (3, 0), reason="python 3 only")

>>> repr(b'foo')
"b'foo'"

As you can see, any names used in the expression passed to if must be present in the document’s namespace. invisible code blocks, setup methods or fixtures are good ways to provide these.

When a condition is used to skip one or more following example, it will be reported as a skipped test in your test runner.

If you wish to have unconditional skips show up as skipped tests, this can be done as follows:

This example is not yet working, but I wanted to be reminded:

.. skip: next "not yet working"

>>> 1.1 == 1.11
True

This can also be done when skipping collections of examples:

And here we can see some pseudo-code that will work in a future release:

.. skip: start "Fix in v5"

>>> helper = Framework().make_helper()
>>> helper.describe(...)

.. skip: end

The above examples could be checked with the following configuration:

from sybil import Sybil
from sybil.parsers.rest import DocTestParser, SkipParser
sybil = Sybil(parsers=[DocTestParser(), SkipParser()])

Clearing the namespace¶

If you want to isolate the testing of your examples within a single source file, you may want to clear the namespace. This can be done as follows:

>>> x = 1
>>> x
1

Now let's start a new test:

.. clear-namespace

>>> x
Traceback (most recent call last):
...
NameError: name 'x' is not defined

The following configuration is required:

from sybil import Sybil
from sybil.parsers.rest import DocTestParser, ClearNamespaceParser
sybil = Sybil(parsers=[DocTestParser(), ClearNamespaceParser()])