Building the Docs

Purpose of this page

This page explains how this project’s documentation is built and published.

For information on maintaining the content of documentation, see Maintaining the Docs.

Introduction

The majority of the documentation in ApprovalTests.cpp is maintained in Markdown format.

However, for a nicer user experience, that documentation is also generated with Sphinx and then published on Read the Docs, at approvaltestscpp.readthedocs.io.

In most cases, it is sufficient to edit the Markdown files, and the Read the Docs site will take care of itself, as soon as changes are pushed to GitHub.

This page describes the mechanics of the documentation processes, in case they need to be worked on in future.

Overview

The mechanism for that publishing is based on Sy Brand’s Clear, Functional C++ Documentation with Sphinx + Breathe + Doxygen + CMake.

That article gives an excellent summary of the technologies and techniques involved.

Required Tools

Tools:

Python3 modules:

  • The required modules are defined in doc/requirements.txt

    • That can be installed by running build/install_python_requirements.sh

  • Currently, they are:

    • breathe

      • “Breathe provides a bridge between the Sphinx and Doxygen documentation systems.”

    • pypandoc

      • “Pypandoc provides a thin wrapper for pandoc, a universal document converter.”

    • sphinx_rtd_theme

CMake Targets

On developer machines, where the required tools are installed, the following CMake targets are created:

CMake targets for building documentation

CMake targets for building documentation

To run all these steps, and open the Sphinx output in a web browser:

# in a cmake-created build tree:
cmake --build . --target Sphinx && open doc/sphinx/index.html

On platforms other than macOS, replace the open command with whatever command opens a file in a web browser.

About Read the Docs

“Read the Docs” builds

  • The documentation is automatically updated whenever changes are pushed to the main branch.

  • It typically takes 1 to 2 minutes for the documentation to be updated

  • We currently don’t get notifications for any build failures

  • There is a “docs” build badge alongside our other badges on the github page, though.

  • To see the build logs:

    1. click on the little green “v.latest” at the bottom of the navigation panel.

    2. click on “Builds”

    3. or click this link to the Builds.

Known Issues

Issues with Sphinx Output

  • Images are loaded from github, rather than being copied in to the Sphinx output. This means that when editing the documentation, changes to images need to be pushed to GitHub to see the effect.

  • Formatting

    • pandoc wraps long lines in tables. This results in long words in tables being broken up, for example there are spaces in some macro names in “Consistent macro names”

    • Formatting of the C++ code isn’t great: for example, it would benefit from more whitespace, for readability.

  • URLs

    • I am unsure about the appearance of /generated_docs/ in the URLs of pages generated from MarkDown.

      • It avoids having to git-ignore some .rst files in a folder that contains a version-controlled .rst file

      • But it does clutter up and create longer URLs

  • Links

    • In the PDF version, there are some broken links

    • Links in the API docs - that were generated from Doxygen - go to the docs on GitHub. It would be much nicer if they jumped to the relevant Sphinx page.

Implementation Details

The rest of this document explains the file conversion processes, in case anyone else needs to maintain them.

Images

  • doc/images/*

    • Images for inclusion in docs

  • doc/images/tutorial/*

    • Images for inclusion in docs

  • doc/images/source/*

    • Sources for some of the images.

  • doc/images/source/generate_images.py

    • Script that runs graphviz’s dot to generate images from some source files.

Step 1: mdsnippets and Markdown Files

mdsnippets Summary

  • Purpose:

    • Update the machine-generated markdown files, which will later be used as inputs to the Sphinx documentation

  • Output Files:

    • doc/*.md

    • doc/explanations/*.md

    • doc/how_tos/*.md

Flow of Markdown files through mdsnippets

Flow of Markdown files through mdsnippets

mdsnippets Details

  • Configuration files:

    • doc/run_mdsnippets/CMakeLists.txt

      • Creates a CMake target RunMdsnippets

        • Note: this is not included in the target all

      • This target makes it convenient to quickly run run_markdown_templates.sh from within CLion, without switching to a console window.

    • mdsnippets.json

      • Configuration used by mdsnippets

  • Input files:

Step 2: Doxygen conversion

Doxygen Summary

  • Purpose:

    • Read the library’s source code, to generate a set of XML files that describe the API

    • These XML files will later be read by Sphinx to create the API documentation

  • Output Files:

    • build_tree/doc/doxygen/xml/index.xml

      • This and related .xml files are supplied to Sphinx, to generate the API docs on Read the Docs

    • build_tree/doc/doxygen/html/index.html

      • This file may be useful on developer machines, to review the documentation generated by Doxygen

Flow of files through doxygen

Flow of files through doxygen

Doxygen Details

  • Configuration files:

    • doc/doxygen/CMakeLists.txt

      • Creates a CMake target Doxygen

        • Note: this is not included in the target all

    • doc/doxygen/Doxyfile.in

      • Template file containing the Doxygen configuration

      • CMake converts it to Doxyfile in the build tree

  • Input files:

    • doc/doxygen_doc/*.dox

      • Files containing descriptive text for including in the Doxygen documentation, which will then be included in the Sphinx docs.

    • doc/ApprovalTests/*.cpp

    • doc/ApprovalTests/*.h

Step3: reStructuredText and Sphinx

Sphinx Summary

  • Purpose:

    • Use the Sphinx system to generate a nicely formatted, usable version of our Markdown and C++ documentation, for serving from Read the Docs

    • This involves processing and converting our Markdown files to reStructuredText format

  • Output Files:

Flow of files through Sphinx

Flow of files through Sphinx

Sphinx Details

  • Configuration files:

    • doc/requirements.txt

      • The Python requirements for running all Python scripts in doc/

      • Can be installed with pip3 by running build/install_python_requirements.sh

    • doc/sphinx/CMakeLists.txt

      • Creates a CMake target Sphinx

        • Note: this is not included in the target all

    • doc/sphinx/_templates/breadcrumbs.html

      • Prevents a broken “Edit on GitHub” from being added to each page on Read the Docs

      • This is because most of our Sphinx Documentation pages are machine-generated, so the “Edit on GitHub” would take users to a non-existent page

  • Scripts:

    • doc/sphinx/__init__.py

    • doc/sphinx/conf.py

      • This contains our Sphinx configuration

      • It also runs the script markdown_conversion.py

      • It has some extra steps that are run only when generating the docs on Read the Docs (where CMake is not available)

    • doc/sphinx/markdown_conversion.py

      • Convert all the *.md files generated by mdsnippets to *.rst files for feeding in to Sphinx

      • For example, it changes some links so that they go to other pages in Sphinx

      • And other links it sends to the GitHub site

  • Tests:

    • doc/sphinx/tests/test_markdown_conversion.py

      • Unit and Approval Tests for doc/sphinx/tests/test_markdown_conversion.py

    • doc/sphinx/tests/test_markdown_conversion_input.md

      • An input file with a range of different types of Markdown constructs, taken from our own documentation

    • doc/sphinx/tests/TestWholeConversion.test_convert_markdown_for_pandoc.approved.md

      • To see the 1st stage of transformations made to markdown files, compare this with:

        • doc/sphinx/tests/test_markdown_conversion_input.md

    • doc/sphinx/tests/TestWholeConversion.test_convert_markdown_for_pandoc.approved.rst

      • To see how converted mardown files appear in .rst format, compare this with:

        • doc/sphinx/tests/TestWholeConversion.test_convert_markdown_for_pandoc.approved.md

  • Input files:

    • doc/sphinx/index.rst

    • doc/sphinx/api/*.rst

    • Plus all the outputs from mdsnippets:

      • doc/*.md

      • doc/explanations/*.md

      • doc/how_tos/*.md

  • Intermediate files:

    • doc/sphinx/generated_docs/*.rst

    • doc/sphinx/generated_docs/explanations/*.rst

    • doc/sphinx/generated_docs/how_tos/*.rst

    • These are all ignored by git