Run Book for Debugging/Fixing Docs and Adding CI

Created by Feanil Patel
Last updated: Oct 19, 2021 by Ned Batchelder

Documentation projects on ReadTheDocs can be broken. This page is about how to fix them.

Getting the login for ReadTheDocs

Ask Ned or Feanil to share the Keeper folder for the readthedocs.org login for the “edx” account.

Go to the build Output

The failing build in RTD (ReadTheDocs) will have a stack trace that you can use to start debugging the build.

  1. Go to https://readthedocs.org/dashboard/

  2. Login Using the credentials from Keeper

  3. Find your failing project in the project list.

  4. Click on the project to go to a project page like: https://readthedocs.org/projects/opaque-keys/

  5. Go to the Builds Section.

  6. Click on a failing build to see the build output.

  7. You should see the build output.

  8. Fix any issues you see.(See the Common Problems section below for help.)

Build docs Feedback Loops

Notify on Failures

Many doc projects don’t notify on failure. If you know how the owning team wants to be notified for failures, add the relevant e-mail address as a notification channel under ‘Admin’->'Notifications'

Build docs for pull requests

The best feedback is to use ReadTheDocs to build the docs for each pull request, with the result added as a check to the pull request.

  • Create a .readthedocs.yml file (see below).

  • Add a readthedocs webhook to the repo, and make sure it has the Pull requests events enabled in addition to the usual events (see below).

  • Enable Build pull requests for this project in the readthedocs settings.

Add doc builds to CI

Another approach is to build the docs as part of the GitHub Action CI. If you are using ReadTheDocs check, this isn’t needed.

Docs can often be built locally by going to the docs directory in the repo and running make html. Add this as a test to CI if the repo doesn’t already build docs as a part of CI.

Example PR of doing this in opaque-keys

Use .readthedocs.yml

To fix most problems, you’ll probably need to update the settings. The best advice is to configure ReadTheDocs with a .readthedocs.yml file in your repo. ReadTheDocs has docs about how to use it (natch): https://docs.readthedocs.io/en/stable/config-file/v2.html

A few of our repos already use this file:

Add a new .readthedocs.yml

If the repo is missing a .readthedocs.yml you’ll want to make a new one. Use this example as a starting point:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 # .readthedocs.yml # Read the Docs configuration file # See https://docs.readthedocs.io/en/stable/config-file/v2.html for details # Required: the version of this file's schema. version: 2 # Build documentation in the docs/ directory with Sphinx sphinx: configuration: docs/conf.py fail_on_warning: true # Optionally build your docs in additional formats such as PDF formats: - pdf - epub # Optionally set the version of Python and requirements required to build your docs python: version: "3.8" install: - requirements: requirements/doc.txt

RTD also has a doc on how to migrate to the config file from the web interface.

Testing .readthedocs.yml Changes

  1. Push your changes up to GitHub on your PR branch.

  2. Follow the directions from the Go to the build output section to go to your repos ReadTheDocs project page.

  3. Go to the Builds page for your project and build the latest version of the docs. (This will pull all the latest code including any new branches).

  4. Go to the Versions page for your project.

  5. Under the Activate a Version section, find the name of your PR branch.

  6. Click Activate on your PR branch.

  7. Set your branch to be active (it will be built) and hidden (it won’t be listed in the versions list) and then save the changes.

  8. Now you should be able to run builds of your branch to test any changes you’re making to the .readthedocs.yml file.

  9. Go back to the Builds screen and build your branch as many times as you need while you iterate on changes.

  10. When you’re done testing, de-activate your branch on the Versions page.

Common Documentation Building Problems

Wrong Python Version on RTD

If RTD is trying to build the docs using an older version of Python (2.x) you can update that version.

If your repo has a .readthedocs.yml, update the Python version in that file and push up a new PR.

If it is still configured via the web interface, consider moving your settings to .readthedocs.yml as a part of the fix. You can also update the setting in the web interface by going to ‘Admin’->'Advanced Settings' and try to re-build the project.

Missing Requirements Files

If you see a ‘No such file or directory’ message on the requirements file, the issue might be with the config in your repo.

If your repo has a .readthedocs.yml, ensure that the correct path to the requirements file is in there.

If it is still configured via the web interface, consider moving your settings to .readthedocs.yml as a part of the fix. You can also update the setting in the web interface by going to ‘Admin’->'Advanced Settings' to put in the correct path to the requirements file needed to build the docs.

Missing webhook

The docs might build fine when triggered manually, but not start builds automatically. This means you need to add or fix the webhook:

  1. Visit the Admin - Integrations section of the project dashboard

  2. You need a “GitHub incoming webhook.” If it’s not there, click “Add integration.”

  3. Select “GitHub incoming webhook” from the Integration type dropdown.

  4. ReadTheDocs will try to add the webhook to GitHub, but will likely fail (TODO: why?)

  5. Copy the URL that starts “readthedocs.org/api/v2/…”

  6. In GitHub, go to the repo Settings - Webhooks. Click “Add webhook”

  7. Paste the URL you copied into the Payload URL. Make sure it starts with “https://”

  8. Choose “Let me select individual events” then choose these events:

    1. Branch or tag creation

    2. Branch or tag deletion

    3. Pull requests

    4. Pushes

  9. Finish with “Add webhook”