Toggles and Settings Documentation

Overview

The following toggles and settings tooling potential tasks around toggles and settings tooling, was generated while working on [BD-21] Toggles/Settings Documentation, Toggles and Settings Doc-a-thon 2021, and earlier work from the Arch-BOM team.

Next Steps

• Write announcement for value created by BD-21.

• Cost of Delay / Prioritization of future work, when we decide to continue.

• Potential for surveys or interviews as input to prioritization.

• Discuss continuation of blended for this work.

Potential Tasks

Linting

• Need to add deprecation ticket to a separate annotation for convenient filtering out during future reviews

• (HIGH) DEPR improvement ideas:

  • https://github.com/openedx/edx-toggles/issues/284

  • https://github.com/openedx/edx-toggles/issues/283.

• (HIGH) https://github.com/openedx/edx-toggles/issues/286.

• (MEDIUM) https://github.com/openedx/edx-toggles/issues/291

• Check toggle_implementation annotation matches actual implementation class used when possible.

  • Note: examples of manually corrected errors.

• No linting exists to ensure settings are annotated.

  • Needs brainstorming if we think this would be important.

• No linting exists to ensure ConfigModels with booleans are annotated.

Improvements to published docs and reporting

• (HIGH) https://github.com/openedx/edx-toggles/issues/287

• (HIGH) https://github.com/openedx/edx-toggles/issues/288

• (HIGH) https://github.com/openedx/edx-toggles/issues/289

• Include more annotated settings in RTD documentation.

  • Context:

    • For settings toggles we currently only capture annotations if they are in lms/envs/common.py or cms/envs/common.py

    • We have settings annotations where the default and annotation live outside of those two files within edx-platform. See an example annotation.

    • We will have annotations where the default live in an external library (edx-when, or other edx owned libraries)

    • See some discussion and disagreements in Slack in #external-openedx-toggles-and-settings.

  • Potential Tickets

    • Discovery: How can we capture settings toggles not documented in lms/envs/common.py or cms/envs/common.py but are still in the edx-platform repo

      • AC:

        • Either a POC that captures settings in other places or an ADR to guide us on where these docs should live.

    • Discovery: How do we collect annotations from libraries installed in edx-platform?

• Allow for formatting in annotations, rather than having everything be plain-text.

  • Consider allowing arbitrary formatted rST. 

• Remove pylint disable pragmas from annotation output.

  • All, or just line-too-long?

  • See examples (including amnesty).

• RTD developer documentation single book for edx-platform

  • The edx-platform toggles and settings documentation was published as a separate book as a starting point, because the rest of the edx-platform docs weren’t published.

  • Would be nice to fix before too many references to docs that might be moving.

• CI testing for docs

  • edx-platform doesn’t test make technical-docs (current toggles and settings docs)

  • Spotty support for testing make docs for other repos.

• Reporting on suspicious toggle state data

  • Examples:

    • Bad data like waffle names with leading or trailing spaces in the database.

    • Potentially orphaned data (data with no annotation and no toggle definition)

  • Toggle State Report may be simplest location, using a new column

Toggle Enhancements

• (HIGH) https://github.com/openedx/edx-toggles/issues/290

• Replace or improve ExperimentWaffleFlag

  • See Learning MFE toggle feedback