Toggles and Settings Documentation

Overview

The following toggles and settings tooling potential tasks around toggles and settings tooling, was generated while working on https://openedx.atlassian.net/wiki/spaces/COMM/pages/1596358943, https://openedx.atlassian.net/wiki/spaces/AC/pages/2248409171, and earlier work from the Arch-BOM team.

Next Steps

  • Write announcement for value created by BD-21.

  • Cost of Delay / Prioritization of future work, when we decide to continue.

  • Potential for surveys or interviews as input to prioritization.

  • Discuss continuation of blended for this work.

Potential Tasks

Linting

  • Need to add deprecation ticket to a separate annotation for convenient filtering out during future reviews

  • (HIGH) Add annotation for toggle_target_value (get name reviewed) to document whether to keep the True or False case once removed.

    • Required for temporary toggles. Is it useful to have this in the code, or is some of this DEPR metadata better kept in the DEPR ticket alone?

  • (HIGH) Add annotation for toggle_depr_ticket to make it simpler to know if a toggle has an associated DEPR ticket.

    • Required for temporary toggles? Would it be better to create the DEPR ticket when creating the toggle?

  • (HIGH) Ensure undefined annotations are not allowed.

  • (MEDIUM) Possible bug where required fields for temporary use case might be missed when multiple use cases are listed.

    • Note: This is not high priority, because we should prioritize switch to new toggle_life_expectancy annotation, which will make this go away.

  • (MEDIUM) Ensure optional annotations are not set to None.

  • [RG]* (HIGH) Ensure a properly formatted date for toggle_target_removal_date annotation.

  • Check toggle_implementation annotation matches actual implementation class used when possible.

  • No linting exists to ensure settings are annotated.

    • Needs brainstorming if we think this would be important.

  • No linting exists to ensure ConfigModels with booleans are annotated.

Improvements to published docs and reporting

  • (HIGH) Add additional existing annotations (e.g. toggle_implementation) to published docs

  • (HIGH) Include annotations from dependencies in the automated docs:

    • Sphinx docs for toggles

    • Sphinx docs for settings (see open questions)

    • Toggle state report jobs (on-hold until we see this used?)

  • (HIGH) Complete implementation of toggle state report outside of edx-platform

  • Include more annotated settings in RTD documentation.

    • Context:

    • Potential Tickets

      • Discovery: How can we capture settings toggles not documented in lms/envs/common.py or cms/envs/common.py but are still in the edx-platform repo

        • AC:

          • Either a POC that captures settings in other places or an ADR to guide us on where these docs should live.

      • Discovery: How do we collect annotations from libraries installed in edx-platform?

  • Allow for formatting in annotations, rather than having everything be plain-text.

  • Remove pylint disable pragmas from annotation output.

  • RTD developer documentation single book for edx-platform

  • CI testing for docs

    • edx-platform doesn’t test make technical-docs (current toggles and settings docs)

    • Spotty support for testing make docs for other repos.

  • Reporting on suspicious toggle state data

    • Examples:

      • Bad data like waffle names with leading or trailing spaces in the database.

      • Potentially orphaned data (data with no annotation and no toggle definition)

    • Toggle State Report may be simplest location, using a new column

Update Toggle and Setting Annotation Definition

Toggle Enhancements

  • (HIGH) Enhance BaseToggle with additional methods.

    • Add is_disabled (not is_enabled), is_on (is_enabled) and is_off (not is_enabled)

      • Question: Would is_true() and is_false() be better than is_on/is_off?

        • [Robert Raposa] I like is_true/is_false better. Someone else should second my opinion and we can update this proposal.

    • Makes a variety of conditions more readable when ENABLE_ or DISABLE_ are in the toggle name.

    • For example:

      • not DISABLE_SOME_FEATURE.is_enabled() # BEFORE; brain twister

      • DISABLE_SOME_FEATURE.is_off() # AFTER

        • Question: Is this simpler to understand?:

          • DISABLE_SOME_FEATURE.is_false() # AFTER

    • This problem was discussed on Slack.

  • Replace or improve ExperimentWaffleFlag