The following toggles and settings tooling potential tasks around toggles and settings tooling, was generated while working on [BD-21] Toggles/Settings Documentation, Toggles and Settings Doc-a-thon 2021, and earlier work from the Arch-BOM team.
Write announcement for value created by BD-21.
Cost of Delay / Prioritization of future work, when we decide to continue.
Potential for surveys or interviews as input to prioritization.
Discuss continuation of blended for this work.
See https://docs.google.com/spreadsheets/d/1PUq0DkNJLp0SuY1Jt38n_USy1IExlACI2yZt0ZmhD8g/edit#gid=0 for prioritization.
See https://docs.google.com/spreadsheets/d/1PUq0DkNJLp0SuY1Jt38n_USy1IExlACI2yZt0ZmhD8g/edit#gid=0 for prioritization.
Need to add deprecation ticket to a separate annotation for convenient filtering out during future reviews
(HIGH) Add annotation for toggle_target_value (get name reviewed) to document whether to keep the True or False case once removed.
Required for temporary toggles. Is it useful to have this in the code, or is some of this DEPR metadata better kept in the DEPR ticket alone?
(HIGH) Add annotation for toggle_depr_ticket to make it simpler to know if a toggle has an associated DEPR ticket.
Required for temporary toggles? Would it be better to create the DEPR ticket when creating the toggle?
(HIGH) Ensure undefined annotations are not allowed.
For example, if toggle_warnings were used instead of toggle_warning, linting would complain.
Without this, the annotation would be visible in code, but it wouldn’t make it to readthedocs and you wouldn’t know it without checking.
Removing the unnecessary ability to add comments in-between annotations would simplify the regex update required for this.
Note: I think this may be causing us not to catch other linting issues with bad indents.
(MEDIUM) Possible bug where required fields for temporary use case might be missed when multiple use cases are listed.
Note: This is not high priority, because we should prioritize switch to new toggle_life_expectancy annotation, which will make this go away.
(MEDIUM) Ensure optional annotations are not set to None.
Applies to specific annotations where None is not a valid value (e.g. toggle_warning).
See example of where None is copied often and leads to a lot of extra code and useless docs.
[RG]* (HIGH) Ensure a properly formatted date for toggle_target_removal_date annotation.
Examples with toggle_target_removal_date of None, even though it is required.
Many of the examples also added unnecessary text to toggle_warnings that toggle_target_removal_date was not set.
Possibly authors documented during docathon, and were afraid to set a date.
How-to was updated with proposal for how to choose a date. Linting failure help text could point to these docs.
Check toggle_implementation annotation matches actual implementation class used when possible.
No linting exists to ensure settings are annotated.
Needs brainstorming if we think this would be important.
No linting exists to ensure ConfigModels with booleans are annotated.
(HIGH) Add additional existing annotations (e.g. toggle_implementation) to published docs
See https://discuss.openedx.org/t/which-toggle-annotations-should-be-published/4690
AC:
Create ADR
Update sphinx plugin to add new annotations.
(HIGH) Include annotations from dependencies in the automated docs:
Sphinx docs for toggles
Sphinx docs for settings (see open questions)
Toggle state report jobs (on-hold until we see this used?)
(HIGH) Complete implementation of toggle state report outside of edx-platform
Apply to credentials, which was already mocked.
Replace credentials mock toggle state endpoint with a real endpoint.
Update the rest of the config according to the how-to.
Update the how-to to provide a simpler example than edx-platform (which overrides the report).
Add to cookiecutter-django-ida in https://github.com/edx/edx-cookiecutters/?
Include more annotated settings in RTD documentation.
Context:
For settings toggles we currently only capture annotations if they are in lms/envs/common.py or cms/envs/common.py
We have settings annotations where the default and annotation live outside of those two files within edx-platform. See an example annotation.
We will have annotations where the default live in an external library (edx-when, or other edx owned libraries)
See some discussion and disagreements in Slack in #external-openedx-toggles-and-settings.
Potential Tickets
Discovery: How can we capture settings toggles not documented in lms/envs/common.py or cms/envs/common.py but are still in the edx-platform repo
AC:
Either a POC that captures settings in other places or an ADR to guide us on where these docs should live.
Discovery: How do we collect annotations from libraries installed in edx-platform?
Allow for formatting in annotations, rather than having everything be plain-text.
Consider allowing arbitrary formatted rST.
Remove pylint disable pragmas from annotation output.
All, or just line-too-long?
RTD developer documentation single book for edx-platform
The edx-platform toggles and settings documentation was published as a separate book as a starting point, because the rest of the edx-platform docs weren’t published.
Would be nice to fix before too many references to docs that might be moving.
CI testing for docs
edx-platform doesn’t test make technical-docs (current toggles and settings docs)
Spotty support for testing make docs for other repos.
Reporting on suspicious toggle state data
Examples:
Bad data like waffle names with leading or trailing spaces in the database.
Potentially orphaned data (data with no annotation and no toggle definition)
Toggle State Report may be simplest location, using a new column
[RG]* (HIGH) Implement ADR 0005-toggle-life-expectancy.rst.
See ADR Consequences for details.
(HIGH) Enhance BaseToggle with additional methods.
Add is_disabled (not is_enabled), is_on (is_enabled) and is_off (not is_enabled)
Question: Would is_true() and is_false() be better than is_on/is_off?
[Robert Raposa] I like is_true/is_false better. Someone else should second my opinion and we can update this proposal.
Makes a variety of conditions more readable when ENABLE_ or DISABLE_ are in the toggle name.
For example:
not DISABLE_SOME_FEATURE.is_enabled() # BEFORE; brain twister
DISABLE_SOME_FEATURE.is_off() # AFTER
Question: Is this simpler to understand?:
DISABLE_SOME_FEATURE.is_false() # AFTER
This problem was discussed on Slack.
Replace or improve ExperimentWaffleFlag