This page is about the Doc-a-thon on Toggles and Settings, taking place January 25-29, 2021, and extended through March 15.

Participation will take a couple of hours to help research and document some of our feature toggles or settings.

Why Document Toggles and Settings?

The documentation for toggles and settings is a crucial source of information for a very wide audience, including platform administrators, product owners and developers.

Additionally, we can help reduce the complexity of our system by properly documenting which toggles and settings were meant to be temporary, along with target removal dates (as detailed later), to help drive deprecation/removal.

Schedule

• January 25, 11am EST / 4pm UTC: (Optional) Opening meeting on Zoom

  • (Optional) Watch recorded Opening meeting (Passcode: $VA7g^s6)

• January 25-29, asynchronous doc-a-thon efforts.

• January 29, 11:30am EST / 4:30pm UTC: Watch recorded Closing meeting (Passcode: N.R8=3Zn)

• UPDATE: doc-a-thon extended through March 15.

Asking for Help

There is a dedicated Open edX Slack channel #doc-a-thon.

• Please join if you want to discuss, ask questions, and exchange information.

• We will have several community or edX team members available there to answer your questions and help.

  • FYI: edX employees can access this same channel from our Slack instance. Just ask to be invited.

Recognition

• Feel great for celebrating the International Day of Education (Jan 24) by making it easier to use the Open edX platform to help educate everyone, everywhere.

• Earn a participation badge or a top-2-participants, badge if badges are your thing.

• Get mentioned or highlighted in our Open edX blog article detailing the results of the doc-a-thon.

Process for PR Authors

1. Short Background Reading

Get familiar with the type of documentation we will be writing, which uses annotations.

• Toggles (includes boolean Django Settings)

  • Read how to document feature toggles (including boolean settings)

• Settings (non-toggle, non-boolean)

  • See an example annotated setting.

  • See the definition for all setting annotations.

    • Note: There are fewer setting annotations than toggle annotations, but there is an optional setting_warning annotation.

2. Sign up

Choosing a setting or toggle in the LMS Sign-up for Toggle and Settings Doc-a-thon spreadsheet:

• First, prioritize documenting Toggles over Settings.

  • All unassigned Toggles have a yellow background on the sheet and have a value of “Toggle” in the “Toggle or Setting” column.

  • Only if you don’t see a Toggle that’s available for you, then sign up for a “Setting”.

  • Why: We have 200 Toggles and 530 Settings. We want to complete Toggles first since they:

    • provide an on-ramp to our deprecation/removal (DEPR) pipeline and allow Open edX operators discoverability into the various features they can turn on and off.

• Next, choose a setting or toggle that matches the complexity-level you are looking for:

  • Most complex:

    • If the sign-up column Bonus Refactor column is True, there is a somewhat more complicated refactor requested (see below). If you don’t want this challenge, please choose a different toggle.

  • Somewhat complex:

    • If the sign-up column class starts with “Legacy”, there is some minor refactoring requested (see below). If you don’t want to do the extra refactoring, please choose a different toggle.

  • Optionally complex:

    • If you choose a toggle setting, there is an optional refactor described below.

  • Easiest:

    • If the module column has a value, and none of the above cases match, you should be able to quickly locate the toggle or setting.

• Next, add your github username to the row to claim it as yours.

  • Choose any that interests you, whether you are an expert or not. If you need help from experts, just ask for help.

3. Research

Research the toggle or setting to prepare for your documentation.

• Find where your toggle or setting is defined.

  • If the sign-up sheet includes a value in the module column, it should help you quickly locate the definition.

    • Many modules should be in edx/edx-platform, but it could be in the repo of a dependency.

  • If the module is not available, you’ll need to use search:

    • https://github.com/search?q=org%3Aedx+TYPE_YOUR_TOGGLE_OR_SETTING_HERE&type=code.

  • If you can’t find the toggle or setting, ask for help and/or add a note to the sign-up spreadsheet and choose a new row.

    • You can use git log search (a.k.a. pickaxe) to find when things were added/removed. This may help you determine if something wasn’t fully cleaned up. Also, here is another article on using pickaxe with additional options.

  • Note: It is possible that some toggles or settings in the sign-up sheet are from Studio and not LMS. Either way, we can document it.

• Learn about the purpose of the toggle.

  • Important: Save any links to useful background and documentation for the PR description.

  • Explore code, git blame, and past PRs and commit comments.

  • Search edX Feature Flags to potentially find past notes.

  • Search the additional reference tab in the sign-up spreadsheet to potentially find notes or date details.

• You may not feel confident in what to fill in for all of the annotations you’ll need to write. That is 100% ok. Please keep going.

  • In the description annotation, feel free to note any hesitations or partial confidence you have in any of the values.

  • An example sentence might read: “Due to a lack of history, it is unclear whether this toggle was meant to be temporary or not.”

4. Write the annotation

Please try to annotate a single toggle or setting in each PR. See “Create your PR” for more details.

• General tips

  • Remember that your documentation will be used by platform administrators, product owners and developers, and should address all of their needs.

  • See “Short Background Reading” (above) for more details on how to write your annotations.

  • Linting:

    • PRs in edx-platform will have linting enabled (soon) to ensure correctness on your annotations.

      • Note: You may be asked to rebase if you create your PR before linting lands.

    • If you are working in a different repo, ask if this linting can be added.

• For toggle setting or non-toggle setting:

  • Whether the setting is a toggle (boolean), or non-toggle (non-boolean), the toggle may be defined in the Open edX codebase, or the envs file may simply be setting the default for a setting defined in a third-party dependency.

  • If the setting is being set for a third-party dependency, please note that in the annotation. If possible, provide a link to the third-party docs for the setting in the description annotation.

  • If the sign-up sheet reads toggle setting, use toggle annotations and if it reads non-toggle setting, use setting annotations. See “Short Background Reading” (above) for details.

  • (Optional) Refactor for toggle setting.

    • The following refactor only applies to Django Settings that are defined in the Open edX codebase, and not for defaults that are being set for settings defined in third-party dependencies.

    • There are new classes named SettingToggle and SettingDictToggle to support boolean Django Settings that are used as a toggle.

      • Read the ADR for SettingToggle and SettingDictToggle for background and sample usage.

      • Add a SettingToggle or SettingDictToggle definition to represent your chosen toggle.

      • This new class should be added to the Django app that most closely relates to the setting.

        • If all usages of the setting exist in one Django app, the choice is simple.

        • If the setting is used in both LMS and Studio, you can add it to openedx/core/toggles.py.

        • If you have questions about where to locate the definition, please ask for help.

      • Once you have added the definition, you can use it in place of settings throughout, as detailed in the ADR above or the sample PR below.

      • See this example PR refactoring to use SettingDictToggle.

• For Legacy waffle classes:

  • If you selected a deprecated Legacy waffle class, please help update it.

    • Import WaffleFlag instead of LegacyWaffleFlag or WaffleSwitch instead of LegacyWaffleSwitch.

    • For initializing these classes, provide an updated string with the flag name containing the namespace followed by a period, in place of the separate namespace string or class.

      • For example, if the legacy namespace was “my_namespace” and the legacy flag name was “my_flag”, then the updated flag name would be “my_namespace.my_flag”.

      • If the code was using LegacyWaffleFlagNamespace or LegacyWaffleSwitchNamespace and it is no longer used, remove it.

• Bonus refactoring:

  • If you signed-up for a toggle with Bonus Refactor as True in the sign-up sheet, there is a slightly more complex refactor requested.

  • You’ll need to replace uses of any of the following functions def waffle(), def waffle_flags(), def waffle_switch(name).

  • In place of these functions that are used to dynamically create the toggle classes, you will create a module level constant for each toggle.

    • See this PR with example replacement of waffle() and waffle_switch(name)

5. Create your PR

As noted above, please try to annotate a single toggle or setting in each PR. If you have reason to believe a PR should include multiple related toggles and settings, please ask. The simpler the PR, the easier it will be to approve and merge.

• Start your PR title with [TSD]. This will designate the PR as part of the Toggle and Setting Doc-a-thon.

• For the PR description:

  • Clearly state the name of the toggle or setting.

  • Include links to any documentation supporting your annotations in the PR description. Don’t make your reviewer repeat all the research you already did.

  • Include any additional details from the sign-in sheet that might be useful, or note any additional refactoring that was requested.

• Add a link to the PR in the PR column in the sign-up spreadsheet.

• We’ll follow the typical review process from here, and merge as soon as we can.

6. Congratulations!

Thank you for documenting one of our toggles or settings! If you want to put your new skills to good use, sign-up for another one and start again.

Process for PR Reviewers

In addition to following the standard PR review process, see the following special considerations:

• Add your github username to the Github reviewer column in the sign-up spreadsheet for the row (or rows) associated with the PR.

  • If the rows are knowingly missing from the sign-up spreadsheet, Robert Raposa can help add it.

  • If the PR link is missing from the sign-up spreadsheet, please update or ask the author to update.

• If the PR title or description doesn’t include the information requested under “Create your PR” above, please ask for it and move on. This information should make your review much simpler.

• Read through the “Process for PR Authors”.

  • Note: some sections may not apply to the PR you are reviewing, and the PR description should help you understand where you need to focus.

• If the PR contains multiple toggles and settings, and it isn’t clear to you why, feel free to ask that it be split into multiple PRs.

• If the PR is in edx-platform, linting will help ensure correctness of the annotations. Your job will be more focused on ensuring the annotation details seem reasonable based on the details provided in the PR description.

• If the PR is for a repo other than edx-platform, ask for help to see if we can enable linting for that repository.

  • Also, linting is still in-progress for edx-platform, so PRs may need to rebase to pick this up once it is ready.

FAQ

Where can I see the toggle and settings documentation?

• Inline in the code where it is defined.

  • Here is an annotated non-toggle setting.

  • Here is an annotated toggle.

• For edx-platform, the annotations are published to readthedocs.

  • Notes:

    • Some toggle annotations are not included in this document, but are used as part of other reporting. Please provide feedback if you’d like to see additional details.

    • Annotations defined in edx-platform dependencies are not yet included in this report.

Where can I see linting results?

Note: Additional linting is still in progress. You may need to rebase your PR to pick up additional linting.

You can see toggle annotation errors by searching the quality pylint errors for type “toggle” or “setting”.

Can I add a new toggle or setting, or should I only document what is on the list?

Sure, you can add new ones. Reach out to Robert Raposa to add new toggles to the sheet. You can ping in the #doc-a-thon Slack channel.

Should I document something I am an expert in or on can I pick something I am not familiar with?

You can pick anything you are interested in, whether you are an expert or not. If you are not an expert, feel free to ask on Slack. 

Are there PR reviewers assigned?

If you are a core contributor and can review, please do so. Also, anyone is welcome to help do a first-pass review. Robert will work on finding reviewers to get the rest of the PRs merged.

If flag is planned to be removed, should I remove it?

For now, please annotate that it is slated for removal/deprecation in the annotation as detailed in the how-to document.

How do I document a setting or toggle that can be configured independently for both, LMS and CMS service variants? do I have to duplicate the annotations in both env files?

There are some settings or toggles that are defined in both, LMS and CMS env files. Before duplicating the annotations in both env files, please perform research on the setting/toggle purpose in both service variants. Then you can proceed according to the following scenarios:

• The setting/toggle servers to the same purpose in both service variants: Document both the LMS and CMS settings using annotations, but make the CMS annotations refer to the LMS annotations rather than duplicating text on both sides.  Use the warning annotation to highlight if the setting/toggle needs to be set in both LMS and CMS with the same value, or any other non-obvious considerations. You can find an excellent example of this case in this pull request, where CMS annotations reference LMS annotations

• The setting/toggle servers to different or additional purposes: make the annotations differ indicating the purpose for every service variant using the regular steps indicated in this document

Retro

Pluses

1. Communications

   1. Announced earlier at contributors meeting

   2. Announced in discourse - including banner!

   3. Announced in slack

   4. Separate slack channel was great!

2. This document was very useful in getting started. Regis’s video was also nice to get started, having the full process recorded would be really nice.

3. Spreadsheet to self-serve for signups and add notes.

4. Opening and closing ceremonies!

   1. With Metrics!

Thoughts

1. Perfect is the enemy of good?

   1. What could have sped up the PR reviews?

      1. (Future) Toggle linting and lint amnesty will provide faster feedback loops.

      2. We were resolving open questions while reviewing.

      3. Splitting delegation and reviewing to different people.

2. Extend the event to a longer duration!

   1. What are the bottlenecks?

   2. Longer time → How to keep the energy up?

      1. Have a goal!

3. Badges and “competition” helps encourage

   1. Would prefer to externalize outside of Discourse.

   2. Top contributor award as well

Future Notes

1. edX Legal team needs to be given heads up about the doc-a-thon. If many new authors participate, signing a new contributor agreement and Legal processing them can become a bottleneck. Also, need this agreement in Docusign!