View Source

Rules of thumb (going forward)

Feature toggles. Use Waffle via waffle_utils for boolean flags to toggle or rollout a feature.
1. Use WaffleSwitch for a simple toggle that can be dynamically changed while the server is running.
2. Use WaffleFlag for a staged rollout of the feature when you want to increase gradually the percentage of the population.
3. Use CourseWaffleFlag for supporting course-level opt-in (during staged rollout) or opt-out (after full rollout) of the feature.
Feature settings. Use Configuration models for more complex settings and non-boolean fields to configure a feature.
1. Configuration models are based on Django models.
2. To remove a model-based config field, use a 2-phase deployment strategy where you remove use of it from the code before removing the field with a migration. See Migration Don'ts.
System settings. Use Django settings for open edX instance/deployment wide configurations that do not change while the server is running.
1. Add the setting's default value to (lms|cms)/envs/common.py (example).
2. Override the value in other relevant (lms|cms)/envs/* files (example).
3. Make sure to update aws.py so the setting can be configurable via JSON files (example).
  1. If the setting is security-sensitive, read its value from AUTH_TOKENS
  2. Else, read the value from ENV_TOKENS

Why are feature flags used?

Staged Rollout:
- To gradually include a feature in releases, without exposing unfinished work to users
Decoupling Deployment from Feature Release:
- To allow organizations running their own edX instances to selectively enable features as they desire

What alternatives are there?

To solve the problem of gradually releasing an unfinished feature, do not be afraid to use a temporary Waffle flag. However, make sure you have a follow-up story to clean it up.
To solve the problem of allowing organizations to pick-and-choose functionality for their instances of the platform:
- Think twice about whether or not the option is necessary. There is a large cost to supporting alternatives. Although it might seem simple to just let people have a flag and decide for themselves, it would be best to avoid options where we can.

How feature flags are currently used (simple case):

As a global boolean, defined in (lms|cms)/envs/common.py
- some hacks put this value in other places too
- ex: (lms/cms)/envs/bok_choy.py for ENABLE_TEAMS
When the flag is needed to “gate” access to the feature, we just ask the global boolean to decide.
- ex: https://github.com/edx/edx-platform/blob/master/lms/djangoapps/teams/__init__.py
This is specifically for edx-platform, and may not be comprehensive.

Feature flag best practices:

If using a flag to have a staged rollout of a feature:
- Use Waffle.
- Work to remove the flag ASAP after full rollout. Track this with a JIRA ticket.
- Be sure that all entry points are covered by a flag check.
- Run tests in 2 ways:
  - With all flags set as expected to be in the next release
  - With all flags set to their eventual end state
If using a flag to enable selective functionality:
- Minimize dependence on code that is covered by the feature flag. Modularity is king.
- Make sure all entry points are masked by a feature flag check
- Run tests both with and without the flag

Using Waffle

Staged Rollout

You can use Waffle and the edx-platform Waffle Utils to have a staged rollout of a feature using a temporary flag. Typically, the sooner you can reach completion, the better. Not all stages or capabilities are required.

Definitions:

CourseWaffleFlag: An edx-platform class that supports Course Overrides of Waffle Flags (in addition to caching and name-spacing). Use this for all waffle flags inside a course. (See Waffle Utils.)
Waffle (Flag): Refers to the out of the box Waffle capabilities.
WaffleFlag: An edx-platform class that supports caching and name-spacing. Use this for all waffle flags outside a course. (See Waffle Utils.)
Waffle Utils: A set of Waffle related utilities, including CourseWaffleFlag and WaffleFlag.

Early Beta

Late Beta (more optional)

Stable

Complete

Use a Waffle Flag to beta test with individual users in Production, both internal and partners. Implement with CourseWaffleFlag or WaffleFlag to enable future rollout capabilities.

Use Waffle Utils Course Override capabilities to add one or more courses to the beta test.

Use Waffle to turn the feature on for everyone in an environment (e.g. edx.org).

Other environments (e.g. edge.edx.org) may lag (or be ahead).

Change the default for the feature to "on" for environments that aren't yet using the flag (e.g. some devstacks, sandboxes, Open edX releases*).

Waffle Utils Course Override could be used to turn the feature off for a course, but this will delay completion, so use sparingly.

The feature has been rolled out to all edX environments. The flag and legacy code should all be removed.

If there is a strong reason to enable the Open edX community to have the same rollout capabilities, consider delaying clean-up until after the next Open edX release.

If you absolutely must support the feature both on and off, transition the Waffle Flag to a more permanent setting or ConfigModel.

Open edX releases*: The aim is for these waffle flags to be short-lived. However, due to timing or complicated rollouts, you may need to consider how to handle the flag across different Open edX releases as described in the table above.

Testing Waffle Flags

Python Unit Tests:

For Waffle, see the waffle docs for unit testing.
In edx-platform, if you are using the WaffleFlag or CourseWaffleFlag, you can instead use the override_waffle_flag decorator implemented in this testutils.py file.

Bok-choy Testing:

Bok-choy tests cannot use the decorator because the server is separate from the test code.
To override in the URL, see the External Test Suites section of the Waffle documentation. Read the following important details as well.
- In order to override a flag, it must first exist in the database.
  - In edx-platform, you can temporarily create a record in common/test/db_fixtures/waffle_flags.json that will be loaded directly into mysql for bok-choy tests only. Note that you can default the flag on or off depending on your needs.
  - Other teams temporarily create a migration which will create the flag in all environments, including Production.
- Here is some example code reloading a page with a waffle flag set to a different value.
- Note: In edx-platform, the WAFFLE_OVERRIDE setting is already taken care of in bok-choy to enable this type of URL override.

Current edX Feature Flag usage, a step-by-step guide:

For a long-lived feature flag, make super sure that the team wants to use a feature flag. Ask many people for their opinions, and explore other options first if they’re available.
Add your feature flag to the doc page edX Feature Flags, along with description, notes, contact, etc. If prior discussions about the flag are on a JIRA ticket, link that in as well!
Modify lms/envs/common.py (or cms/envs/common.py) to include your feature flag (in ALL_CAPS), along with its default value.
At the points where you’ll be “gating” access, get the boolean value like this: settings.FEATURES.get('ENABLE_TEAMS', False). This will check for the feature flag in common.py and use its value if present, or default to false.

Reference: http://martinfowler.com/bliki/FeatureToggle.html