Moving preexisting feature flags to configuration models

So, you have a cool feature flag or settings variable defined in envs/*.py. It works, it's available where you need it, and you can easily have different values for devstack, aws, bokchoy, whatever. Now, the time has come - you want to move this value in a configuration model, to allow admin panel access to change values without building a new AMI. Read on for details.

Big picture things to keep in mind:

• If done properly, this will take 2 release cycles to allow for seamless deployment of edx.org without outages
• These changes need to be communicated to the open edx community
• Test databases need love too
• This whole process will take ~3 points of developer time, and cannot be done all at once. The process is very release-bound.

Step 1 - Adding your configuration model

(Note that the PR in which I did this process, https://github.com/edx/edx-platform/pull/12376, will be referred to throughout this document)

The first thing to do is, of course, to add your configuration model. This is a full django model, and will have a migration allowing it to be added to the database. Get this step done first - we want the new model to exist in the database, without any responsibilities, ASAP. This will allow for admin access to be granted, and "seed values" put into place before the config model is given the responsibility of providing these values. See https://github.com/edx/edx-platform/pull/12376/files#diff-ca0c4cdbea57f8defaabc59512eba59a for the model I added in my PR, https://github.com/edx/edx-platform/pull/12376/files#diff-bd2b3986dd3a496d43b8dd9ccfeab74a for registering it as an admin config model, and https://github.com/edx/edx-platform/pull/12376/files#diff-a660ac61c9983985db65ce0a28953ed1 for the accompanying migration.

Step 2 - Ensure correct initial values

This step involves your friendly local devops rep, and should be done after the model exists and the migration has been run as part of the weekly release. This step does not block step 3 until the end, they can be done more-or-less simultaneously.

First, figure out what the values you need to seed are. Devstack, aws, staging, prod, and edge can all have different values. Check all the envs/*.py files in edx-platform (run a git grep for your soon-to-be-removed setting variable), and have devops help you look for these values in configuration (and configuration-secure) as well.

Once you know what the values to be added are, set them. Either ask devops nicely, or get admin permissions for yourself using the permissions repo. Once you or your devops buddy has the requisite access, you can set the variables in your configuration model to match the effective settings variable for all environments. The fact that these values now match is what will allow for a seamless, no down time rollout of the remainder of your changes.

Step 3 - Move responsibilities to the new model

This step can be mostly done while step 2 is in-flight, but should not be deployed until step 2 is finished.

This is where you can begin looking around for where the previous settings variable was checked, and instead check the new config model. In my PR, the gatekeeping duties had previously been performed by this function https://github.com/edx/edx-platform/pull/12376/files#diff-990c8d66bbe04a524be83d8f73c1b321L60, but were moved to this one instead https://github.com/edx/edx-platform/pull/12376/files#diff-ca0c4cdbea57f8defaabc59512eba59aR271.

Note: if you have not provided any seed values, the check Flag.is_enabled() will return False, as outlined in this code: https://github.com/edx/django-config-models/blob/424edae2976ce8cc21ca545f3f76069bd4a00be7/config_models/models.py

Step 3.5 - Don't forget about tests!

If your quest to move to a config model is going to explode in scope, this is likely to be the reason why.

• Python tests - You'll need to create/delete model objects where you had previously patched settings.FEATURES. For an example, see https://github.com/edx/edx-platform/pull/12376/files#diff-1966eaf949ad6e452c5e28e161b9d8a4
  • (From ChristinaR (Deactivated)) Actually, the deletion of model objects in tearDown is not necessary. As long as you are extending the right base test cases, this is done for you. Make sure to extend CacheIsolationTestCase (or something that extends it).
• JS tests - I didn't run into any overlap with js tests. If you did, fill in this section afterwards. I can't imagine there being a lot of cases where js tests rely on python code very extensively though - if that;s the case, you may need to rethink your testing strategy.
• Lettuce tests - If you had a lettuce test that relied on settings variables, it will now cease to function. You'll need to move to a bok choy test instead (which is the direction we're moving anyways - way to fix that tech debt!)
• Bokchoy tests - if your settings variable had previously been in bokchoy.py, or if a common.py variable was access in bokchoy tests, you'll need to add a database fixture to add your new values when running these tests. In my case, that fixture was https://github.com/edx/edx-platform/pull/12376/files#diff-ec25386326e9984d0dec39f816799ba4. After writing and adding that seed value, you'll need to Update the bok-choy DB cache, which will touch a lot of files. Check them in, and be sure to flag testeng on your PR.

Step 4 - Merge/Release/Deploy

If you were smart and split this work across 2 release cycles, you'll have an easy time of this. After merging, just confirm that the expected functionality continues to work in stage, and your initial variable definitions are good. Be sure to add release notes detailing the change (for open-source followers who stay up with release), and make sure the openedx team knows to call this setting out in the next named release.

Step 5 - Delete now-unused variables

Now that your shiny new configuration model is handling things, find all occurrences of the previous settings variable and delete them, as they are no longer used. Be sure that they get removed in configuration and configuration-secure as well!

And that's it, you've done it. Congratulations! Your setting can now be changed live by an administrator, without the need for a full AMI deploy.

Note about scripting ConfigurationModel updates

If you have a need to serialize and/or script your ConfigurationModels (for example, to sync up the state of your devstack or sandbox with the ConfigurationModel values in production), you can make use of existing ConfigurationModel serializers/managements command and associated Ansible scripts. These are generic and should work for any ConfigurationModel.

See https://github.com/edx/edx-platform/pull/12726 and https://github.com/edx/configuration/pull/3195 for the location of these files.