So, you have a cool feature flag or settings variable defined in envs/*.py. It works, it's available where you need it, and you can easily have different values for devstack, aws, bokchoy, whatever. Now, the time has come - you want to move this value in a configuration model, to allow admin panel access to change values without building a new AMI. Read on for details.
...
- Python tests - You'll need to create/delete model objects where you had previously patched settings.FEATURES. For an example, see https://github.com/edx/edx-platform/pull/12376/files#diff-1966eaf949ad6e452c5e28e161b9d8a4
- (From ChristinaR (Deactivated)) Actually, the deletion of model objects in tearDown is not necessary. As long as you are extending the right base test cases, this is done for you. Make sure to extend CacheIsolationTestCase (or something that extends it).
- JS tests - I didn't run into any overlap with js tests. If you did, fill in this section afterwards. I can't imagine there being a lot of cases where js tests rely on python code very extensively though - if that;s the case, you may need to rethink your testing strategy.
- Lettuce tests - If you had a lettuce test that relied on settings variables, it will now cease to function. You'll need to move to a bok choy test instead (which is the direction we're moving anyways - way to fix that tech debt!)
- Bokchoy tests - if your settings variable had previously been in bokchoy.py, or if a common.py variable was access in bokchoy tests, you'll need to add a database fixture to add your new values when running these tests. In my case, that fixture was https://github.com/edx/edx-platform/pull/12376/files#diff-ec25386326e9984d0dec39f816799ba4. After writing and adding that seed value, you'll need to Update the bok-choy DB cache, which will touch a lot of files. Check them in, and be sure to flag testeng on your PR.
...
Now that your shiny new configuration model is handling things, find all occurrences of the previous settings variable and delete them, as they are no longer used. Be sure that they get removed in configuration and configuration-secure as well!
And that's it, you've done it. Congratulations! Your setting can now be changed live by an administrator, without the need for a full AMI deploy.
Note about scripting ConfigurationModel updates
If you have a need to serialize and/or script your ConfigurationModels (for example, to sync up the state of your devstack or sandbox with the ConfigurationModel values in production), you can make use of existing ConfigurationModel serializers/managements command and associated Ansible scripts. These are generic and should work for any ConfigurationModel.
See https://github.com/edx/edx-platform/pull/12726 and https://github.com/edx/configuration/pull/3195 for the location of these files.