Page Comparison

...

1. Release:
   • Add the new field to the model.
     • If the old field has null=False, blank=False, and no default:
       • If the model is used in forms (django admin, or other forums):
         • Create the new field with null=True, editable=False.
         • disabling editable removes the field from
       • else:
         • Create the new field with null=True.
     • else if the old field is a BooleanField:
       • You might need to change the old field type to NullableBooleanField so that unit tests in release 2 will be happy when the old field is removed from code but not sqlite3.
       • Create the new field with BooleanField and the same signature, assuming there's a default set.
     • else if the old field has null=true:
       • Create the new field with the same field signature as the old.
   • Update any place where there are creates or updates on the field
     • Write the same value into both fields
     • If there is a Django admin page or other form and it is used regularly to create/update rows:
       • Register a signal handler to the model to update the new field whenever the old field changes or a new row is created.
2. Release:
   • Create a data migration to copy the values from the old field into the new field.
     • If the table is large, consider disabling atomicity and batching the copy.
   • Remove all references to the old field in the code.
     • Including removing the old field from the model in the code.
     • If this change is in the edx-platform codebase, add a skip to the test_migrations_are_in_sync unit test.
     • DO NOT include the migration for removing the old column (yet).
   • If you create the new field with a different field signature than the old, then update it now to be the same as the old.
     • e.g. change null back to False and editable back to True (the default).
       • CAUTION: Changing to null=False will cause a table rebuild during the ALTER. When performing this migration on a table with a large number of rows, degraded performance/downtime will likely result.
       • See 158766629 below.
     • include the migration that goes with this, but NOT the migration to remove the old field.
3. Release:
   • Run makemigrations, this should pick up the field removal from the previous stage.
   • If this change is in the edx-platform codebase, remove the skip to the test_migrations_are_in_sync unit test.

How to drop a table

Check in with Data Engineering/Data Science & Analytics to understand the downstream impacts of removing a table. (TBA: more detail on the process of doing this check-in.)

TWO releases:

1. Remove all references to the table by removing references to the model and the model itself
   1. If this change is in the edx-platform codebase, add a skip to the test_migrations_are_in_sync unit test.
2. Remove the table with a migration
   1. Remove the skip if you added one

Once a table is removed, a Data Engineering ticket should be created to clean up the table from downstream consumers of data.

How to delete a Django app containing tables

...

When using AWS Aurora, a nullable column can be added to existing large (100k rows+?) tables without causing downtime. However, the migration may still timeout in GoCD - so please coordinate the release with DevOps.

1. Make a new field addition in model with null=True.
2. Generate the model-change migrations locally.
3. Create a pull request containing the model change and newly-generated migrations.
4. Merge the migration pull request.
5. The release process will run the migration and add the nullable column to table.

...

On AWS Aurora, indexes can be build on large tables without causing downtime, but this requires devops coordination as the migration may timeout in GoCD.

1. Add the new index on model fields.
2. Generate the model-change migrations locally.
3. Make pull request containing the model change and newly-generated migrations.
4. Merge the migration pull request.
5. Release will run the migration and add the index to table.

...