Page Comparison

...

Because we operate in an environment where new and old code are running simultaneously against the same database, new code must always be compatible with the older database schema. Newer deployments can add tables and columns, but neither can be deleted unless the old code is no longer referencing the deleted tables or columns.

How to drop a column

Nullable/Non-Nullable Columns

...

Migration Unit Test

In the edx-platform codebase there is a unit test test_migrations_are_in_sync in test_db.py which ensures that django migrations and models are in sync. Migrations to drop columns or tables generally require at least two releases, one which removes references and one which has the drop migration. The first release will fail the unit test. For this reason you will need to skip that unit test during your release sequence and restore it when you are done. This also applies to libraries used by edx-platform, such as edx-proctoring, the test will fail when edx-platform receives the interim version.

The skip should include a ticket number and brief info on what it's for:

    @unittest.skip(
        "Temporary skip for TICKET-1234 while the fnord column is removed from the snood table"
    )

How to drop a column

Nullable/Non-Nullable Columns

For either a nullable or non-nullable column, first make sure there are no other models or code that actually use the column. If there are, make adjustments to those first before working through the deletion flow.

...

1. Remove all usages of the column, including updating the model to not refer to the field/column anymore (i.e. Model field must be removed in this step)Drop the column (with a migration)in this step)
   1. If this change is in the edx-platform codebase, add a skip to the test_migrations_are_in_sync unit test.
2. Drop the column (with a migration).
   1. If this change is in the edx-platform codebase, remove the skip to the test_migrations_are_in_sync unit test.

For NOT-NULLABLE columns, this involves THREE releases:

1. Update the model and generate a migration making the column nullable (`null=True`)
2. Remove all usages of the column, including updating the model to not refer to the field/column anymore (i.e. Model field must be removed in this step)
   1. If this change is in the edx-platform codebase, add a skip to the test_migrations_are_in_sync unit test.
3. Drop the column (with a migration).
   1. If this change is in the edx-platform codebase, remove the skip to the test_migrations_are_in_sync unit test.

Returning to our example with the auth_user table. If we still want to drop the full_name column, we should do the following:

1. Remove every usage of the full_name column in our codebase. Skip the unit test. Release that change to production, and ensure older code is no longer running. (We once had a stale ASG in production a few hours after a release, and it caused a few issues when we dropped a column.)
2. Create a database migration to drop the column. Restore the unit test. Release it.
3. (This step intentionally left bank...because nothing broke in production!)

...

When using AWS Aurora, a nullable column can be added to existing large (100k rows+?) tables without causing downtime. However, the migration may still timeout in GoCD - so please coordinate the release with DevOps.

1. Make a new field addition in model with null=True.
2. Generate the model-change migrations locally.
3. Create a pull request containing the model change and newly-generated migrations.
4. Merge the migration pull request.
5. The release process will run the migration and add the nullable column to table.

...

On AWS Aurora, indexes can be build on large tables without causing downtime, but this requires devops coordination as the migration may timeout in GoCD.

1. Add the new index on model fields.
2. Generate the model-change migrations locally.
3. Make pull request containing the model change and newly-generated migrations.
4. Merge the migration pull request.
5. Release will run the migration and add the index to table.

...