Some Things About Manually Rolling Back Migrations

Overview

This page is a guide for SREs and engineers who are undoing strange things. Please update it.

Some steps in this guide assume you have access to the root user of a database

Workflows

Testing Locally

Before you do anything in Prod you should test in stage or devstack

Running Migrations via `manage.py` Manually

TBD

Note: Be careful about what version of the code you have since you can’t migrate back without the migration files (except via GoCD)

Note also: you’ll need access to edx-secure to run migrate. (One can run showmigrations as documented below without such permissions.) Passwords (e.g. the ones here) are necessary to have access to a database user with sufficient permissions.

Example Flow:

ssh <sre_user>@<ip_from_aws_console>
sudo su - www-data -s /bin/bash
cd /edx/app/<service>/
. venvs/<service>/bin/activate
. <service>_env
cd <service>/

(<service>) www-data@ip-10-3-117-24:/edx/app/ecommerce/ecommerce$ python manage.py showmigrations <model>
communication
 [X] 0001_initial

Dropping Rows from The `django_migrations` Table

Only do this if you’re sure you can’t roll back any other way and that the currently deployed code does not rely on the changes from the migration.
Check the table
```
SELECT * FROM django_migrations;
```

Pick your ID:

SELECT * FROM django_migrations WHERE id = <id>;

Check what the migration was doing
1. Shell in to a box
2. Run something like this:
```
python manage.py sqlmigrate catalog 0028 --backwards
python manage.py sqlmigrate catalog 0028
```
3. Look at what tables are being touched.
Look to see if anything is writing to the tables;
```
show processlist;
```

Count rows in the related tables
Example:

select count(*) from catalog_contentmetadatatoqueries;

Delete the record of the migration

DELETE * FROM django_migrations WHERE id = <id>;

Drop the table
```
DROP TABLE truncate catalog_contentmetadatatoqueries;
```
1. Note, you don’t usually need to truncate.
Check New Relic or Performance Insights to see that things look ok.
Profit