Some Things About Manually Rolling Back Migrations
- Adam Blackwell (Deactivated)
- Matt Hughes (Deactivated)
Overview
This page is a guide for SREs and engineers who are undoing strange things. Please update it.
Some steps in this guide assume you have access to the root user of a database
Workflows
Testing Locally
Before you do anything in Prod you should test in stage or devstack
Running Migrations via manage.py Manually
Note: Be careful about what version of the code you have since you can’t migrate back without the migration files (except via GoCD)
Note also: you’ll need access to edx-secure to run migrate. (One can run showmigrations as documented below without such permissions.) Passwords (e.g. the ones here) are necessary to have access to a database user with sufficient permissions.
TBD
Example Flow:
ssh <sre_user>@<ip_from_aws_console>
sudo su - www-data -s /bin/bash
cd /edx/app/<service>/
. venvs/<service>/bin/activate
. <service>_env
cd <service>/
(<service>) www-data@ip-10-3-117-24:/edx/app/ecommerce/ecommerce$ python manage.py showmigrations <model>
communication
[X] 0001_initial
Dropping Rows from The django_migrations Table
Only do this if you’re sure you can’t roll back any other way and that the currently deployed code does not rely on the changes from the migration.
Check the table
SELECT * FROM django_migrations;Pick your ID:
SELECT * FROM django_migrations WHERE id = <id>;Check what the migration was doing
Shell in to a box
Run something like this:
Look at what tables are being touched.
Look to see if anything is writing to the tables;
Count rows in the related tables
Example:Delete the record of the migration
Drop the table
Check New Relic or Performance Insights to see that things look ok.
Profit