Motivation:
Once the RC is on stage, hotfixes become pretty much prohibitively expensive, since stage has to be replaced with the hotfix code, interrupting the verification workflow. Since the RC is usually on stage much longer than the 24 hours the release schedule allows for it, it makes the window for hotfixes uncomfortably narrow.
Proposal:
Devops mentioned that it would be possible to set up a second staging environment. I do not know what kind of work is needed to set that up. Assuming that is possible, here is a proposal for doing hotfixes when the RC is on stage:
- Make a hotfix branch off of the current release branch (not the rc)
- Deploy this branch to a second set of staging servers and verify the hotfix
- Deploy that hash to production
- Merge the hotfix branch into release
- Merge the release branch into the release candidate
- Re-deploy the RC to the first set of stage servers
- Confirm the fix is on the RC