...
Upgrading to 4.2 should be sufficient for the next Open edX release, since end of life has not been announced for it.
If we decide to pursue 4.4 in the future, tickets we created and cancelled for it can be found on this page: MongoDB 4.2 Upgrade Progress Tracking
Dependencies
Mongo upgrade can be done independently from other upgrade projects, e.g. Elasticsearch 7.10 , Django 3.2 Upgrade , etc. We have not identified any other dependencies.
...
One known risk is performance issues once we upgrade Prod, due to size of the DB there. Stage DB is much smaller than Prod DB, so scaling might be an issue which we would not be able to detect during review and testing in Sandbox or Stage.
Rollout and Contingency
...
Rollout Plan
Stage will be upgraded, then prod and edge
New instances on the new version are added to the cluster one at a time andthe data is synced
There is 30 seconds or less of downtime when the cluster cuts over to a primary node on the new version
No downtime notifications should be needed
Old version nodes are then removed from the cluster
Rollback plan
“Rollback” would be swapping back to older version as primary node if switching primary to new version causes problems.
Cluster will have nodes of two versions at the same time for a while and app servers will start using nodes running newer Mongo as soon as they complete syncing data
Rollback to old version can be done until we update the
FeatureCompatibilityVersion
variable in Mongo. After it goes forward we can’t go backward without dump and restore.
...
Though we need to complete this anytime before October 15, SRE teams wants to complete in August 2021 if possible.
Project Team
Name | Squad | Role |
---|---|---|
SRE | Service Owner | |
Open edX | Project Manager | |
SRE (Cambridge) | Lead SRE Engineer | |
SRE (Arbisoft) | Consulting SRE Engineer | |
TNL (owner of Mongo usage in edx-platform) | TNL Lead | |
TNL (owner of Mongo usage in edx-platform) | Review (edx-platform) | |
Infinity squad (IDA owner of cs-comments-service) | Review (cs-comments-service) | |
Open edX | Stakeholder (Infrastructure) | |
Open edX | Stakeholder (Open edX named release) | |
Open edX Community | Open edX Community | Stakeholder (Consumers of Maple named release) |
Communication Plan
Due to a small number of stakeholders we will mostly likely communicate ad hoc via Slack. Dedicated channel is #mongodb-4-4.
...