Upgrade base OS from Ubuntu 16.04 to 20.04
This article describes the upgrade path and expectations for IDAs moving from ubuntu 16.04 LTS to Ubuntu 20.04 LTS. We will be skipping over 18.04.
This page is public, but some of the details here pertain only to edX deployments, and many of the links are to protected pages.
External Links
Platform, Blockstore and Designer Portal use Aurora clusters in stage or production so they will not be able to be upgraded to Ubuntu 20.04 until the databases are on MySQL 5.7. Ubuntu 20.04 requires TLS 1.2 system wide which isn’t supported for 5.6. You’ll see SSL errors for your database connections if you try.
For regular MySQL RDS, TLS 1.2 is supported for MySQL 5.6.46 and later
Instructions
Upgrading your testing environment
Sandboxes
Update the base AMI on your sandbox build to Ubuntu_20.04
Run the sandbox build for your application, fix bugs, repeat.
Travis CI or CircleCI
This might require you to update the devstack docker container your app runs in. If your app runs travis integration tests in a docker container pulled from the edxops org, you may have to do step 5 (below) first.
Update the
dist:
field in your.travis.yml
file tofocal
Identifying dependency changes
Ansible system packages
Look at the package list passed into the
edx_django_service_debian_pkgs_extra
parameter for your ansible role. This will be in themeta/main.yaml
file, and will probably reference a variable set indefaults/main.yaml
Look at
tasks/main.yaml
and look for calls toapt
.Look up the differences between an 16.04 sandbox vs. a 20.04 sandbox (created above) by running
dpkg -l | grep packagename
on each machine type.on the 20.04 server, run
apt-get changelog packagename
to see the changelog. read through the changelog to determine if any changes need to be accounted for in your use of that dependency. Ask SRE if you need specific advice.
Upgrading Devstack
Find your app in https://github.com/edx/configuration/tree/master/docker/build
change the line
From edxops/xenial-common
toFrom edxops/focal-common
in theDockerfile
run the jenkins job that rebuilds the base image. (https://tools-edx-jenkins.edx.org/job/DockerCI/job/image-builders/)
Upgrading your deployment (this may be a separate ticket)
Make sure all Jenkins jobs that hit this service support the correct version of python and Ubuntu
GoCD Base AMI
Find your IDA’s staging environment config in https://github.com/edx/gomatic-internal/tree/master/gocd/vars/tools/environment-deployment-play
Change the value for
base_ami_ubuntu_version
to20.04
Your pipelines should reseed after your changes are merged, if you have issues ask SRE to reseed your pipelines manually.
Significant Changes since 16.04
Python 2 is no longer in the base image. It will be installed by the ansible bootstrap method, but it may not be identical to what you had on 16.04.
NTPd is no longer used, and instead timesyncd is used.
Timeline
Service | Owner | Current Ubuntu Version | In Open edX? | Ubuntu 20.04 DevStack Upgrade due November 9th | Ubuntu 20.04 Travis Upgrade due November 9th | Ubuntu 20.04 Stage Upgrade Dependent on Aurora upgrade Due January 15 | Ubuntu 20.04 Prod Upgrade Dependent on Aurora upgrade Due January 15 | Ubuntu 20.04 Edge Upgrade Due January 15 |
---|---|---|---|---|---|---|---|---|
Platform (LMS + CMS) | Arbi-BOM - @Jeremy Bowman (Deactivated) | 20.04 | Yes | No Travis Docker Tests | (dedicated aurora) | (dedicated aurora) https://openedx.atlassian.net/browse/BOM-2068 BLOCKED | ||
eCommerce & Workers | Engagement - @Shelby Quinn (Deactivated) / @Seth McCann (Deactivated) Dev - @Emma Green (Deactivated) | 20.04 | Yes |
| ||||
Aperture - @Matt Tuchfarber (Deactivated) | 20.04 | Yes |
|
| ||||
Aperture - | 20.04 | No | Upgraded as of October 9 |