Juniper Upgrade instructions

We list here specific points that should be paid attention to when upgrading an Open edX platform from Ironwood to Juniper. This is not meant as a step-by-step manual, but rather as a high-level guide for operators who already know how to operate an Open edX platform.

Python upgrade

With the new Juniper release, the platform is upgraded to Python 3 and Django 2.2 and Python 2.7 support is completely removed. Perform a triage of all xblocks currently used by your instance and check for compatibility with Python 3 and Django 2.2. If they don't already support python 3 and django 2.2, an upgrade is recommended. For example, xblock-vimeo xblock upgrade.

Upgrade of custom branches

edx-platform

If using a custom edx-platform branch for your deployment, identify the code drift with the corresponding upstream branch. Ensure that these are compatible with python 3 and django 2.2 to add them to your new custom branch that is based on juniper.

configuration repository

There have been some changes made in the juniper release of the configuration repo (e.g. https://github.com/edx/configuration/pull/5691 https://github.com/edx/edx-platform/pull/22416 ). If you are overriding the configuration variables for your deployment, check if the variables being passed to the ansible playbooks are changed in the juniper release and update accordingly.

JWK (authentication) setup

The JWT Cookies require JWK setup. This was disabled in Ironwood. The generate_jwt_signing_key LMS management command can be used to generate these keys.

Mongodb upgrade

Mongodb needs to be upgraded from 3.2 to 3.6. To do so, you should first upgrade to 3.4, and then to 3.6. Here are the official instructions for a standalone node:

• 3.2 → 3.4: https://docs.mongodb.com/manual/release-notes/3.4-upgrade-standalone/

• 3.4 → 3.6 https://docs.mongodb.com/manual/release-notes/3.6-upgrade-standalone/

In a nutshell, here are the necessary steps:

1. Upgrade to 3.4

2. Set the feature compatibility version: mongo --eval 'db.adminCommand({ setFeatureCompatibilityVersion: "3.4" })'

3. Upgrade to 3.6

4. Set the feature compatibility version: mongo --eval 'db.adminCommand({ setFeatureCompatibilityVersion: "3.6" })'

Juniper defaults to SCRAM for the authentication mechanism. if a different authentication mechanism is being used, it can be specified for forum, the only affected service, by using the FORUM_MONGO_AUTH_MECH configuration variable. https://github.com/edx/configuration/pull/5755  

MySQL upgrade

Check the versions of databases and other services used by your instance for Juniper supported versions (e.g. MySQL-5.7, mongo-3.6). If using an older version, upgrade is recommended.

Devstack

edX devstack now supports running multiple releases. In order to use this on your devstack, pull the latest master and simply replace the container reference commands like:

with calls to docker-compose:

Cached Files

With juniper upgrade the cached files generated are not consistent, leading to 404 errors when multiple hosts are used for deployment.

• Theme asset compilation:
  CachedFilesMixin is deprecated in Django 2.2. CachedFilesMixin is mainly used during theme assets compilation. It needs to be replaced by ManifestFilesMixin. If not, then it results in static files with different hashes on each build. See for more details.

• Webpack build:
  Same behaviour can be seen during the webpack build process i.e. the cached files are generated with different hashes on each build, resulting in 404 errors while loading course assets on studio and/or LMS. A quick fix can be to use MD5 hash plugin like in https://github.com/open-craft/edx-platform/pull/268. But a better solution is to use Webpack's Content Hash and upgrade webpack to v4.44.2. Efforts are being made towards upstreaming the fix.

Misc python modules upgrade

With the upgrade to python3, some of the python modules were upgraded to newer versions that can cause a few issues. For example, python-swiftclient caused a duplicate Content-Length header to be inserted and unicodecsv expected a bytestream instead of unicode. These issues are fixed as and when they are discovered.

Themes upgrade

If you are using custom themeing for your instance, check if any stying changes made for juniper are needed and update your custom theme accordingly.

You might need to get the latest templates for some pages and apply your changes on top.

Single Server Install

Mostly using the steps in upgrading from the ironwood release. The first 5 steps assume that MongoDB is running on a separate instance and you are using a RDS Instance for MySQL. If you are running everything locally, skip to step 5 (and ignore 6.b).

1. Create mongodb instance.

2. From old open edx instance, use mongodump to backup and then mongorestore into new mongodb instance.

3. On the new mongodb instance, create edxapp and cs_comments_service users in the respective databases.

4. Create new RDS instance from a snapshot of your ironwood instance.

5. Create new EC2 instance for your open edx instance.

6. Copy config.yml

   1. Set the following variables to False: ecommerce_create_demo_data, credentials_create_demo_data, DEMO_ROLE_ENABLED

   2. (Optional) Add variables MYSQL_HOST and MONGO_HOSTS and use them as values for EDXAPP_MYSQL_HOST, EDXAPP_MONGO_HOSTS and any other service you want installed (DISCOVERY_MYSQL, DISCOVERY_MYSQL_REPLICA_HOST, ECOMMERCE_DATABASE_HOST, FORUM_MONGO_HOSTS…)

7. Copy my-passwords.yml

8. Run install script without the password creation - example script