The 10th Open edX release will be named Juniper.

Put stuff here that we have to remember when we start packaging up for Juniper.  Especially important is information that system installers or operators will need to know. Please include your name when you add an item, so that we can get back to you with questions.

Juniper is gated by the upgrade to Django 2.2

As of April 27, it looks like all services will be upgraded and in production this week.  The master branches can be created soon after that point.  We are triaging issues (https://openedx.atlassian.net/browse/CRI-171) and will decide when to make the master branches based on that triaging.

A high-level upgrade guide is being collaboratively created here.

• HTTP Cookie Sharing

  • Studio redirects to the LMS for login and registration. (@Robert Raposa, @Nimisha Asthagiri (Deactivated)) 

    • The DISABLE_STUDIO_SSO_OVER_LMS setting used in Ironwood has been removed. See https://github.com/edx/edx-platform/pull/22416 for background on why this change was made.

    • Note: for community members who will have difficulty sharing cookies across Studio and LMS, see https://github.com/edx/edx-platform/pull/19845#issuecomment-559154256 for possible alternative solutions from @Felipe Montoya. These solutions can be discussed here: https://discuss.openedx.org/t/deprecation-removal-complete-removal-of-studio-signin-and-signup-pages-relates-to-depr-6/962
      
      

• Runbook Configuration

  • We are looking at doing large migrations for the courseware_studentmodule table to prevent running out of primary keys. These will likely be large, semi-manually done migrations on CSM and it's history table(s). Nothing is settled yet, but if we get this in before Juniper we will need to share our runbooks for anyone upgrading to it. (@Brian Mesick (Deactivated))
    
    

  • Part of the django upgrade includes a migration on the auth-user table to increase the size of the last_name column from 30 to 150 characters.  If your user table is above a million rows, the to make this update might be long enough to cause downtime.  edX.org has opted to skip this migration and has limited the length of the last name field to the previous 30 characters in the user change forms.  It's recommend that openedx do the same.

    • The relevant command to run before you run the rest of the migration is the following: 

      sudo -HEu www-data /edx/bin/python.edxapp /edx/bin/manage.edxapp lms migrate --fake auth 0009

      • Run this command after loading the new code but before running the migrations.

• Django Configuration and SetUp

  • Remote Configuration (@Cory Lee (Deactivated), @Bill DeRusha (Deactivated))

    • TBD - Need to provide description of the impact of this change.

    • lms/cms removal of aws.py https://openedx.atlassian.net/browse/DEPR-14 plugins should use production.py at this point.
      
      

  • JWT Cookies require JWK setup. This was disabled in Ironwood.  Thanks to @Pierre Mailhot for the following steps to generate and configure the JWT signing keys. (@Robert Raposa, @Nimisha Asthagiri (Deactivated))

    Alternatively, the following piece of code (created for Tutor) can be generated to manually generate the private/public signing keys.

  • Third Party Auth Organizations linking is no longer managed via ApplicationOrganization model.  This model will be removed and the linking will be done by explicitly defining filters for the application.  See relevant decisions:

    • https://github.com/edx/edx-platform/blob/master/openedx/core/djangoapps/oauth_dispatch/docs/decisions/0011-scope-filter-support.rst

    • https://github.com/edx/edx-platform/blob/master/openedx/core/djangoapps/oauth_dispatch/docs/decisions/0012-scope-and-filter-for-third-party-auth.rst

  • We upgraded sorl-thumbnail from a version that didn't have migrations to one that does, so the initial migration needs to be faked in any environment that previously installed the old version: ./manage.py lms migrate thumbnail --fake .

• Management Commands

  • As part of https://github.com/edx/edx-platform/pull/20698, send_verification_expiry_email command arguments, --resend-days and --days-range were removed from the management command and moved to lms common settings. (@Zainab Amir (Deactivated))
    
    

• Celery

  • The django-celery library has been removed from edx-celeryutils. The corresponding management command has now been deleted and should be removed from upgrade instructions. (@Tim McCormack, @Diana Huang)

• Inter-service Communication Impact
  
  

  • OAuth2 DOP→DOT Migration (@Nimisha Asthagiri (Deactivated), @Julia Eskew (Deactivated), @Troy Sankey, @Robert Raposa, @Diana Huang, @Jeremy Bowman (Deactivated))

    • https://openedx.atlassian.net/browse/DEPR-47

    • Any OAuth Applications/Clients registered in the LMS that are using the old Django-OAuth-Provider will need to be migrated to Django-OAuth-Toolkit.  These applications include your Open edX Mobile Apps (iOS, Android) as well as your Open edX services (e-commerce, Insights, Credentials, etc.) 

      • For updating your Open edx services, see Setup OAuth Client for Internal Services (Django Oauth Toolkit version)

        • Note: there exist management commands used in devstack, but edx.org did this manually.
          
          

      • For updating your mobile apps:

        • Create a new OAuth application at https://<lms>/admin/oauth2_provider/application/ for each of your mobile app types (iOS and Android).

          • Client_id: <autogenerated>
            User: select the "service user" you've created specifically for this mobile app type
            Redirect Uris: <blank>
            Client type: Public (differs from services above)
            Authorization grant type: Resource owner password-based (differs from services above)
            Client secret: <autogenerated>
            Name: a unique name you choose to identify this mobile app type

        • Update your mobile app configuration:

          • OAUTH_CLIENT_ID should be updated to the Client_id from the step above.

          • The oauth2/access_token URL defined in the code should already work as-is.

  • A new OAuth2 scope, 'user_id', is required for SSO access between the LMS and satellite Open edX services (e-commerce, Insights, Credentials, etc.)  For existing Open edX instances, please add the 'user_id' scope for each of your satellite services at /admin/oauth_dispatch/applicationaccess/.  You may use a management command to do so (see https://github.com/edx/edx-platform/pull/20076).  For newly provisioned instances, this scope will be added automatically with https://github.com/edx/devstack/pull/394. (@Robert Raposa)
    
    

  • Ecommerce expects to find a unique user id (LMS user id) for each user, either in a JWT or social auth. If it cannot find this id, a MissingLmsUserIdException will be raised; this exception can be silenced by enabling the allow_missing_lms_user_id waffle switch. The LMS user id can be back-filled for existing users by running the ecommerce/core/management/commands/import_user_ids.py management command. See ecommerce/docs/decisions/0004-unique-identifier-for-users.rst. (@crice (Deactivated))
    
    

  • In edx-platform/lms/djangoapps/courseware/courses.py - The function check_course_access has been renamed to check_course_access_with_redirect.  A new implementation of check_course_access returns AccessResponse objects, rather than performing redirects when access is denied.  check_course_access_with_redirect is a drop-in replacement for the old behavior of check_course_access.  This is likely a breaking change for any code outside of edx-platform (such as plugin apps) that was using the check_course_access function directly.  See https://github.com/edx/edx-platform/pull/23651 for more information.

• Mobile Apps

• Forum:

  • The MONGOID_AUTH_MECH environment variable needs to be defined, even if just empty. Otherwise the following error will be triggered: "in `encoded_name': undefined method `encode' for nil:NilClass(NoMethodError)".

• Database migration:

  • Due to a migration in "SiteConfiguration", it is not possible to have an Ironwood instance serving traffic while a Juniper one is being set up with zero downtime. It will result with OperationalError exceptions being raised until the Ironwood instance is shutdown. (Slack thread)