Juniper

The 10th Open edX release will be named Juniper.

Put stuff here that we have to remember when we start packaging up for Juniper.  Especially important is information that system installers or operators will need to know. Please include your name when you add an item, so that we can get back to you with questions.

Timeline

Juniper is gated by the upgrade to Django 2.2

As of April 27, it looks like all services will be upgraded and in production this week.  The master branches can be created soon after that point.  We are triaging issues ( ) and will decide when to make the master branches based on that triaging.

Upgrading from Ironwood

A high-level upgrade guide is being collaboratively created here.

Installation and Operations


  • Inter-service Communication Impact

    • OAuth2 DOP→DOT Migration (Nimisha AsthagiriJulia EskewTroy Sankey, Robert RaposaDiana HuangJeremy Bowman)
      • Any OAuth Applications/Clients registered in the LMS that are using the old Django-OAuth-Provider will need to be migrated to Django-OAuth-Toolkit.  These applications include your Open edX Mobile Apps (iOS, Android) as well as your Open edX services (e-commerce, Insights, Credentials, etc.) 
        • For updating your Open edx services, see Setup OAuth Client for Internal Services (Django Oauth Toolkit version)
          • Note: there exist management commands used in devstack, but edx.org did this manually.

        • For updating your mobile apps:
          • Create a new OAuth application at https://<lms>/admin/oauth2_provider/application/ for each of your mobile app types (iOS and Android).
            • Client_id: <autogenerated>
              User: select the "service user" you've created specifically for this mobile app type
              Redirect Uris: <blank>
              Client type: Public (differs from services above)
              Authorization grant type: Resource owner password-based (differs from services above)
              Client secret: <autogenerated>
              Name: a unique name you choose to identify this mobile app type
          • Update your mobile app configuration:
            • OAUTH_CLIENT_ID should be updated to the Client_id from the step above.
            • The oauth2/access_token URL defined in the code should already work as-is.
    • A new OAuth2 scope, 'user_id', is required for SSO access between the LMS and satellite Open edX services (e-commerce, Insights, Credentials, etc.)  For existing Open edX instances, please add the 'user_id' scope for each of your satellite services at /admin/oauth_dispatch/applicationaccess/.  You may use a management command to do so (see https://github.com/edx/edx-platform/pull/20076).  For newly provisioned instances, this scope will be added automatically with https://github.com/edx/devstack/pull/394. (Robert Raposa)

    • Ecommerce expects to find a unique user id (LMS user id) for each user, either in a JWT or social auth. If it cannot find this id, a MissingLmsUserIdException will be raised; this exception can be silenced by enabling the allow_missing_lms_user_id waffle switch. The LMS user id can be back-filled for existing users by running the ecommerce/core/management/commands/import_user_ids.py management command. See ecommerce/docs/decisions/0004-unique-identifier-for-users.rst. (crice)

    • In edx-platform/lms/djangoapps/courseware/courses.py - The function check_course_access has been renamed to check_course_access_with_redirect.  A new implementation of check_course_access returns AccessResponse objects, rather than performing redirects when access is denied.  check_course_access_with_redirect is a drop-in replacement for the old behavior of check_course_access.  This is likely a breaking change for any code outside of edx-platform (such as plugin apps) that was using the check_course_access function directly.  See https://github.com/edx/edx-platform/pull/23651 for more information.
  • Mobile Apps

  • Forum:
    • The MONGOID_AUTH_MECH environment variable needs to be defined, even if just empty. Otherwise the following error will be triggered: "in `encoded_name': undefined method `encode' for nil:NilClass(NoMethodError)".
  • Database migration:
    • Due to a migration in "SiteConfiguration", it is not possible to have an Ironwood instance serving traffic while a Juniper one is being set up with zero downtime. It will result with OperationalError exceptions being raised until the Ironwood instance is shutdown. (Slack thread)

Impact on Your Code

  • edx-platform
    • The courseware Django app is now only imported as "lms.djangoapps.courseware".  If you import it as "courseware", you may get errors.  Fix the imports.


    • Comprehensive theming is now disabled by default. To enable it, make sure to add ENABLE_COMPREHENSIVE_THEMING = True in the edx-platform settings of both the LMS and the CMS.
  • Frontend
    • Paragon enhancements and edx-bootstrap (Adam Butterworth)
      • The Paragon reusable component and pattern library has seen significant investment and refinement since the last Open edX release. A number of components have been re-engineered for ease of use and flexibility. We’ve also launched an interactive documentation site (https://edx.github.io/paragon) where you can explore the components, their implementation status, as well as the library’s foundational CSS utility classes.
    • frontend-platform (David Joy)
      • The Frontend Platform is a framework for Open edX micro-frontend applications. It provides a number of common services used by frontend apps, including analytics (Segment), logging (New Relic), authentication and HTTP/AJAX request client (Axios/JWT), and internationalization (React Intl). It also includes a number of helpful React components and utilities.  The framework is highly configurable and extensible, allowing override implementations of analytics and logging, as well as a number of customization hooks for the application. It should be used as a basis for new micro-frontends. A detailed documentation site can be found here: https://edx.github.io/frontend-platform

    • Frontend Repos

    • We have created a new Microfrontend named Publisher (https://github.com/edx/frontend-app-publisher/) for creating courses and course runs. Although primarily used to keep track of course metadata (such as marketing information), Publisher can also help manage ecommerce products. Publisher integrates between edx-platform, ecommerce, and course-discovery. (Dillon Dumesnil)
      • Note: At this point in time, there is no standard process for installing Microfrontends, but Publisher is still being provided so the community can become more familiar with it and possibly configure and install it on their own.
  • Upgrades
    • Python-3: OEP-7Python 3 How-To Note: Ubuntu 16.04 (and thus the ansible playbooks) ships with Python 3.5.1, but Python 3.5.9 was found to work in Tutor, and it works "better").
    • Django-2: Django 2.2 Upgrade Plan
    • Node-12
    • Ruby was upgraded to v2.5.7 in cs_comments_service (forum). Bundler v1.11.2 and Rake v10.4.2 versions remain identical in the configuration repo, but I (regisb) am completely unable to get them to work in Tutor. I had to upgrade to Bundler v1.17.3 and Rake v13.0.1.
    • MongoDb: upgraded from 3.2.16 to 3.6.17
    • Pip: upgraded from 19.3.1 to 20.0.2
    • Setuptools: upgraded from 39.0.1 to 44.1.0
  • DEPRs (see below)

  • Relevant ADRs

Deprecations and Removals

Based on Juniper Dates: 2019-01-17 to 2020-05-27

Deprecations

Removals

    • Removal of microsites includes the removal of several tables that were in this djangoapp: https://github.com/edx/edx-platform/pull/21355/
    • We have removed the entire djangoapp, but if installations would like to clean up these tables after upgading, they can drop all the tables prefixed by microsite_configuration.
    • Removed now unused /login_post endpoint, which was just used by Studio login.  Note: /login_ajax points to the same login code.  (was ARCH-1253)
    • Several breaking changes were made to clean-up login_user, used by our login POST endpoints.
      • JSON is now always returned, including for the third-party auth errors.
      • 400 status is now used for errors instead of 200. JSON included 'success': 'failure' both before and after the change.
  •  : https://github.com/edx/edx-platform/pull/22006 (Bill DeRusha (Deactivated))
  • edx-certificates is no longer installed. This was an error.  edx-certificates is still part of Juniper.

Feature Changes

  • Will Public Course Content be on by default?

  • Learner Portal. learner-portal (https://github.com/edx/frontend-app-learner-portal) and it's accompanying configuration service (https://github.com/edx/portal-designer) were created

  • SAML. Irfan Ahmad: As part of https://github.com/edx/edx-platform/pull/20665 we have updated the username match criterion for SAML SSO users to be based on email only instead of both email & username.

  • e-Commerce
    • As part of https://github.com/edx/ecommerce/pull/2621, the payment page microfrontend is the default payment page for edx ecommerce. To use your own microfrontend, you need to add enable_microfrontend_for_basket_page to your site configuration as well as payment_microfrontend_url . If you do not use the microfrontend, you can use the waffle flag disable_microfrontend_for_basket_page to see the old basket page for testing, etc. (Brandon Baker (Deactivated))
    • Performance improvements, etc. (Emma Green)
      • TBD - Need to provide a description of feature changes in ecommerce since April 2019 as it impacts the community. (Perhaps links to ADRs.)

  • LMS
    • Learner Dashboard. Adeel Ehsan: As part of https://github.com/edx/edx-platform/pull/20439 by default LMS dashboard will show only limited courses based on the course limit set in configuration. Show all the courses link will be at bottom if enrolled courses count is greater than course limit.

    • Instructor Dashboard
    • Gradebook
      • TBD - Need to provide a description of changes in gradebook since April 2019 as it impacts the community.

    • Proctoring
      • TBD - Need to provide a description of changes in proctoring since April 2019 as it impacts the community.

    • ORA-2
      • TBD - Need to provide a description of changes in ORA-2 since April 2019 as it impacts the community.
         
    • Course Dates
      • TBD - Need to provide a description of changes to course dates since April 2019 as it impacts the community.

    • SGA xBlock
      • Python3 and Django 2.2 compatibility added in version 0.11.0. This version is not backward compatible with Python2 and Ironwood.