Global roll-out of database-backed Organizations

Context

Open edX supports the concept of an “Organization” – that is, a university, business, institution, or other group that publishes content to edx.org, edge.edx.org, or any other instance.

Open edX has always had nominal level of support for tying content to Organizations. The key of every course run, content library, or block usage contains an “org slug” (i.e. organization identifier). For example, in these example keys, one can parse out the organization’s identifier as myOrg, or edX, or State_Univ, respectively.

  • myOrg/myCourse/myRun (version 0 course keys)

  • course-v1:edX+Example+2017 (version 1 course keys)

  • library-v1:State_Univ+awesome-library (version 1 library keys)

This scheme is great, but on its own its utility is limited. Since Organizations were not defined in any central location, there was no way to:

  • validate the organization identifiers (what if someone enters eedX by accident?),

  • efficiently get a list of all existing organizations (gotta check the org of every course!),

  • centralize the full names/logos of organizations (should certificates say “State Univ.”, “State U”, or “State University”?),

  • link users to organizations in the database (no table to link to).

So around 2015, database tables representing organizations and organization-course relationships were added to LMS and Studio on edx.org and stage.edx.org. Rules were added to Studio that prevented course runs from being created it their org slug did not map to an Organization in the database.

However, since its inception, the database-backed Organization feature has always been optional in Open edX. Notably, edge.edx.org, which operates as a separate Open edX instance, does not enable database-backed Organizations. This creates complexity when developing new features, as they must be written to handle both Organization-aware and Organization-unaware version of Open edX.

Decision

In 2020, while working on building the next generation of content libraries, we decided it would be best to enable database-backed Organizations on all Open edX environments, include edge.edx.org.

However, we decided not to universally enable a certain aspect of database-backed organizations on all environments: strict validation of the organization identifiers of new course runs. This validation will continue to occur on stage.edx.org and edx.org (and will be expanded to include validation for content libraries), but it will not be turned on for edge.edx.org or the default Open edX installation.

The technical details of the decision are described in this decision record.

Consequences

For edx.org , stage.edx.org, and edge.edx.org, the following changes took effect between November 2020 and January 2021. We aimed to make the changes as non-disruptive as possible.

  • edx.org and stage.edx.org:

    • Content libraries must be created with a valid Organization identifier that maps to an Organization saved in the database. (This has been required on edx.org for course runs since ~2016).

    • We checked for any course runs or content libraries currently in the system that lacked a link to a valid a Organization. We created inactive Organization and Organization-Course linkages to account for these. In total, ~2000 inactive organizations were automatically created, mostly due to content libraries with invalid organization slugs.

  • edge.edx.org:

    • The organizations database tables (link restricted to organizations admins) were automatically populated based on the org slugs of all course runs and content libraries in the system.

    • Newly-created course runs and content libraries will automatically link be linked in the database to Organizations. When there is no Organization to link to, one will automatically be created.

    • At this time, these changes will not be visible to anyone access to the organizations database.

  • All environments:

    • Product managers and developers can now assume Organizations that can be queried and linked to in MySQL database.

    • Developers can now assume that each Organization in the LMS/Studio database has a unique short_name (i.e. key/slug/identifier).

For other Open edX instances, these changes will be part of the Lilac release (Spring 2020). Migration instructions for operators will be included in the release notes.