Links:

• BD-14 Library use cases implementation discovery

• BD-14 Content Libraries and Editing Project Overview 

• BD-14 Blockstore Powered Content Libraries Taxonomies 

The Most Important Thing

In the second half of 2023, the 2U>Teaching & Learning team (T&L) will finally be completing the multi-year BD-14 initiative by rolling out a new Content Library authoring experience (aka “V2”) to edx.org Studio Users.  Primarily, this means: rolling out the library-authoring MFE, and migrating Modulestore-backed “V1” libraries to Blockstore.  Our work will be the culmination of a long series of projects that will enable these features.

T&L will offer as much of this effort as possible back to the community in the following ways:

• Include the updated Library Authoring MFE with the Quince release which will work with V2 libraries, OFF by default

• Include a set of management commands with the Quince release which will provide a path to migrate existing Content Libraries from modulestore to blockstore.

T&L will leverage previously created mockups for the Library Authoring MFE UI.

Key Requirements

1. Replace legacy Studio Content Library UI with performant, modern React-based MFEs.

2. Maintain continuity of Content Library feature offerings and experience.

3. Introduce the ability to statically reference library content in courses.

4. Move current content libraries data from course-oriented Modulestore to block-oriented Blockstore.

High Level Design

Three dominant threads are guiding this project, which are detailed below.

Integrate Existing Solutions

This is not a project which offers novel architectural solutions. Past iterations of BD-14 have: created the library authoring MFE, created v2 blockstore-powered libraries, and moved blockstore to be a python plugin instead of an independently deployed service. In each of these cases, the initial implementations do not fully provide the functionality and features we need. These deficiencies and their resolutions include:

• The library Authoring MFE needs to be updated to use new Xblock editors, and other ux/ui improvements

• No path exists for converting V1 content libraries to V2 libraries. We will create a series of management commands and runbooks to do that. Those operations are roughly: copy V1 libraries into V2 Libraries, update all references to V1 libraries in courses to be the corresponding V2 library, then delete all the V1 Libraries once we are confident enough to do so.

• We need to reconfigure our infrastructure to handle blockstore as a python library. We will have no need for the service anymore, but will need to keep the database infrastructure.

Maintain Parity and Seamlessness

We want to maintain continuity of Studio feature offerings and experience. The existing breadth, flexibility, and reliability of Studio is a draw for content authors. We will have to implement a few frontend features (like xblock editing in V2 libraries) to do so. We are hoping to perform a seamless conversion of content. To do so, we will craft runbooks and exercise them in increasingly complex environments.

To prioritize release of the high-value static content reference feature, we will implement parity for the following features after the initial release:

• Import and export of Blockstore-backed content libraries.

• Text templates in the library-authoring experience.

• Filtering content library problems by type when referencing them in courses.

Get Ready to slash-n-burn!

This project will deprecate a number of features and areas of code. The first is Blockstore’s capability to run as a separate service. The second, and largest task, will be to enable the removal of all of the V1 content libraries code.

In addition, we will be removing the library_source block, an initial attempt to add specific library reference. Instead, we will be offering one library block, the library content block, which will allow for both randomized and specific content.

Design Diagram

Open

This is a high level diagram of the feature implementation. Note that Library content is consumed in courses by xblocks, which contain a reference to that library content in their metadata. Learning contexts can then use that reference to “pull” in the required content from the library. Someday, courses might also store their metadata and content in a fashion similar to blockstore, as it is more performant. It is more performant because V2 xblocks better differentiate their content from metadata, and therefore can operate more efficiently. In addition, S3 provides an elegant solution for object storage, which opens the door for better static asset management and flexibility in learning context structure.

Out of Scope Items

This project will not include the introduction of any new features that Blockstore enables.  This would cause the scope of the project to balloon and put the rollout success of an MFE at risk due to adoption of unverified capabilities.

This project will not build support for a temporary “v1+v2 hybrid” experience.  There will instead be a defined cutover event after extensive internal and beta testing cycles.

High Level Work Plan

T&L is splitting this work into two streams, since parallelization is possible.  Dave Ormsbee and Kyle McCormick from Axim will also contribute to the project with code and reviews, focusing mostly on the backend work. Braden MacDonald from openCraft will also contribute to the project with technical/domain guidance.

Rollout Strategy

The rollout of Library Authoring v2 will be atomic in nature (aka flipping the waffleflag for all users), and will require a scheduled Studio downtime window to ensure the migration of Content Library data is comprehensive.  The rollout date will be heavily communicated with edx.org content authors through all available communication channels.  

Prior to rollout, T&L will run an extended beta test period where a beta test group will use a sandbox to provide feedback about the new experience.  

Rollback will be possible both during and for a short time after the scheduled downtime window begins.  Since we will not be changing the v1 libraries or the legacy UI at all, a rollback of the Course references will be needed for which we will have a management command prepared in advance.  The longer the v2 experience stays rolled out, the more drift between the v1 and v2 data.  This drift will quickly remove a rollback option completely.

Risks

Some risks include:

• The migration event might run into trouble.  There is a lot of data creation and manipulation to be done on MongoDB assets.  If any of the steps runs into a clerical, technical, or runtime error the timeline and investment scope will be at risk.

• Studio users may desire a rollback due to difficulties with the new experience, and data drift may make that impossible.

Measurements

1. Pertinent Studio Page usage metrics, e.g. “How many users visited Library pages today?  Where are they located?  How long was their session?”

2. The most pertinent performance metric is likely User-centric page load times from New Relic.  Our goal will be to maintain a consistent experience to the current legacy Studio baseline.

3. “Happiness”-style qualitative user feedback obtained via beta testing, interviews, and surveys.  If our power users are unhappy with the experience, T&L may choose to delay or partially roll out these changes.

Community Considerations

This work will be done primarily in an openedx repo, and we will heavily document the steps that our instance follows to convert to Library Authoring Experience v2 in the openedx wiki.  T&L expects these will become available to community members starting with Quince.  T&L will need to update both developer and user documentation as part of this project.

The openedx\frontend-app-library-authoring repo is already officially maintained by T&L.

Community submitted issue #68 will be resolved with this project.