Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  1. Store library content in Blockstore. This has been a goal since the outset of the project – it’s in the original specification of it. Just as LabXChange does today, we will store the master copy of v2 library content in Blockstore.

    • Why Blockstore? The decision to move away from Modulestore and towards Blockstore has been thoroughly hashed out at several levels, and Blockstore Design shares some of the justification. LabXChange is currently successfully proving out Blockstore’s utility in a production environment. Happy to add more details here if helpful.

  2. Do NOT store library content in Modulestore. This may seem obvious given the first goal, but it’s not. Since courses are currently stored in Modulestore, we need some way of referencing library-based content from the context of Modulestore-backed content. One way of doing this would be to copy all course-referenced library content from Blockstore into Modulestore. While technically an option, we would like to avoid doing this, because (i) it is hugely ineffecient space-wise, and (ii) Modulestore is fraught with complexity and performance issues, which we are trying to avoid coupling BD-14 as much as possible.

  3. DO store library content customizations with the course in Modulestore. Although we want library content to live firmly in Blockstore, we still want to any customizations that course authors make to referenced content to live within the course structure/definition. This helps us establish and maintain a distinction between (a) original library content and (b) course-local tweaks that authors may make, a distinction which we believe is important for both technical and user-facing coherence.

    • An intentional implication of this is that library content customizations get exported along with course content, whereas the original library content does not. Furthermore, re-runs of a course will copy any library content customizations into the new course run, with further customization of library content in the original run not affecting the new run, and vice versa. These behaviors mirror the intended behavior of v1 libraries, but the v2 implementation should achieve them with more clarity and fewer edge-cases.

  4. Build new frontend features using the micro-frontend (MFE) framework. This is also a stated goal of the original BD-14 pitch. Wherever feasible, we will build all new frontend features using edX’s React-based micro-frontend framework, as opposed to building within the legacy Django-templated frontend upon which most of Studio is currently implemented. We have already begun this process: the Library Authoring MFE has been created and is currently deployed for preview in the staging environment.

    • One area where this will get interesting is the editor rewrites. It is not yet clear whether or not we can leverage the MFE framework in the editors.

  5. Leverage LTI as the mechanism for referencing library content from courses. LTI (learning tools interoperability) is an established standard for sharing learning content between LMSs. Viewing Open edX courses as “LTI consumers” and v2 Open edX libraries as “LTI providers”, we believe that the “referencing” part of BD-14 can be essentially implemented as a thin layer on top of the LTI specification. The Open edX platform is already capable of consuming and providing LTI v1.3 content, giving us a head start, although the existing “provider” functionality may need some enhancements to meet BD-14’s requirements.

    Why leverage LTI? It ensures that library content can be shared not just within a single Open edX instance, but across Open edX instances, and to a lesser extent, across any LTI-consuming LMS. This makes Open edX more powerful

    Continue allowing for library content to be exposed via LTI.

    • There is currently an LTI Provider implementation in the content_libraries app. It is not used on edX.org, but is presumably used by other instances (some OpenCraft-hosted ones?).

    • We believe that enabling library content to be exposed via LTI is a big value-add for V2 libraries, making Open edX more powerful in any setting where different LMSs, CMSs, or SISs are being integrated (as edX’s and 2U’s systems may eventually be).

Product terminology

Bold terms are already used in production. Underlined terms are new with BD-14 and thus could still be revised.

...

Term

...

Definition

...

Notes

...

Content Library

...

A collection of reusable components.

...

Legacy Content Library

...

Our user-facing term for v1 (modulestore-backed) libraries, once BD-14 lands.

...

Problem Library

...

A v2 (blockstore-backed) library that may only contain problem components.

...

Video Library

...

A v2 library that may only contain video components.

...

Complex Library

...

A v2 library that may contain any mixture of component types.

...

Library Content Reference

...

A single location in a course in which component(s) from a library are included.

...

In BD-14, we will build a “library reference block” to implement this operation.

...

Library content referencing

...

The act of using a Library Content Reference to include one or more components from a library into a course.

...

I chose this term in favor of “inclusion” or “sourcing” because I think it more accessible and less ambiguous. Open to other opinions here. More radical idea: call it a “launch” as a callback to LTI. -Kyle

...

Library content customization

...

The act of making edits to components included via a Library Reference. The edits are local to the Reference within the course; they are not reflected back in the source library.

...

“customization” is my suggestion, happy to use a different term instead.

...

Library Version

...

An immutable snapshot of a library, as published at a certain point in time.

...

Accompanied by a version name and comment, perhaps?

...

Library publishing

...

The act of releasing all changes to a v2 library since its last publish, creating a new Library Version.

...

v1 libraries do not support the idea of “publishing”.

...

Course

...

A run of course, which is authorable in Studio and hosted in LMS.

...

More precisely, this is called a “course run” in order to differentiate it from a “catalog course” (which are advertised on the marketing site).

...

Course publishing

...

The act of pushing out all edits to a course made in Studio such that they manifest in the LMS. Until a course is published, the edits only manifest in Studio.

...

Some edits must be explicitly published using the “Publish” button. However, some edits, especially structural or course-wide changes, automatically trigger a publish.

...

Component

...

A single piece of course content. Examples include HTML, a discussion, a video, an ORA, a problem (note that a problem may contain multiple responses), and advanced components.

...

Every component is an XBlock.

...

Advanced Component

...

Components other than the core five (HTML, discussion, video, problem, ORA), some of which are authored by external providers, and many of which are not fully supported. Must be explicitly enabled in advanced settings.

...

Unit

...

A series of zero or more components, displayed on a single page.

...

Under the hood these are called “verticals” sometimes. In the Learning MFE, the contents of a unit are rendered together within an iframe.

Background and technical terminology

Libraries, blocks, courses, and contexts

A content library (or just “library”) is a collection of reusable content. Each reusable piece of content is an XBlock usage, aka a block. These blocks are the same pieces that make up courses; by storing them in libraries, though, they can be authored, versioned, and referenced independent of any course-authoring workflow.

At first, this document we will focus on libraries made of component-level blocks; that is, blocks that are individual bites of content, such as problems, videos, HTML, and advanced components (polls, ORA, etc). Future iterations of this document may consider libraries that contain structural blocks, such as units, sequences, and sections.

On edx.org, content from libraries is primarily intended to be reused in courses. With the advent of LabXChange, though, the Open edX platform has also begun serving learning content from a labxchange pathway, which is a “a short collection of XBlocks that a student works through in a linear sequence”. Excitingly, these pathways actually store all their content in v2 libraries, although the content inclusion mechanism and authoring interface is different than what we will be building for edx.org courses.

So, we have generalized courses into the idea of learning contexts, or just “contexts”. Courses, labxchange pathways, and even libraries themselves are types of contexts, and all benefit from the ability to consume reusable content. So, although BD-14 product messaging sometimes presents libraries as collections of content for re-use across courses, in reality content libraries are a collection of blocks authored for re-use across learning contexts.

Storage backends

Libraries exist for use on edx.org today, backed by the split-mongo modulestore (aka “split-mongo”), our MongoDB-backed, versioning, immutable-definition content storage system. We refer to this generation of the content library feature as version 1 (“v1”). For a dive into how v1 libraries are implemented, check out Dave’s v1 library writeup.

All active edx.org courses are also stored in split-mongo. In this document, we will call these courses v1 courses. There also exist courses in the deprecated old-mongo modulestore (which one may call v0 courses), but since old-mongo cannot store libraries nor courses referencing library content, we will not talk about it further in this document.

BD-14 aims to replace v1 libraries with the version 2 (“v2”) implementation, backed instead by blockstore, our SQL + Amazon S3 -backed, versioning, immutable-definition content storage system, which is:

  • not dependent on edx-platform

  • significantly simpler than either modulestore, and

  • designed ground-up with content reuse in mind.

The content in v2 libraries will need to be usable by split-mongo-backed courses, although the technical design of v2 libraries will keep in mind our desire to eventually move all course content to blockstore, which we’ll speculatively refer to as v2 courses. This should be easy, since v2 libraries already serve content to labxchange pathways, which are blockstore-backed.

Keys

Content stored in the Open edX platform is referenced by a variety of types of opaque keys, which are generally semi-human-readable, URL-safe, immutable, and stable string identifiers. The “opaque” adjective describes that each key should generally be treated as indivisible, allowing us to change keys' structure over time without breaking assumpions. For example, URL parsers should not assume that learning context keys being with course-v1:, because that URL may one day need to handle keys prefixed with lx-pathway: or lib: instead.

Different types of keys refer to different entities and have different structures, as shown below. Key parts written in CAPITALS are variables that would be substituted with specific content information.

context keys identify learning contexts.

...

course-v1:ORG+COURSE+RUN is a v1 course key.

...

library-v1:ORG+LIBRARY is a v1 library key.

...

lib:ORG:SLUG is a v2 library key.

...

lx-pathway:PATHWAY_UUID is a labxchange pathway key.

...

    • .

    • Although BD-14 will not focus on developing this aspect of content_libraries, we would like to leave the door open for future improvements and usage of the LTI Provider functionality. Phrased another way, we shouldn’t “build against” the LTI provider implementation.

    • This is an amendment to an older goal that we dropped (see “How is referencing implemented?” below for context):

      • Leverage LTI as the mechanism for referencing library content from courses. LTI (learning tools interoperability) is an established standard for sharing learning content between LMSs. Viewing Open edX courses as “LTI consumers” and v2 Open edX libraries as “LTI providers”, we believe that the “referencing” part of BD-14 can be essentially implemented as a thin layer on top of the LTI specification. The Open edX platform is already capable of consuming and providing LTI v1.3 content, giving us a head start, although the existing “provider” functionality may need some enhancements to meet BD-14’s requirements.

Product terminology

Bold terms are already used in production. Underlined terms are new with BD-14 and thus could still be revised.

Term

Definition

Notes

Content Library

A collection of reusable components.

Legacy Content Library

Our user-facing term for v1 (modulestore-backed) libraries, once BD-14 lands.

Problem Library

A v2 (blockstore-backed) library that may only contain problem components.

Video Library

A v2 library that may only contain video components.

Complex Library

A v2 library that may contain any mixture of component types.

Library Content Reference

A single location in a course in which component(s) from a library are included.

In BD-14, we will build a “library reference block” to implement this operation.

Library content referencing

The act of using a Library Content Reference to include one or more components from a library into a course.

I chose this term in favor of “inclusion” or “sourcing” because I think it more accessible and less ambiguous. Open to other opinions here. More radical idea: call it a “launch” as a callback to LTI. -Kyle

Library content customization

The act of making edits to components included via a Library Reference. The edits are local to the Reference within the course; they are not reflected back in the source library.

“customization” is my suggestion, happy to use a different term instead.

Library Version

An immutable snapshot of a library, as published at a certain point in time.

Accompanied by a version name and comment, perhaps?

Library publishing

The act of releasing all changes to a v2 library since its last publish, creating a new Library Version.

v1 libraries do not support the idea of “publishing”.

Course

A run of course, which is authorable in Studio and hosted in LMS.

More precisely, this is called a “course run” in order to differentiate it from a “catalog course” (which are advertised on the marketing site).

Course publishing

The act of pushing out all edits to a course made in Studio such that they manifest in the LMS. Until a course is published, the edits only manifest in Studio.

Some edits must be explicitly published using the “Publish” button. However, some edits, especially structural or course-wide changes, automatically trigger a publish.

Component

A single piece of course content. Examples include HTML, a discussion, a video, an ORA, a problem (note that a problem may contain multiple responses), and advanced components.

Every component is an XBlock.

Advanced Component

Components other than the core five (HTML, discussion, video, problem, ORA), some of which are authored by external providers, and many of which are not fully supported. Must be explicitly enabled in advanced settings.

Unit

A series of zero or more components, displayed on a single page.

Under the hood these are called “verticals” sometimes. In the Learning MFE, the contents of a unit are rendered together within an iframe.

Background and technical terminology

Libraries, blocks, courses, and contexts

A content library (or just “library”) is a collection of reusable content. Each reusable piece of content is an XBlock usage, aka a block. These blocks are the same pieces that make up courses; by storing them in libraries, though, they can be authored, versioned, and referenced independent of any course-authoring workflow.

At first, this document we will focus on libraries made of component-level blocks; that is, blocks that are individual bites of content, such as problems, videos, HTML, and advanced components (polls, ORA, etc). Future iterations of this document may consider libraries that contain structural blocks, such as units, sequences, and sections.

On edx.org, content from libraries is primarily intended to be reused in courses. With the advent of LabXChange, though, the Open edX platform has also begun serving learning content from a labxchange pathway, which is a “a short collection of XBlocks that a student works through in a linear sequence”. Excitingly, these pathways actually store all their content in v2 libraries, although the content inclusion mechanism and authoring interface is different than what we will be building for edx.org courses.

So, we have generalized courses into the idea of learning contexts, or just “contexts”. Courses, labxchange pathways, and even libraries themselves are types of contexts, and all benefit from the ability to consume reusable content. So, although BD-14 product messaging sometimes presents libraries as collections of content for re-use across courses, in reality content libraries are a collection of blocks authored for re-use across learning contexts.

Storage backends

Libraries exist for use on edx.org today, backed by the split-mongo modulestore (aka “split-mongo”), our MongoDB-backed, versioning, immutable-definition content storage system. We refer to this generation of the content library feature as version 1 (“v1”). For a dive into how v1 libraries are implemented, check out Dave’s v1 library writeup.

All active edx.org courses are also stored in split-mongo. In this document, we will call these courses v1 courses. There also exist courses in the deprecated old-mongo modulestore (which one may call v0 courses), but since old-mongo cannot store libraries nor courses referencing library content, we will not talk about it further in this document.

BD-14 aims to replace v1 libraries with the version 2 (“v2”) implementation, backed instead by blockstore, our SQL + Amazon S3 -backed, versioning, immutable-definition content storage system, which is:

  • not dependent on edx-platform

  • significantly simpler than either modulestore, and

  • designed ground-up with content reuse in mind.

The content in v2 libraries will need to be usable by split-mongo-backed courses, although the technical design of v2 libraries will keep in mind our desire to eventually move all course content to blockstore, which we’ll speculatively refer to as v2 courses. This should be easy, since v2 libraries already serve content to labxchange pathways, which are blockstore-backed.

Keys

Content stored in the Open edX platform is referenced by a variety of types of opaque keys, which are generally semi-human-readable, URL-safe, immutable, and stable string identifiers. The “opaque” adjective describes that each key should generally be treated as indivisible, allowing us to change keys' structure over time without breaking assumpions. For example, URL parsers should not assume that learning context keys being with course-v1:, because that URL may one day need to handle keys prefixed with lx-pathway: or lib: instead.

Different types of keys refer to different entities and have different structures, as shown below. Key parts written in CAPITALS are variables that would be substituted with specific content information.

  • context keys identify learning contexts.

    • course-v1:ORG+COURSE+RUN is a v1 course key.

    • library-v1:ORG+LIBRARY is a v1 library key.

    • lib:ORG:SLUG is a v2 library key.

    • lx-pathway:PATHWAY_UUID is a labxchange pathway key.

    • Note: Some of these keys can also contain version info (+version@abc123...), but this version info should be stripped out before use in LMS.

  • definition keys point to a block’s content in the data storage backend.

    • def-v1:DEFINITION_ID+type@BLOCK_TYPE is a v1 block definition id, identifying a block definition within split-mongo modulestore.

    • bundle-olx:BUNDLE_UUID:VERSION:BLOCK_TYPE:OLX_PATH is a bundle definition key, identifying a block definition with a blockstore OLX bundle.

    • i4x://ORG/COURSE/RUN/SLASH_SEPARATED_BLOCK_IDS is a v0 block location, which served as both a usage and definition key for old-mongo course blocks.

  • usage keys locate the usage of a defined block in a learning context (definition + context = usage).

    • block-v1:ORG+COURSE+RUN+type@BLOCK_TYPE+block@BLOCK_ID is a v1 block usage key, identifying the usage of a block in a v1 course.

    • lb:ORG:SLUG:BLOCK_TYPE:USAGE_ID is a v2 library block usage key, identifying the usage of a block in a v2 library. Usages of blocks in libraries are not intended for learner consumption, but instead for referencing from other contexts. However, library block usages can be interacted with directly by authors as they preview their work – user state is stored ephemerally.

    • lx-pb:PATHWAY_UUID:BLOCK_TYPE:USAGE_ID and lx-pb:PATHWAY_UUID:BLOCK_TYPE:USAGE_ID:CHILD_USAGE_ID are labxchange pathway usage keys. The former format identifies a top-level block usage within the pathway, whereas the latter format identifies a nested block.

    • i4x://ORG/COURSE/RUN/SLASH_SEPARATED_BLOCK_IDS is a v0 block location, which served as both a usage and definition key for old-mongo course blocks.

...

How does access control work across Open edX instances?

TODO

Export and Import

Warning

The export/import section is still under construction.

How are V2 libraries exported and imported?

What does the exported file format look like?

TODO

...

Answer: Didn’t get a chance to dive into this.

What happens to V2-library-referenced content when a course is exported and imported?

...

Decreasing library centralization

Decreasing size of course exports

(a) Library only exists on original instance, so course-library relationship is severed when exporting to other instances.

(b) Library versions are centrally managed by original instance, but replicated on other instances.

(c) Library versions managed in a distributed fashion across instances, identifying versions by git-like hashes.

(1) Entire library is bundled with the course export.

🥡 = ( 🎓 + 📚 )

(1a) nonsensical

(1b) The original library is brought to the new instance along with the course, although the library does not become part of the instance’s list of libraries available for authoring. Instead, the library maintains a connection with the original instance, and can receive updates from the library on that instance. In this sense, the library copy is a replicate of the “centrally” managed library.

Pro: Libraries can be used across instances, while still maintaining simplicity of central management.

Con: Library bundling increases export size.

Challenge: Would need to figure out how to stop remote library keys (from original instance) from clashing with local library keys (on new instance).

Challenge: We’d need to figure out the replication strategy. Push-by-original-instance? Or pull-by-new-instance? And, how we keep track of the URL of the original instance?

(1c)

Specific version(s) of each library can be chosen for export. Multiple versions of the same library can be imported into an Open edX instance without disrupting one another, and importing a library at a version does not necessarily mark it as the “newest”. Library versions are uniquely identified and referenced by a hash of their contents, avoiding conflicts that may arise between human-friendly version names/numbers across instances.

If the required version of the required library exists on an Open edX instance, then the course automatically uses it. Otherwise, library blocks in the course gracefully degrade to rendering hidden blocks.

Example story: https://lucid.app/lucidchart/4950383b-a703-4bd8-b08e-f94a6f144c1a/edit?invitationId=inv_73bdba67-80b6-45d3-9c7b-bf2e5d8e0ac3&page=Mm5j-6d_dnBg# (in the “Import/Export Story” tab, start at the very top and read downwards.

(2) Necessary pieces of library are bundled with course export.

🥡 = ( 🎓 + 📘 )

🥡 = 📚

(2a) Essentially, library blocks turn into ordinary blocks when a course is exported.

Pro: Courses will work out-of-the-box when exported and imported. We avoid wrestling with cross-instance library versioning altogether.

Con: Use of a library is confined to a single instance.

Challenge: Would need to develop a non-library version of randomized content banks.

(2b) Same pros and cons of 2b, except:

  • remove Con: Library bundling increases export size.

  • new Challenge: Including just the necessary library blocks in the course export.

(2c) Same pros and cons of 1c, except:

  • remove Con: Library bundling increases export size.

  • new Challenge: Including just the necessary library blocks in the course export.

(3) Library is excluded from course export.

🥡 = 🎓

🥡 = 📚

(3a) nonsensical

(3b) Same pros and cons of 1b, except:

  • remove Con: Library bundling increases export size.

  • New Con: Course doesn’t work out-of-the box until first replication-from-original-instance occurs.

(3c) Same pros and cons of 1c, except:

  • remove Con: Library bundling increases export size.

  • New Con: Course doesn’t work out-of-the box. User must import correct library version into instance.

Decision

Ideas: I have a couple competing ideas for how import/export could work…

Approach 1: Distributed version control

This is the first approach that came to my mind, after thinking about how to preserve course-library relationships across instances while avoiding library version conflicts from arising.

  • Library blocks don’t export with a course’s OLX. However, library content customizations do export with the OLX (since they are saved on the LibraryReferenceBlock).

  • Libraries can be exported. Notably, specific version(s) of each library can be chosen for export. Multiple versions of the same library can be imported into an Open edX instance without disrupting one another, and importing a library at a version does not necessarily mark it as the “newest”. Library versions are uniquely identified and referenced by a hash of their contents, avoiding conflicts that may arise between human-friendly version names/numbers across instances.

  • If the required version of the required library exists on an Open edX instance, then the course automatically uses it. Otherwise, library blocks in the course gracefully degrade to rendering hidden blocks.

Example story: https://lucid.app/lucidchart/4950383b-a703-4bd8-b08e-f94a6f144c1a/edit?invitationId=inv_73bdba67-80b6-45d3-9c7b-bf2e5d8e0ac3&page=Mm5j-6d_dnBg# (in the “Import/Export Story” tab, start at the very top and read downwards.

Approach 2: Centralized version control

Dave Ormsbee , in reaction to Approach 1:

It might be useful to export [library blocks], even if it doesn’t import back on the other side. There are a number of tools that do content analysis of exports, and if you’re using content from multiple libraries, it’d be convenient to have that in the same export file.

….

I’m leery of making that step [(exporting at any version)] into full blown distributed version control. A question: Can we get away with only having one primary source of a library, and allow instances to essentially make read-replicas?

So for instance, a particular library could exist on edx.org and have some set of users who can modify it, and a simple linear history. But we can configure something so that Edge keeps an updated read-replica, so that it should theoretically always have that library’s data available.

This could be low level functionality that gets built at the blockstore layer as well. Some kind of namespaced replication so that Edge’s blockstore still has edx.org / {library_uuid} . Right now, the OLX export for a library assumes that the library belongs on whatever instance the course is in, but we could explicitly reference a fully qualified domain.

Essentially, instead of relying on library content hashes to uniquely identify libraries across instances, we’d somehow annotate and/or namespace libraries with their origin Open edX instance. Furthermore, libraries would be bundled with course exports.

TODO: write more

Approach 3: Insanity

TODO write more

Decision

Urgency: High

Answer: TBD

What does an OLX export with V2 library content look like?

Potential augmentation of the current course OLX export:

...

breakoutModewide

...

Urgency: High

Answer: Approach 1b.

Some additional decisions & implications:

  • To support this approach, library keys ought to be namespaced by “domain” (a string identifying the Open edX instance, if not the literal domain of the instance). For example, instead of library keys like lib:MITx:Problems1, you’d have lib:edx.org:MITx:Problems1 and lib:labxchange.org:LabXChange:lib-abcdef123.

  • Course authors in an Open edX instance ought to be able to “subscribe” to libraries on other instances, an action that would require an “access key” for non-public libraries. For example, if a hypothetical user of studio.labxchange.org wanted to use the library lib:edx.org:HarvardX:ScienceLib, they would need to request an access token from a ScienceLib staff user, and enter it into LabXChange’s Studio instance in order to make the external library available to LX authors.

What does an OLX export with V2 library content look like?

Urgency: High

Answer, Iteration 1:

Code Block
breakoutModewide
./                         # Root directory of an exported .tar.gz for a course.
  course/                  #   As before, the ./course/ folder contains:
    assets/                #  # Root directory of an exported .tar.gz for a course.
  course/                  #   As before, the ./course/ folder contains:
    assets/                #     (1) an assets folder,
    chapter/               #     (2) one folder for
    .../                   #         each
    vertical/              #         block
    video/                 #         type, and
    course.xml             #     (3) the root course.xml file.
  libraries/               #   New folder! Contains all referenced librari1es, one XML file per library.
    Greendale_VideoLib.xml #     Given a library with the key `lib:ORG:LIBSLUG`, filename is `ORG_LIBSLUG.xml`.
    Greendale_ChemLib.xml  #     <- For example, this defines 'lib:Greendale:ChemLib'.
    CityClg_mathlib.xml    #     XML files contain just metadata, including bundle UUIDs, but no block content.
  bundles/                 #   New folder! Contains definitions for bundles that were referenced in ./libraries/.
    a578e5...5b181d/       #     In the blockstore-course future, ./course/ would also refernce ./bundles/.
    259fa0...71f87f/       #     The idea is that all content lives in ./bundles/ and other folders are metadata.
    d8b24f...2640c8/       #     Bundle folder names are the full 32-char, lowercase, un-hyphenated bundle UUID.
      bundle.xml           #       bundle.xml contains title, slug, version #, and snapshot hash.
      files/               #       Finally, files/ contains the OLX serialization of the bundle contents.
        problem/           #
        .../               #
        video/             #

Urgency: High

Answer: TBD

Referencing V2 library content from V2 courses

V2 (blockstore-backed) courses don’t exist yet, so we don’t need to worry about implementing this as part of BD-14. Rather, these questions are here so that we keep in mind the eventual migration to V2 courses, avoiding decisions during BD-14 that would complicate that process.

How might V2 library content referencing differ in V2 courses (as opposed to V1 courses)?

TODO

Archive

...

Dave pointed out that the structure o the bundles/ directory would not be very human-friendly, even if it reflects the blockstore data model.

Answer, Iteration 2:

Code Block
breakoutModewide
./                         # Root directory of an exported .tar.gz for a course.
  course/                  #   As before, the ./course/ folder contains:
    assets/                #     (1) an assets folder,
    chapter/               #     (2) one folder for
    ...                    #         each
    vertical/              #         block
    video/                 #         type, and
    course.xml             #     (3) the root course.xml file.
  libraries/               #   New folder! Contains all referenced libraries and their content.
    edx.org/               #     Folder is structured by domain,
       Greendale/          #       then organization,
         VideoLib/         #         then library slug,
           v3/             #           and finally library version, mirroring the library key structure.
             ...           #
         ChemLib/          #         Each library contain an OLX serialization of its contents,
           v2/             #
             meta.xml      #           along with a "meta.xml" file to hold versioning #, snapshot hash, et al.
             problem/      #
             ...           #
             video/        #
       CityCollege         #
         mathlib/          #
           v12/            #
             ...           #
           v13/            #
             ...           #
    labxchange.org/        #
      ...                  #

Referencing V2 library content from V2 courses

V2 (blockstore-backed) courses don’t exist yet, so we don’t need to worry about implementing this as part of BD-14. Rather, these questions are here so that we keep in mind the eventual migration to V2 courses, avoiding decisions during BD-14 that would complicate that process.

How might V2 library content referencing differ in V2 courses (as opposed to V1 courses)?

We didn’t end up getting a change to dive into this, but it’s worth keeping in mind.

Archive

Misc notes and stuff

Expand
titlea couple competing ideas for how import/export could work…

Approach 1: Distributed version control

This is the first approach that came to my mind, after thinking about how to preserve course-library relationships across instances while avoiding library version conflicts from arising.

  • Library blocks don’t export with a course’s OLX. However, library content customizations do export with the OLX (since they are saved on the LibraryReferenceBlock).

  • Libraries can be exported. Notably, specific version(s) of each library can be chosen for export. Multiple versions of the same library can be imported into an Open edX instance without disrupting one another, and importing a library at a version does not necessarily mark it as the “newest”. Library versions are uniquely identified and referenced by a hash of their contents, avoiding conflicts that may arise between human-friendly version names/numbers across instances.

  • If the required version of the required library exists on an Open edX instance, then the course automatically uses it. Otherwise, library blocks in the course gracefully degrade to rendering hidden blocks.

Example story: https://lucid.app/lucidchart/4950383b-a703-4bd8-b08e-f94a6f144c1a/edit?invitationId=inv_73bdba67-80b6-45d3-9c7b-bf2e5d8e0ac3&page=Mm5j-6d_dnBg# (in the “Import/Export Story” tab, start at the very top and read downwards.

Approach 2: Centralized version control

Dave Ormsbee , in reaction to Approach 1:

It might be useful to export [library blocks], even if it doesn’t import back on the other side. There are a number of tools that do content analysis of exports, and if you’re using content from multiple libraries, it’d be convenient to have that in the same export file.

….

I’m leery of making that step [(exporting at any version)] into full blown distributed version control. A question: Can we get away with only having one primary source of a library, and allow instances to essentially make read-replicas?

So for instance, a particular library could exist on edx.org and have some set of users who can modify it, and a simple linear history. But we can configure something so that Edge keeps an updated read-replica, so that it should theoretically always have that library’s data available.

This could be low level functionality that gets built at the blockstore layer as well. Some kind of namespaced replication so that Edge’s blockstore still has edx.org / {library_uuid} . Right now, the OLX export for a library assumes that the library belongs on whatever instance the course is in, but we could explicitly reference a fully qualified domain.

Essentially, instead of relying on library content hashes to uniquely identify libraries across instances, we’d somehow annotate and/or namespace libraries with their origin Open edX instance. Furthermore, libraries would be bundled with course exports.

TODO: write more

Expand
titleMisc notes from convo w/ jennifer et al on how v1 libraries are used today, specifically in reference to library content customization
  • jennifer’s opinion

    • content should be edited in lib

    • settings should be edited in course

    • distinction is sometimes fuzzy–e.g. content is sometimes corrected within the context of a course today (minor changes to text)

  • fields that are ambiguous content v setting?

    • display_name?

      • renaming after import isn’t usually done in v1 libraries

      • confusion around problem titles coming from libraries sometimes

        • Names in libraries often have different meaning: e.g. 1a, 1b, 1c. vs. what it would be in the course (“Problem 1: {helpful description}”).

    • video start time

      • not used much currently?

      • would need to verify with CourseGraph

  • in units with library content, is there other stuff?

    • often a preamble HTML followed by random problems

    • mixture of HTML+video+problems in library very uncommon, since specific-block selection has been broken for a while

    • why: a single LTI launch for the entire unit would be cool

      • save ourselves an iframe layer

  • stress test

    • 30 blocks pulled in from a library of 50 in a single Unit

      • this is because it’s the only way right now to make sure that content doesn’t repeat.

        • interesting data model implications. Have a saved state for this inclusion of this library, but allow it to be launched across multiple pages?

  • LMS interactions

    • Can one LTI-launched thing map to multiple scores? This would be nice for grading purposes, but may be hard to retrofit.

      • Reactive use case: Need to correct and re-score one problem from the possible selection of the library.

      • Grades information.

      • Deep linking here?

    • Does it make sense for there to be two inclusion modes: one with single and one for a whole unit?

    • Settings tend to be re-used across a whole subsection, or even type “e.g. when using it as a HW, always give them five attempts”

      • Rare case: We find that one problem is too hard and adjust it to allow for more attempts.

...