Fun with LibraryContentBlock export/import and duplication
Legend
opaque_key{ < settings-scoped defaults > } { < settings-scoped fields > } { < content-scoped fields > } # commentchild_1_opaque_keyetc
Scenario
We have a course with a library_content block pointing at version 5 of a library, containing blocks W, X, Y, Z.
The course overrides the titles (display_name) for X and Y, but uses the default titles of W and Z.
It edits the capa content (data) for Y and Z, but uses the upstream content of W and X.
The latest version of the library (6) makes some changes:
W and X are deleted.
Y and Z are kept, although Z’s content is edited.
Q is added.
Here’s what the lib looks like:
lib:O:LVersion 5
lb:O:L:problem:libBlockW{ display_name: “title W” } { data: “www” }lb:O:L:problem:libBlockX{ display_name: “title X” } { data: “xxx” }lb:O:L:problem:libBlockY{ display_name: “title Y” } { data: “yyy” }lb:O:L:problem:libBlockZ{ display_name: “title Z” } { data: “zzz” }
Version 6 {latest}
# libBlockW removed
# libBlockX removed
lb:O:L:problem:libBlockY{ display_name: “title Y” } { data: “yyy” }lb:O:L:problem:libBlockZ{ display_name: “title Z” } { data: “zzz_updated” } # content changed since v5lb:O:L:problem:libBlockQ{ display_name: “title Q” } { data: “qqq” } # new
And here’s what the course tree looks like:
course-v1:O+C+Rblock-v1:O+C+R+type@chapter+block@...block-v1:O+C+R+type@sequence+block@...block-v1:O+C+R+type@vertical+block@...block-v1:O+C+R+type@library_content+block@myLCB{ source_library_id: “lib:O:L”, source_library_version: “5” }block-v1:O+C+R+type@problem+block@{DERIVE_ID("myLCB", "lb:O:L:problem:libBlockW")}{ display_name: “title W” } { data: “www” }block-v1:O+C+R+type@problem+block@{DERIVE_ID("myLCB", "lb:O:L:problem:libBlockX")}{ display_name: “title X” } { display_name: “override title X” } { data: “xxx” }block-v1:O+C+R+type@problem+block@{DERIVE_ID("myLCB", "lb:O:L:problem:libBlockY")}{ display_name: “title Y” } { display_name: “override title Y” } { data: “yyy_edit” }block-v1:O+C+R+type@problem+block@{DERIVE_ID("myLCB", "lb:O:L:problem:libBlockZ")}{ display_name: “title Z” } { data: “zzz_edit” }
What does the user see for the blocks under myLCB? Unsurprisingly:
title W
www
override title X
xxx
override title Y
yyy_edit
title Z
zzz_edit
Curveball 1: Import/Export
We export course-v1:O+C+R and then import it into a new course run, course-v1-O+C+R2. Unfortunately, the library’s default settings don’t go into in the export!
Here’s what R2 looks like:
course-v1:O+C+R2block-v1:O+C+R2+type@chapter+block@...block-v1:O+C+R2+type@sequence+block@...block-v1:O+C+R2+type@vertical+block@...block-v1:O+C+R2+type@library_content+block@myLCB{ source_library_id: “lib:O:L”, source_library_version: “5” }block-v1:O+C+R2+type@problem+block@{DERIVE_ID("myLCB", "lb:O:L:problem:libBlockW")}{ data: “www” }block-v1:O+C+R2+type@problem+block@{DERIVE_ID("myLCB", "lb:O:L:problem:libBlockX")}{ display_name: “override title X” } { data: “xxx” }block-v1:O+C+R2+type@problem+block@{DERIVE_ID("myLCB", "lb:O:L:problem:libBlockY")}{ display_name: “override title Y” } { data: “yyy_edit” }block-v1:O+C+R2+type@problem+block@{DERIVE_ID("myLCB", "lb:O:L:problem:libBlockZ")}{ data: “zzz_edit” }
What does the user see now? Well, wherever a title override isn’t set, they see the the ProblemBlock’s provided default title:
Problem
www
override title X
xxx
override title Y
yyy_edit
Problem
zzz_edit
How do we work around this for V1 libs?
We re-load the blocks from the library at the proper version (5). This gives us our defaults back, but it does blow away any content edits :(
R2 would then look like this:
course-v1:O+C+R2block-v1:O+C+R2+type@chapter+block@...block-v1:O+C+R2+type@sequence+block@...block-v1:O+C+R2+type@vertical+block@...block-v1:O+C+R2+type@library_content+block@myLCB{ source_library_id: “lib:O:L”, source_library_version: “5” }block-v1:O+C+R2+type@problem+block@{DERIVE_ID("myLCB", "lb:O:L:problem:libBlockW")}{ display_name: “title W” } { data: “www” }block-v1:O+C+R2+type@problem+block@{DERIVE_ID("myLCB", "lb:O:L:problem:libBlockX")}{ display_name: “title X” } { display_name: “override title X” } { data: “xxx” }block-v1:O+C+R2+type@problem+block@{DERIVE_ID("myLCB", "lb:O:L:problem:libBlockY")}{ display_name: “title Y” } { display_name: “override title Y” } { data: “yyy” }block-v1:O+C+R2+type@problem+block@{DERIVE_ID("myLCB", "lb:O:L:problem:libBlockZ")}{ display_name: “title Z” } { data: “zzz” }
And the user would see the right titles, but no content edits:
title W
www
override title X
xxx
override title Y
yyy
title Z
zzz
How do we want to work around this for V2 libs?
Well, first of all the current V2 content_libraries API does not support loading blocks for old library versions. We could add support for that, but it wouldn’t really fit efficiently or elegantly with Learning Core’s data model, which optimizes for loading versions of individual components rather than content libraries.
Instead, we’d rather add defaults to the OLX export, so that the export comes back with defaults intact, and removes the need to re-load library blocks, thus preserving library edits The structure is now:
course-v1:O+C+R2block-v1:O+C+R2+type@chapter+block@...block-v1:O+C+R2+type@sequence+block@...block-v1:O+C+R2+type@vertical+block@...block-v1:O+C+R2+type@library_content+block@myLCB{ source_library_id: “lib:O:L”, source_library_version: “5” }block-v1:O+C+R2+type@problem+block@{DERIVE_ID("myLCB", "lb:O:L:problem:libBlockW")}{ display_name: “title W” } { data: “www” }block-v1:O+C+R2+type@problem+block@{DERIVE_ID("myLCB", "lb:O:L:problem:libBlockX")}{ display_name: “title X” } { display_name: “override title X” } { data: “xxx” }block-v1:O+C+R2+type@problem+block@{DERIVE_ID("myLCB", "lb:O:L:problem:libBlockY")}{ display_name: “title Y” } { display_name: “override title Y” } { data: “yyy_edit” }block-v1:O+C+R2+type@problem+block@{DERIVE_ID("myLCB", "lb:O:L:problem:libBlockZ")}{ display_name: “title Z” } { data: “zzz_edit” }
Yielding:
title W
www
override title X
xxx
override title Y
yyy_edit
title Z
zzz_edit
OPEN QUESTION… how will this work for courses which were exported before the v1/v2 transition?
TODO… write down schema for defaults, something like:
<problem display_name="bob" default-display_name="superbob" />TODO… write down default vs inheritance semantics, something like:
Parent { setting1: parentValue }
Child {defaults : { setting1: value1 }} → setting1: parentValue
Unit { setting1: parentValue }
LCB
Child {defaults : { setting1: value1 }} → setting1: parentValue
Curveball 2: Duplication
Back to the original course, let’s say the user duplicates LCB. The standard handling of “duplicate a block” would randomly generating id strings for its children:
course-v1:O+C+Rblock-v1:O+C+R+type@chapter+block@...block-v1:O+C+R+type@sequence+block@...block-v1:O+C+R+type@vertical+block@...block-v1:O+C+R+type@library_content+block@myLCB{ source_library_id: “lib:O:L”, source_library_version: “5” }block-v1:O+C+R+type@problem+block@{DERIVE_ID("myLCB", "lb:O:L:problem:libBlockW")}{ display_name: “title W” } { data: “www” }block-v1:O+C+R+type@problem+block@{DERIVE_ID("myLCB", "lb:O:L:problem:libBlockX")}{ display_name: “title X” } { display_name: “override title X” } { data: “xxx” }block-v1:O+C+R+type@problem+block@{DERIVE_ID("myLCB", "lb:O:L:problem:libBlockY")}{ display_name: “title Y” } { display_name: “override title Y” } { data: “yyy_edit” }block-v1:O+C+R+type@problem+block@{DERIVE_ID("myLCB", "lb:O:L:problem:libBlockZ")}{ display_name: “title Z” } { data: “zzz_edit” }
block-v1:O+C+R+type@library_content+block@dupeLCB{ source_library_id: “lib:O:L”, source_library_version: “5” }block-v1:O+C+R+type@problem+block@deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdead1{ display_name: “title W” } { data: “www” }block-v1:O+C+R+type@problem+block@deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdead2{ display_name: “title X” } { display_name: “override title X” } { data: “xxx” }block-v1:O+C+R+type@problem+block@deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdead3{ display_name: “title Y” } { display_name: “override title Y” } { data: “yyy }block-v1:O+C+R+type@problem+block@deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdead4{ display_name: “title Z” } { data: “zzz” }
That would seem all well and good, until the course author updates the library’s source version…. at which point the children with randomly-generated IDs would be thrown away in favor of new children with library-block-id-derived IDs. We’d expect that the settings overrides and student state for blocks W and X get thrown away, since those blocks are removed in library version 6. But we wouldn’t expect that the settings and student state of Y and Z would also be thrown out! (Note that the content edits are also thrown out, but that’s standard when it comes to updating a library to the latest version.) The messed-up structure would look like this:
course-v1:O+C+Rblock-v1:O+C+R+type@chapter+block@...block-v1:O+C+R+type@sequence+block@...block-v1:O+C+R+type@vertical+block@...block-v1:O+C+R+type@library_content+block@myLCB{ source_library_id: “lib:O:L”, source_library_version: “5” }block-v1:O+C+R+type@problem+block@{DERIVE_ID("myLCB", "lb:O:L:problem:libBlockW")}{ display_name: “title W” } { data: “www” }block-v1:O+C+R+type@problem+block@{DERIVE_ID("myLCB", "lb:O:L:problem:libBlockX")}{ display_name: “title X” } { display_name: “override title X” } { data: “xxx” }block-v1:O+C+R+type@problem+block@{DERIVE_ID("myLCB", "lb:O:L:problem:libBlockY")}{ display_name: “title Y” } { display_name: “override title Y” } { data: “yyy_edit” }block-v1:O+C+R+type@problem+block@{DERIVE_ID("myLCB", "lb:O:L:problem:libBlockZ")}{ display_name: “title Z” } { data: “zzz_edit” }
block-v1:O+C+R+type@library_content+block@dupeLCB{ source_library_id: “lib:O:L”, source_library_version: “6” }# libBlockW was deleted from the library – it gets removed here, as expected.
# libBlockX was deleted from the library – it gets removed here, as expected.
block-v1:O+C+R+type@problem+block@{DERIVE_ID("dupeLCB", "lb:O:L:problem:libBlockY")}{ display_name: “title Y” } { data: “yyy” } # R.I.P. settings and student state :(block-v1:O+C+R+type@problem+block@{DERIVE_ID("dupeLCB", "lb:O:L:problem:libBlockZ")}{ display_name: “title Z” } { data: “zzz_updated” } # R.I.P. settings and student state :(block-v1:O+C+R+type@problem+block@{DERIVE_ID("dupeLCB", "lb:O:L:problem:libBlockQ")}{ display_name: “title Q” } { data: “qqq” } # Brand new
How does V1 handle this? Well, it special-cases the duplication function, telling it to not duplicate the library_content block’s children:
course-v1:O+C+Rblock-v1:O+C+R+type@chapter+block@...block-v1:O+C+R+type@sequence+block@...block-v1:O+C+R+type@vertical+block@...block-v1:O+C+R+type@library_content+block@myLCB{ source_library_id: “lib:O:L”, source_library_version: “5” }block-v1:O+C+R+type@problem+block@{DERIVE_ID("myLCB", "lb:O:L:problem:libBlockW")}{ display_name: “title W” } { data: “www” }block-v1:O+C+R+type@problem+block@{DERIVE_ID("myLCB", "lb:O:L:problem:libBlockX")}{ display_name: “title X” } { display_name: “override title X” } { data: “xxx” }block-v1:O+C+R+type@problem+block@{DERIVE_ID("myLCB", "lb:O:L:problem:libBlockY")}{ display_name: “title Y” } { display_name: “override title Y” } { data: “yyy_edit” }block-v1:O+C+R+type@problem+block@{DERIVE_ID("myLCB", "lb:O:L:problem:libBlockZ")}{ display_name: “title Z” } { data: “zzz_edit” }
block-v1:O+C+R+type@library_content+block@dupeLCB{ source_library_id: “lib:O:L”, source_library_version: “5” }# no children yet
Then, in the studio_post_duplicate callback, it reloads the blocks from the library, yielding:
course-v1:O+C+Rblock-v1:O+C+R+type@chapter+block@...block-v1:O+C+R+type@sequence+block@...block-v1:O+C+R+type@vertical+block@...block-v1:O+C+R+type@library_content+block@myLCB{ source_library_id: “lib:O:L”, source_library_version: “5” }block-v1:O+C+R+type@problem+block@{DERIVE_ID("myLCB", "lb:O:L:problem:libBlockW")}{ display_name: “title W” } { data: “www” }block-v1:O+C+R+type@problem+block@{DERIVE_ID("myLCB", "lb:O:L:problem:libBlockX")}{ display_name: “title X” } { display_name: “override title X” } { data: “xxx” }block-v1:O+C+R+type@problem+block@{DERIVE_ID("myLCB", "lb:O:L:problem:libBlockY")}{ display_name: “title Y” } { display_name: “override title Y” } { data: “yyy_edit” }block-v1:O+C+R+type@problem+block@{DERIVE_ID("myLCB", "lb:O:L:problem:libBlockZ")}{ display_name: “title Z” } { data: “zzz_edit” }
block-v1:O+C+R+type@library_content+block@dupeLCB{ source_library_id: “lib:O:L”, source_library_version: “5” }block-v1:O+C+R+type@problem+block@{DERIVE_ID("dupeLCB", "lb:O:L:problem:libBlockW")}{ display_name: “title W” } { data: “www” }block-v1:O+C+R+type@problem+block@{DERIVE_ID("dupeLCB", "lb:O:L:problem:libBlockX")}{ display_name: “title X” } { display_name: “override title X” } { data: “xxx” }block-v1:O+C+R+type@problem+block@{DERIVE_ID("dupeLCB", "lb:O:L:problem:libBlockY")}{ display_name: “title Y” } { display_name: “override title Y” } { data: “yyy }block-v1:O+C+R+type@problem+block@{DERIVE_ID("dupeLCB", "lb:O:L:problem:libBlockZ")}{ display_name: “title Z” } { data: “zzz” }
That’s better (although content edits are lost, just like they are for V1 import ). Now, if dupeLBC’s library version were to be updated, we’d retain settings overrides and student state for the blocks that stick around between versions 5 and 6:
course-v1:O+C+Rblock-v1:O+C+R+type@chapter+block@...block-v1:O+C+R+type@sequence+block@...block-v1:O+C+R+type@vertical+block@...block-v1:O+C+R+type@library_content+block@myLCB{ source_library_id: “lib:O:L”, source_library_version: “5” }block-v1:O+C+R+type@problem+block@{DERIVE_ID("myLCB", "lb:O:L:problem:libBlockW")}{ display_name: “title W” } { data: “www” }block-v1:O+C+R+type@problem+block@{DERIVE_ID("myLCB", "lb:O:L:problem:libBlockX")}{ display_name: “title X” } { display_name: “override title X” } { data: “xxx” }block-v1:O+C+R+type@problem+block@{DERIVE_ID("myLCB", "lb:O:L:problem:libBlockY")}{ display_name: “title Y” } { display_name: “override title Y” } { data: “yyy_edit” }block-v1:O+C+R+type@problem+block@{DERIVE_ID("myLCB", "lb:O:L:problem:libBlockZ")}{ display_name: “title Z” } { data: “zzz_edit” }
block-v1:O+C+R+type@library_content+block@dupeLCB{ source_library_id: “lib:O:L”, source_library_version: “6” }# libBlockW was deleted from the library – it gets removed here, as expected.
# libBlockX was deleted from the library – it gets removed here, as expected.
block-v1:O+C+R+type@problem+block@{DERIVE_ID("dupeLCB", "lb:O:L:problem:libBlockY")}{ display_name: “title Y” } { display_name: “override title Y” } { data: “yyy” } # Student state remains!block-v1:O+C+R+type@problem+block@{DERIVE_ID("dupeLCB", "lb:O:L:problem:libBlockZ")}{ display_name: “title Z” } { data: “zzz_updated” } # Student state remains!block-v1:O+C+R+type@problem+block@{DERIVE_ID("dupeLCB", "lb:O:L:problem:libBlockQ")}{ display_name: “title Q” } { data: “qqq” } # Brand new
How do we want to do this for V2 libraries? Better, that’s how. Essentially, we want to copy the blocks, thus preserving content overrides, but we’re going to explicitly set the usage keys to what their derived values should be, thus linking up the duplicated child blocks with their upstream library counterparts.
The thing is, this requires knowing what the source library blocks' keys were. In these visuals, we’ve been spelling them out (lb:O:L:problem:libBlock_), but the LibraryContentBlock doesn’t actually store them--it just stores its children, whose usage keys have been spit out from the non-reversible DERIVE_ID function.
So how do we get back the library blocks' usage keys? Well, it’s simple. We just grab the blocks in the latest version of the library, loop through them calling DERIVE_KEY, and rebuild a mapping from child keys to library keys! That was probably hard to follow. Here’s some pseudocode:
def compute_library_content_children_mapping(original_lcb: LibraryContentBlock, dupe_lcb: LibraryContentBlock) -> dict[UsageKey, UsageKey]:
original_child_ids_to_library_keys: dict[str, LibraryBlockLocatorV2] = {
DERIVE_ID(original_lcb.usage_key.block_id, source_lib_block_key): source_library_usage_key
for source_lib_usage_key in get_library_block_usage_keys(original_lcb.source_library_id)
}
original_child_ids_to_dupe_child_ids: dict[str, str] = {
(
# Block exists in latest lib version. Upgrading lib should preserve it. Derive dupe child key so that it matches!
DERIVE(dupe_lcb.usage_key.block_id, original_child_ids_to_library_keys[original_child_key.block_id])
if original_child_key. block_id in original_child_ids_to_library_keys
# Block is gone in latest lib version. Upgrading lib will destroy it. So, its duplicated child key is arbitrary!
else f"{original_child_key.block_id}_unlinked_copy"
)
for original_child_key in original_lcb.children
}
return {
original_lcb.context_key.make_usage_key(original_child_id)
dupe_lcb.context_key.make_usage_key(dupe_child_id)
for original_child_id, dupe_child_id in original_child_ids_to_dupe_child_ids.items()
}Performing this operation, we get:
course-v1:O+C+Rblock-v1:O+C+R+type@chapter+block@...block-v1:O+C+R+type@sequence+block@...block-v1:O+C+R+type@vertical+block@...block-v1:O+C+R+type@library_content+block@myLCB{ source_library_id: “lib:O:L”, source_library_version: “5” }block-v1:O+C+R+type@problem+block@{DERIVE_ID("myLCB", "lb:O:L:problem:libBlockW")}{ display_name: “title W” } { data: “www” }block-v1:O+C+R+type@problem+block@{DERIVE_ID("myLCB", "lb:O:L:problem:libBlockX")}{ display_name: “title X” } { display_name: “override title X” } { data: “xxx” }block-v1:O+C+R+type@problem+block@{DERIVE_ID("myLCB", "lb:O:L:problem:libBlockY")}{ display_name: “title Y” } { display_name: “override title Y” } { data: “yyy_edit” }block-v1:O+C+R+type@problem+block@{DERIVE_ID("myLCB", "lb:O:L:problem:libBlockZ")}{ display_name: “title Z” } { data: “zzz_edit” }
block-v1:O+C+R+type@library_content+block@dupeLCB{ source_library_id: “lib:O:L”, source_library_version: “5” }block-v1:O+C+R+type@problem+block@{DERIVE_ID("myLCB", "lb:O:L:problem:libBlockW")}_unlinked_copy{ display_name: “title W” } { data: “www” }block-v1:O+C+R+type@problem+block@{DERIVE_ID("myLCB", "lb:O:L:problem:libBlockX")}_unlinked_copy{ display_name: “title X” } { display_name: “override title X” } { data: “xxx” }block-v1:O+C+R+type@problem+block@{DERIVE_ID("dupeLCB", "lb:O:L:problem:libBlockY")}{ display_name: “title Y” } { display_name: “override title Y” } { data: “yyy_edit” } # Edit remains!!block-v1:O+C+R+type@problem+block@{DERIVE_ID("dupeLCB", "lb:O:L:problem:libBlockZ")}{ display_name: “title Z” } { data: “zzz_edit” } # Edit remains!!
And, if the user upgraded the dupe’s source version, they’d happily see:
course-v1:O+C+Rblock-v1:O+C+R+type@chapter+block@...block-v1:O+C+R+type@sequence+block@...block-v1:O+C+R+type@vertical+block@...block-v1:O+C+R+type@library_content+block@myLCB{ source_library_id: “lib:O:L”, source_library_version: “5” }block-v1:O+C+R+type@problem+block@{DERIVE_ID("myLCB", "lb:O:L:problem:libBlockW")}{ display_name: “title W” } { data: “www” }block-v1:O+C+R+type@problem+block@{DERIVE_ID("myLCB", "lb:O:L:problem:libBlockX")}{ display_name: “title X” } { display_name: “override title X” } { data: “xxx” }block-v1:O+C+R+type@problem+block@{DERIVE_ID("myLCB", "lb:O:L:problem:libBlockY")}{ display_name: “title Y” } { display_name: “override title Y” } { data: “yyy_edit” }block-v1:O+C+R+type@problem+block@{DERIVE_ID("myLCB", "lb:O:L:problem:libBlockZ")}{ display_name: “title Z” } { data: “zzz_edit” }
block-v1:O+C+R+type@library_content+block@dupeLCB{ source_library_id: “lib:O:L”, source_library_version: “6” }# libBlockW was deleted from the library – it gets removed here, as expected.
# libBlockX was deleted from the library – it gets removed here, as expected.
block-v1:O+C+R+type@problem+block@{DERIVE_ID("dupeLCB", "lb:O:L:problem:libBlockY")}{ display_name: “title Y” } { display_name: “override title Y” } { data: “yyy” } # Student state remains!block-v1:O+C+R+type@problem+block@{DERIVE_ID("dupeLCB", "lb:O:L:problem:libBlockZ")}{ display_name: “title Z” } { data: “zzz_updated” } # Student state remains!block-v1:O+C+R+type@problem+block@{DERIVE_ID("dupeLCB", "lb:O:L:problem:libBlockQ")}{ display_name: “title Q” } { data: “qqq” } # Brand new