DraftModuleStore (Old Mongo Modulestore)

Description

The DraftModuleStore (also sometimes referred to as "Old Mongo") is the interface used to store courseware content for courses with course run keys of the format "Org/Course/Run", e.g. "edX/Demo/2012". Newer courses of the format "course-v1:Org+Course+Run" use the DraftVersioningModuleStore (also commonly called "Split Mongo"). This has been available since the Birch release and the vast majority of sites have never used DraftModuleStore. One of the main motivations for the newer modulestore was atomic publishes, as it was possible to get partially applied updates if there was an error during the import process with DraftModuleStore.

Because DraftModuleStore has a very different data structure from DraftVersioningModuleStore, supporting both formats simultaneously has resulted in significant complexity, bugs, and performance issues going as far back as the initial implementation of cohorts. There are also thousands of extra tests run specifically to ensure compatibility across ModuleStores.

Some sites intentionally continued to use DraftModuleStore because of storage-related concerns. DraftVersioningModuleStore did not free up disk space used by old versions of the content. However, this problem has been addressed with tubular's structures.py script, which can be run on a regular basis to prune unused old versions of course content.

Proposal:

  • Juniper will display a message in Studio for all courses using DraftModuleStore, saying that this course format will no longer be supported, and urging people to create a re-run of the course (which will make a copy of that course in the DraftVersioningModuleStore).

  • Koa will remove all support for DraftModuleStore. This will involve removing or modifying thousands of tests, as well as removing the MixedModulestore proxy class.

  • Course Overviews using the old course format should still be supported. Old-style courses should not suddenly disappear from your list of enrollments, but any attempt to access courseware content within them (learning sequences, files and uploads) will fail.

  • For the Koa release, there will be no Studio access at all for DraftModuleStore, and it will not be possible to do a data export from Studio for these courses.

Note: In previous conversations, I had discussed the possibility of having a data migration that would convert DraftModuleStore course content into DraftVersioningModuleStore courses while preserving IDs. I created a proof of concept for this approach as a hackathon project. This has the upside of letting us get rid of a chunk of the old code without giving up compatibility, but it also had a number of strong drawbacks, including:

  1. We would have to maintain a large set of tests that used both ID formats for course keys.

  2. It would subtly change opaque keys such that two different keys would serialize identically–we would be adding course-run and version information to "i4x:..." style keys, but for data compatibility reasons we would have to serialize without that information. This has implications for course key caching and would make debugging much trickier–the newer modulestore itself derives its own keys to pass around, and we’d end up in a spaghetti of keys which sometimes have or don’t have run and version information.

Because usage of the old modulestore outside of edX itself is limited and we did not want to introduce any more complexity to what is already a major source of bugs in edx-platform, this DEPR is going the simpler route of removing support altogether.

Compatibility notes:

  1. We won't explicitly delete the old course content. If an Open edX site upgrades to Koa without realizing the implications and rolls back to Juniper, their DraftModuleStore content should still be there.

  2. We won't delete any student course state for these courses, so module state in courseware_studentmodule will be preserved.

  3. The relevant key types (SlashSeparatedCourseKeys and Locations) will not be removed from opaque-keys.

  4. Our goal would be to preserve the functionality of other pages that are not directly courseware, such as the student dashboard. However some functionality is so dependent on the modulestore’s existence that it’s likely they will be disabled for these courses rather than trying to port them to work without a backing modulestore. While more investigation needs to be done, it is likely that most if not all of the Instructor Dashboard for old-style courses will be unavailable in Koa.

 

Status

Assignee

Unassigned

Reporter

David Ormsbee

Labels

None

Removal Date

None

Deprecation Proposal Accepted Date

None

Priority

Unset
Configure