In a world where course teams change content, including removing/adding blocks, the following options allow learners to see different snapshots (versions) of the course content based on which version they initially interacted with, and thus, locked in.

Option 1. Expose versions from the modulestore and access version-specific data

• Allow access to older versions of the course.
• Persist the course's version number along with the learner's submission.
  • Continue to use the granularity of subsections to pin down the version number.
  • Or consider the granularity of verticals if we want to be more fine-grained.
• To avoid race conditions, capture the version number at the time of rendering the problem.
• Update the modulestore interface to:
  • return the version number of the returned block
  • take (an older) version number as an optional parameter and return that version of the block (only in the Published branch as read-only data).

Note: This is the most preferred option since it makes use of the versioning technology inherently available and using it for the purpose it was originally built.

Option 2. For each graded subsection, store required information of all scored sub-components, and only soft-delete blocks

• Along with a subsection's grade, also store all metadata about the subsection's subtree that is needed to re-compute the subsection's grade.  This includes
  • block-ids, weights, max_scores of all scorable (has_score=True) blocks within the subtree that are accessible by the user
  • this is equivalent to storing the data collected by the GradesTransformer
• When returning an already computed subsection's grade, use the stored metadata about the blocks in the subsection.
  • The scores for all blocks, including removed ones, will continue to remain in CSM.
• In order prevent a confusing learner experience where learners cannot access removed blocks that are still listed in their grades, consider never actually deleting blocks from a live course.
  
  • Instead, just mark the block as inactive so it doesn't appear in the natural flow of the courseware, but can still be accessed via a deep-link when needed.
  • Inactive blocks can be purged when a rerun of the course is created.

Note: This option is proposed in case Option 1 is determined as not possible.  However, it's not ideal as it requires duplicating subsections' subtree data and maintaining that additional storage.

Option 3. Prevent addition/removal of blocks in a live course

• A live course can be defined as a course whose courseware is released and accessible by active learners.  That is, start_date is in the past with learner-visible content.

• Disable addition/removal/moving of course blocks in studio for course authors.
• Allow super-users to continue to do so for exceptional cases.

Note: This is not an ideal option for the long-term since editing of content in self-paced courses is required.