This custom plugin for Open edX called progress-edx-platform-extensions (or just “progress”), together with corresponding features in the Solutions REST API, provides an implementation of completion that meets some of the goals in this document. 

• User completion per-XBlock is stored in a model called CourseModuleCompletion.

• Note for people checking out the code: the "stage" column now seems to be unused.

• Another model called StudentProgress summarizes the student’s progress through the whole course (as a single number)

• Beyond simple tracking of “completed” / “incomplete”, this progress plugin can:

• Ignore certain block types.
  

• It has some drawbacks:

  • It is designed for server-to-server access, so any authorized API client can read/write the progress of any user on the system.

  • Not known to be scalable (or possibly known not to be scalable)

  • It is not part of the core Open edX platform

  • Statistics can be inaccurate or out of date in some edge cases; if the cached data is wrong, it is necessary to forcibly updated using the “recalculate_progress_for_users” command.
  • Some of the code necessary for actually computing a user's progress as a percentage is maintained separately in a different closed-source repository.
• What is considered “completion”?
  • When a user interacts with an XBlock and receives a grade, the act of getting a grade will mark that XBlock as “completed”.
  • An XBlock can also emit a “progress” event, which will mark that XBlock as completed, without issuing a grade.
    • e.g. When Problem Builder is used to show a long answer recap (no submit button), it uses this event to mark itself as completed once the user views it.
• What is the API?
  • Generally, completion is written/updated automatically by the LMS
  • Completion can be read from GET /api/server/courses/{course_id}/metrics/completions/leaders/ or /api/server/courses/:course_id/completions/ but only by a trusted server (shared secret API authentication only)
    • The /completions/ API requires the API client to do further computation to determine a user’s overall completion; the /leaders/ API “actually gives the completion of a single user or N top users in a course and client does have not make any further computations” so it is more scalable and consistent.
      

  • Completion can be written by POST to /api/server/courses/:course_id/completions/ but only by a trusted server. We are not aware of anyone/anything that actually uses this method to update completion, however.

Implementation Proposal Overview

1. Create a new model in the DB (BlockCompletion) that can indicate when a user has “completed” a completable XBlock.

1. Fields: user, course_key, usage_key, block_type, completed (float [0.0, 1.0]) + fields from TimeStampedModel.

2. A record with "completed" == 0.0 is equivalent to no record that exists.

3. It should have appropriate indexes for fast lookups.
   

4. This is roughly based on the CourseModuleCompletion model from progress-edx-platform-extensions but with some changes. The “get_actual_completions” code and the "stage" column would not be used.

2. Create a new model in the DB (AggregateCompletion) that indicates for each block whose block-type is an aggregator what percentage of the contained completable blocks have been completed. 
   1. Fields: user, course_key, usage_key, aggregation_name (equivalent to block_type for simple aggregations covered by this proposal), earned (float), possible (float), + fields from TimeStampedModel.
   2. When a BlockCompletion record is updated, a signal handler will trigger an asynchronous task to update the AggregateCompletion records for all ancestor aggregators of the updated block.  The possible value is the sum of the completable blocks contained within the calculable block.  The earned value is the sum of the completed values for all contained completable blocks.  See the note above about DAGs.
   3. Available property: percent (float [0.0, 1.0]) = earned / possible.  An AggregateCompletion record with a possible value of 0.0 will have a percent of 1.0, reflecting the observation that if there is nothing to complete, the learner is done with the block, and that a learner should expect to be able to reach 100% completion for all aggregator blocks once they have completed the course.
   4. Note: Course-level completions will also be stored in this table, with "course" as the aggregate type.
3. A mechanism will be provided to prevent returning invalid completion data for any users that start a course before completion data was captured, as there is no simple way to backfill this data.
   1. Proposal 1: A system-wide setting, "COMPLETION_TRACKING_START_DATE" enables calculation of aggregator completion values for all enrollments that start after the specified date.  This exists to prevent returning invalid completion data for any courses that started before the feature was implemented, as there is no way to backfill the data.  This will be calculated using the new "Content Availability Date" (CAD). A per-course start date model config will provide a way to do staged-rollout of completion aggregation on a course-by-course basis, but calculation will still be subject to the restrictions of the CAD.
   2. Proposal 2: Tracking per user when they started the course and when completion data had begun to be captured, in order to determine whether the completion data is useful for that particular user.

How is completion updated/written?

1. The LMS should be modified to mark an XBlock as “completed” using this logic:

1. If the block type indicates that it is not completable (completion_method=EXCLUDED or completion_method=AGGREGATOR), then the block is ignored.

2. If the XBlock indicates that it is aware of the Completion API, and has a custom implementation of completion (“has_custom_completion” is true): Wait for the XBlock to emit a “complete” event with a value in the range [0.0, 1.0] (also support the deprecated name “progress” for this event, which is how it currently works for progress-edx-platform-extensions), and then mark it as completed.

   1. We will create a mixin to provide a template for custom completion, and tools for making it more simple, with has_custom_completion set to True, and an emit_completion method.

3. Otherwise, if the XBlock indicates that it is gradable (“has_score” is true): when the XBlock emits a grade, have a signal handler emit a completion event.

   1. An XBlock can also emit a “completion” event with a value of 0.0, to indicate that the user has no longer completed the XBlock.

4. Otherwise: when the XBlock is first viewed by the student (student_view is loaded and rendered), mark it as completed.

   1. The code to emit the event for default blocks will live in the "vertical" XBlock JS, so that when a unit becomes visible, it emits the "complete" event for every descendent that does not match the criteria above (that is, when completion_method == COMPLETABLE && has_custom_completion == False && has_score == False).

   2. A block will be marked complete when it has been visible in the browser window for N seconds, or has had focus for N seconds (to account for users whose interface does not contain a view window).
      

2. When the completion subsystem receives an emitted completion event, it will check if the block has track_completion set to False, and otherwise, it will update_or_create the relevant BlockCompletion object.

3. Whenever the "completed" value of an XBlock change, the LMS will call an asynchronous task to update the corresponding AggregateCompletion table rows. The asynchronous task request will contain a modified timestamp of the completed block and will check the modified timestamp on those rows to verify they are out of date before performing an update. (This task must not be processed until after the completable block update is committed).

   1. This may require optimization to avoid computing aggregators too frequently.
4. Any addition or removal of completable blocks from a subsection or course should cause cached "possible" values to be recomputed asynchronously for all enrolled users, using the same mechanism as for changed completion values, above, though the user will be less sensitive to stale values in this situation.
   
   1. TODO: Figure out all the conditions that will require recomputing completion values.
      1. Created or removed blocks.
      2. Visibility changes.

5. If a user completes blocks while offline (this could be fairly common in a phone app, but is also possible in a browser), they will need to see their completion updated when they come back online.  This is being explored further in a separate proposal for Offline XBlock support.  Some draft ideas:
   

1. There could be an API endpoint that can accept a mapping of block IDs to completion ratios, and then the app can mark blocks as completed for the current user when they come back online.

2. Alternative: Allow both JavaScript XBlock runtimes and native iOS/Android XBlock views to emit events (“complete”, “submit”, etc.) somewhat like current handler calls, but asynchronous and offline-compatible. When the app is online again, submit all the timestamped events to the LMS server. Any XBlocks that received a “complete” event will be marked as complete. (This allows the definition of completion to be controlled by the XBlock, e.g. in offline webviews, rather than requiring the native app to code its own definition of complete for each XBlock).