Deadline for comments: July 22, 2015
Status of the document: Proposal
Collaborators: Jim Abramson (Deactivated)

Overview

In-course reverification is a feature that allows students to verify their identities at checkpoints within a course. Currently, the feature is implemented as an XBlock that course authors can add to a course. The XBlock allows students to enter a "reverification" flow in which students can submit photos of their faces. These photos are sent to an identity verification service (Software Secure), which approves or denies the user’s verification attempt.

The diagram below shows how a student transitions between different verification states at an in-course reverification checkpoint:

In the diagram:

The "photo verification service" is provided by an external vendor called Software Secure.
At the reverification checkpoint, users submit photos of their faces, which the photo verification service compares with photos of the user’s government-issued photo ID. Users submits ID photos during an "initial" verification process after enrolling in a course.
Users are allowed to resubmit (shown as a dashed line) only if the assessment is open and the user has not exceeded the attempt limit, which is configurable per-checkpoint.

In its current implementation, in-course reverification does not prevent users from accessing exam content before they submit photos for verification. Before we release this feature, we need to introduce an access control mechanism that can handle the following cases.

ICRV controls access to exam content.

When a student reaches an ICRV checkpoint, the student does not have access to exam content.
When a student skips (by clicking the "skip" button in the ICRV XBlock) or submits a reverification attempt (by completing the ICRV reverification flow), the user is allowed access to exam content.

Enrollment mode controls access to ICRV. When the student is not enrolled as "verified", all ICRV blocks should be hidden and the user should have access to all exam content.
ICRV controls access to other ICRV blocks. When the student has failed or skipped an ICRV verification, all ICRV blocks should be hidden and the user should have access to all exam content. (Edit: confirmed with product that it's okay for all ICRV blocks to be hidden, not just "later" ICRV blocks)

This document presents a design that satisfies these requirements. The design attempts to balance:

Course Author UX: It should be easy for course authors to add in-course reverification checkpoints to a course. In slightly more pessimistic terms, ICRV should be difficult to misconfigure.
Encapsulation: As much as possible, business logic specific to the verification process should be encapsulated from the core of the system. One way to achieve this is to re-use existing access control mechanisms, which are likely to provide better interoperability, performance, and security while minimizing development time and complexity.

Proposal

We propose a solution built on group-based access control for courseware content:

Implement a partition scheme that assigns users to groups based on their verification status and enrollment mode.
For each in-course reverification XBlock in a course, define a partition with three groups:

non_verified: The ReverificationBlock is hidden, and the exam content is visible.
verified_allow: Both the ReverificationBlock and exam content are visible.
verified_deny: The ReverificationBlock is visible, and the exam content is hidden.

Set the allowed groups for the ICRV XBlock to ["verified_allow", "verified_deny"]
Set the allowed groups for gated content to ["non_verified", "verified_allow"].

To minimize course author configuration, steps (2), (3), and (4) occur automatically when a version of the course is published.

To illustrate this, we consider an example course after it has been published:

In the diagram, ReverificationBlock has groups ["verified_allow", "verified_deny"], so it will be hidden for users not in the verified track and shown for users in the verified track (regardless of their verification status).

The sibling Vertical has groups ["non_verified", "verified_allow"], so they will be hidden to users who are in the verified track but have not yet submitted/skipped verification for this checkpoint.

Creating User Partitions (On Publish)

On course publish, new user partitions will be added to the course for each in-course reverification checkpoint. If there are existing partitions for checkpoints that have been deleted, they will be removed.

Reverification checkpoints are uniquely identified within a course by the location of the ReverificationBlock. To minimize course configuration outside of Studio, we propose extending the "User Partition" schema to include a new dictionary field called "parameters". This field can be used to pass information to the partition scheme -- in this case, we will use it to pass the location of the ReverificationBlock so the partition scheme can locate the associated reverification checkpoint.

Each user partition will be assigned a unique ID of the form "verification:{location}". The "verification" prefix is meant to avoid namespace collisions with other user partitions defined in the course.

For example, the user partition for an in-course reverification checkpoint might look like this:

{
    "version": 3,
	"id": "verification:block-v1:edX+DemoX+Demo+type@reverification+block@f02a238efac644e389e930fbf275953e",
	"name": "Verification Checkpoint for Midterm A",
	"description": "Verification Checkpoint for Midterm A",
	"scheme": "verification",
    "parameters": {
		"location": "block-v1:edX+DemoX+Demo+type@reverification+block@f02a238efac644e389e930fbf275953e"
    },
    "groups": [
         {
             "version": 1,
             "id": "non_verified",
             "name": "Not enrolled in a verified track",
         },
         {
	         "version": 1,
	         "id": "verified_allow",
	         "name": "Enrolled in a verified track and has access"
         },
         {
	         "version": 1,
	         "id": "verified_deny",
	         "name": "Enrolled in a verified track and does not have access"
         }
	]
}

It is important that modifications to the course’s user partitions within the publish signal handler do NOT trigger an additional publish signal. Otherwise, the signal receiver would invoke itself indefinitely.

Configuring Groups (On Publish)

On publish, the following modifications will be made to group_access values of course content:

Every ReverificationBlock will have group_access for the associated user partition set to [“verified_allow”, “verified_deny”]
Set group_access for the associated user partition set to [“non_verified”, “verified_allow”] for

All siblings of the ReverificationBlock.
Every sibling of the parent Vertical, but only if the ReverificationBlock’s parent is a Vertical and its parent is a Subsection (sequential)

These steps will configure groups correctly in most cases. There are two cases that can lead to unusual behavior:

If the ReverificationBlock is part of a content experiment, its grandparent will not be a Subsection, so it will control access to its siblings within the experiment, but not any surrounding content. This is arguably a reasonable way to handle this case.
If two or more ReverificationBlocks are children of the same Vertical or grandchildren of the same Subsection, then users in a verified track will not be able to see either block (or any of the content). Intuitively, reverification checkpoint #1 prevents access to checkpoint #2, and vice-versa. While potentially confusing, there is at least an easy way for course authors to fix the issue: delete all but one of the ReverificationBlocks in a particular subsection.

We can potentially mitigate these issues by enforcing additional validation checks to Studio. These validation checks would restrict where course authors can add ReverificationBlocks to a course. If possible, we would like to avoid doing this, however, since it introduces significant complexity to Studio. While not ideal, we feel that the default behavior in these cases is reasonable.

Partition Scheme

The LMS supports pluggable partition schemes. We will define a new partition scheme that places users into one of the three groups as follows:

def get_group_for_user(cls, course_key, user, user_partition):
    checkpoint = user_partition.parameters[“location”]

	if (
        not _is_enrolled_in_verified_mode(user, course_key) or
        _was_denied_at_any_checkpoint(user, course_key)
    ):
		return NON_VERIFIED_GROUP
    elif (
         _has_skipped_any_checkpoint(user, course_key) or
         _has_completed_checkpoint(user, course_key, checkpoint)
    ):
      	return VERIFIED_ALLOW_GROUP
    else:
	    return VERIFIED_DENY_GROUP

All of the information required for these checks is already available through existing SQL tables:

Enrollment mode is available from the CourseEnrollment table (in the student app).
When a user skips verification for a checkpoint in a course, a SkippedReverification record is created (in the verify_student app).
A user’s verification status at a checkpoint is stored in a VerificationStatus table, with a foreign key to a VerificationCheckpoint table (in the verify_student app).

Preview

Course staff need to be able to preview exam content that is gated by an in-course reverification. Our existing access control logic will automatically grant global and course staff access to the ICRV XBlock and exam content regardless of the user's group. In other words, we get the desired behavior for course staff with no additional development effort.

Messaging for Blocked Content

Currently, when a user is denied access to courseware content, the content is completely hidden from the user. One potential enhancement would be for the access checks to provide additional messaging in certain cases to explain to users why they were blocked from accessing content. This proposal does not address messaging for blocked content, except to note that it is compatible with this approach.

There is currently no way to display a “disabled” version of blocked content (for example, displaying the content as “grayed out” with functionality disabled). It is unclear how we could support this functionality without significant changes to the LMS.

Performance

Access Checks

The main performance risk with this approach is that the user partition scheme will be called whenever a user tries to load a courseware component. Unlike with cohorts or content experiments, a user’s group can change as the user’s verification and enrollment states change. For this reason, we need a slightly more sophisticated cache invalidation strategy.

We propose caching the following information:

Cache Key	Value	Invalidation Strategy
enrollment.{username}.{course_key}.mode	The user’s current enrollment mode.	Invalidate on save to the CourseEnrollment model.
verification.{username}.{course_key}	The user’s verification status for each checkpoint in the course (a dictionary)	Invalidate on save to VerificationStatus

We expect a high cache hit rate as the user navigates content within a course. A cold cache should result in an additional four database queries (to the four SQL tables mentioned in “Partition Scheme” above). Once the cache is populated for a particular user in a course, there should be one multi-get cache request and no additional database queries.

Course Publish

The proposed approach requires querying and modifying course content on publish. To avoid a potential race condition, this operation will be performed synchronously on publish. Publishing in Studio happens frequently, so these modulestore operations need to be efficient. We can optimize by retrieving all in-course reverification blocks, then following “parent” and “children” links to find the blocks we need to update.

We will need to test the performance of this operation on a course with many components. If the operation is slow enough that it negatively impacts the course authoring experience, we have the option of making the update asynchronously. This introduces at least two potential race conditions:

Students can access gated content after it is published but before the asynchronous task updates it.
Multiple worker processes attempt to update a course at the same time, overwriting changes from other processes.

Addressing these issues would likely require introducing a locking mechanism to coordinate task execution among workers, which would involve additional design, development, and testing.

Alternative Approaches

Parent “Exam” Block

One alternative we considered was to introduce a new navigational XBlock for representing exam content. Assessment content would be added as children of the parent exam block. The exam block could then query the user’s permission to access exam content through an XBlock service and dynamically show or hide its children.

Although we believe that this approach is technically feasible, we feel that there is value in using a more constrained API for access control. To determine the structure of a course with XBlocks that have dynamic children, the XBlocks must be instantiated. This makes it difficult to represent and reason about the course structure. It also prevents future optimizations such as caching the course structure for users in a particular group.

Milestones

Another alternative we considered was to use the Milestones microservice. This service is currently used to implement entrance exams (prevent access to a course until the user passes an exam) and pre-requisite courses (prevent access to a course until the user completes a prerequisite course).

Milestones provide an intuitive model for binary states: for example, a user has either completed a prerequisite exam or the user hasn’t. However, verification states are not binary -- this is why we introduced three partition groups when modeling the problem as user partitions. It may be possible to model these states using multiple milestones, but this introduces significant complexity.

In addition, milestones require that we keep the user’s acquired milestones in sync with the user’s verification and enrollment states. This could introduce errors that are difficult to prevent, diagnose, and fix. In contrast, user partition schemes dynamically calculate the user’s group based on the user’s verification and enrollment state, so no synchronization is necessary.

Linearized Navigation

We considered performing a depth-first search of the course structure and saving the traversal order to a database table. We could then add access checks of the form: “restrict access to every component after location X”.

We ultimately rejected this approach after reviewing the courseware navigation API proposed by the mobile team. Enforcing a linear order on courseware content would make it difficult to support different navigation strategies, especially for adaptive learning.

Architecture and Engineering

In-Course Reverification Access Control