Deadline for comments: July 22, 2015
Status of the document: Proposal
Collaborators: Jim Abramson (Deactivated)
...
The diagram below shows how a student transitions between different verification states at an in-course reverification checkpoint:
In the diagram:
The "photo verification service" is provided by an external vendor called Software Secure.
At the reverification checkpoint, users submit photos of their faces, which the photo verification service compares with photos of the user’s government-issued photo ID. Users submits ID photos during an "initial" verification process after enrolling in a course.
Users are allowed to resubmit (shown as a dashed line) only if the assessment is open and the user has not exceeded the attempt limit, which is configurable per-checkpoint.
In its current implementation, in-course reverification does not prevent users from accessing exam content before they submit photos for verification. Before we release this feature, we need to introduce an access control mechanism that can handle the following cases.
ICRV controls access to exam content.
When a student reaches an ICRV checkpoint, the student does not have access to exam content.
When a student skips (by clicking the "skip" button in the ICRV XBlock) or submits a reverification attempt (by completing the ICRV reverification flow), the user is allowed access to exam content.
Enrollment mode controls access to ICRV. When the student is not enrolled as "verified", all ICRV blocks should be hidden and the user should have access to all exam content.
ICRV controls access to other ICRV blocks. When the student has failed or skipped an earlier ICRV verification, all later ICRV all ICRV blocks should be hidden and the user should have access to all exam content. (Edit: confirmed with product that it's okay for all ICRV blocks to be hidden, not just "later" ICRV blocks)
This document presents a design that satisfies these requirements. The design attempts to balance:
Course Author UX: It should be easy for course authors to add in-course reverification checkpoints to a course. In slightly more pessimistic terms, ICRV should be difficult to misconfigure.
Encapsulation: As much as possible, business logic specific to the verification process should be encapsulated from the core of the system. One way to achieve this is to re-use existing access control mechanisms, which are likely to provide better interoperability, performance, and security while minimizing development time and complexity.
Proposal
We propose a solution built on group-based access control for courseware content:
Implement a partition scheme that assigns users to groups based on their verification status and enrollment mode.
For each in-course reverification XBlock in a course, define a partition with three groups:
non_verified: The ReverificationBlock is hidden, and the exam content is visible.
verified_allow: Both the ReverificationBlock and exam content are visible.
verified_deny: The ReverificationBlock is visible, and the exam content is hidden.
Set the allowed groups for the ICRV XBlock to ["verified_allow", "verified_deny"]
Set the allowed groups for gated content to ["non_verified", "verified_allow"].
To minimize course author configuration, steps (2), (3), and (4) occur automatically when a version of the course is published.
To illustrate this, we consider an example course after it has been published:
In the diagram, ReverificationBlock has groups ["verified_allow", "verified_deny"], so it will be hidden for users not in the verified track and shown for users in the verified track (regardless of their verification status).
The sibling Vertical has groups ["non_verified", "verified_allow"], so they will be hidden to users who are in the verified track but have not yet submitted/skipped verification for this checkpoint.
Creating User Partitions (On Publish)
On course publish, new user partitions will be added to the course for each in-course reverification checkpoint. If there are existing partitions for checkpoints that have been deleted, they will be removed.
Reverification checkpoints are uniquely identified within a course by the location of the ReverificationBlock. To minimize course configuration outside of Studio, we propose extending the "User Partition" schema to include a new dictionary field called "parameters". This field can be used to pass information to the partition scheme -- in this case, we will use it to pass the location of the ReverificationBlock so the partition scheme can locate the associated reverification checkpoint.
Each user partition will be assigned a unique ID of the form "verification:{location}". The "verification" prefix is meant to avoid namespace collisions with other user partitions defined in the course.
For example, the user partition for an in-course reverification checkpoint might look like this:
| Code Block | ||
|---|---|---|
| ||
{
"version": 3,
"id": "abcd1234verification:block-v1:edX+DemoX+Demo+type@reverification+block@f02a238efac644e389e930fbf275953e",
"name": "Verification Checkpoint for Midterm A",
"description": "Verification Checkpoint for Midterm A",
"scheme": "verification",
"parameters": {
"location": "block-v1:edX+DemoX+Demo+type@reverification+block@f02a238efac644e389e930fbf275953e"
},
"groups": [
{
"version": 1,
"id": "non_verified",
"name": "Not enrolled in a verified track",
},
{
"version": 1,
"id": "verified_allow",
"name": "Enrolled in a verified track and has access"
},
{
"version": 1,
"id": "verified_deny",
"name": "Enrolled in a verified track and does not have access"
}
]
} |
Configuring Groups (On Publish)
Every ReverificationBlock will have group_access for the associated user partition set to [“verified_allow”, “verified_deny”]
The following blocks will have Set group_access for the associated user partition set to [“non_verified”, “verified_allow”] for
Include all All siblings of the ReverificationBlock.
If the Every sibling of the parent Vertical, but only if the ReverificationBlock’s parent parent is a Vertical and its parent is a a Subsection (sequential), then include every sibling of the parent Vertical.
These steps will configure groups correctly in most cases. There are two cases that can lead to unusual behavior:
If the ReverificationBlock is part of a content experiment, its grandparent will not be a Subsection, so it will control access to its siblings within the experiment, but not any surrounding content. This is arguably a reasonable way to handle this case.
If two or more ReverificationBlocks are children of the same Vertical or grandchildren of the same Subsection, then users in a verified track will not be able to see either block (or any of the content). Intuitively, reverification checkpoint #1 prevents access to checkpoint #2, and vice-versa. While potentially confusing, there is at least an easy way for course authors to fix the issue: delete all but one of the ReverificationBlocks in a particular subsection.
Partition Scheme
The LMS supports pluggable partition schemes. We will define a new partition scheme that places users into one of the three groups as follows:
| Code Block | ||
|---|---|---|
| ||
def get_group_for_user(cls, course_key, user, user_partition):
checkpoint = user_partition.parameters[“location”]
if (
not _is_enrolled_in_verified_mode(user, course_key) or
_was_denied_at_otherany_checkpoint(user, course_key, location)
):
return NON_VERIFIED_GROUP
elif (
_has_skipped_any_checkpoint(user, course_key) or
_has_completed_checkpoint(user, course_key, checkpoint)
or _has_preview_permission(user, course_key)
):
return VERIFIED_ALLOW_GROUP
else:
return VERIFIED_DENY_GROUP |
All of the information required for these checks is already available through existing SQL tables:
Enrollment mode is available from the CourseEnrollment table (in the student app).
When a user skips verification for a checkpoint in a course, a SkippedReverification record is created (in the verify_student app).
A user’s verification status at a checkpoint is stored in a VerificationStatus table, with a foreign key to a VerificationCheckpoint table (in the verify_student app).Certain users (e.g. course staff) will be given permission to preview the exam content without needing to submit a verification attempt. A user’s permission can be calculated from the user’s roles. (The mapping from roles to permissions would be handled by a separate module, not within the verify_student app.)
Preview
Course staff need to be able to preview exam content that is gated by an in-course reverification. We enable this by automatically placing users with the appropriate permission into the “verified_allow” group. This will allow course staff to view both the ReverificationBlock and associated content without requiring them to enroll as verified or submit photos.It would also be possible to allow users with the appropriate permission to force themselves into a particular group. This could be similar to how we allow course staff to preview different groups in content experiments Our existing access control logic will automatically grant global and course staff access to the ICRV XBlock and exam content regardless of the user's group. In other words, we get the desired behavior for course staff with no additional development effort.
Messaging for Blocked Content
Currently, when a user is denied access to courseware content, the content is completely hidden from the user. One potential enhancement would be for the access checks to provide additional messaging in certain cases to explain to users why they were blocked from accessing content. This proposal does not address messaging for blocked content, except to note that it is compatible with this approach.
There is currently no way to display a “disabled” version of blocked content (for example, displaying the content as “grayed out” with functionality disabled). It is unclear how we could support this functionality without significant changes to the LMS.
Performance
Access Checks
The main performance risk with this approach is that the user partition scheme will be called whenever a user tries to load a courseware component. Unlike with cohorts or content experiments, a user’s group can change as the user’s verification and enrollment states change. For this reason, we need a slightly more sophisticated cache invalidation strategy.
We propose caching the following information:
| Cache Key | Value | Invalidation Strategy |
|---|---|---|
enrollment.{username}.{course_key}.mode | The user’s current enrollment mode. | Invalidate on save to the CourseEnrollment model. |
verification.{username}.{course_key} | The user’s verification status for each checkpoint in the course (a dictionary) | Invalidate on save to VerificationStatus |
We expect a high cache hit rate as the user navigates content within a course. A cold cache should result in an additional four database queries (to the four SQL tables mentioned in “Partition Scheme” above). Once the cache is populated for a particular user in a course, there should be one multi-get cache request and no additional database queries.
Course Publish
The proposed approach requires querying and modifying course content on publish. To avoid a potential race condition, this operation will be performed synchronously on publish. Publishing in Studio happens frequently, so these modulestore operations need to be efficient. We can optimize by retrieving all in-course reverification blocks, then following “parent” and “children” links to find the blocks we need to update.
We will need to test the performance of this operation on a course with many components. If the operation is slow enough that it negatively impacts the course authoring experience, we have the option of making the update asynchronously. This introduces at least two potential race conditions:
Students can access gated content after it is published but before the asynchronous task updates it.
Multiple worker processes attempt to update a course at the same time, overwriting changes from other processes.
Addressing these issues would likely require introducing a locking mechanism to coordinate task execution among workers, which would involve additional design, development, and testing.
Alternative Approaches
Parent “Exam” Block
One alternative we considered was to introduce a new navigational XBlock for representing exam content. Assessment content would be added as children of the parent exam block. The exam block could then query the user’s permission to access exam content through an XBlock service and dynamically show or hide its children.
Although we believe that this approach is technically feasible, we feel that there is value in using a more constrained API for access control. To determine the structure of a course with XBlocks that have dynamic children, the XBlocks must be instantiated. This makes it difficult to represent and reason about the course structure. It also prevents future optimizations such as caching the course structure for users in a particular group.
Milestones
Another alternative we considered was to use the Milestones microservice. This service is currently used to implement entrance exams (prevent access to a course until the user passes an exam) and pre-requisite courses (prevent access to a course until the user completes a prerequisite course).
Milestones provide an intuitive model for binary states: for example, a user has either completed a prerequisite exam or the user hasn’t. However, verification states are not binary -- this is why we introduced three partition groups when modeling the problem as user partitions. It may be possible to model these states using multiple milestones, but this introduces significant complexity.
In addition, milestones require that we keep the user’s acquired milestones in sync with the user’s verification and enrollment states. This could introduce errors that are difficult to prevent, diagnose, and fix. In contrast, user partition schemes dynamically calculate the user’s group based on the user’s verification and enrollment state, so no synchronization is necessary.
Linearized Navigation
We considered performing a depth-first search of the course structure and saving the traversal order to a database table. We could then add access checks of the form: “restrict access to every component after location X”.
We ultimately rejected this approach after reviewing the courseware navigation API proposed by the mobile team. Enforcing a linear order on courseware content would make it difficult to support different navigation strategies, especially for adaptive learning.