Overview

The goal of this project is to replace the current vendor for identity verification (IDV) on edx.org, Software Secure, with a new vendor, Persona.

We propose leveraging the various extensibility mechanisms that the platform supports, but we need guidance around how “pluggable” the solution must be and what IDV-related community needs must be met, particularly since 2U uses IDV for a smaller set of features than the platform currently supports.

Problem

The backend implementation of IDV is in the edx-platform verify_student Django application, and the frontend implementation of IDV and the Name Affirmation feature that uses IDV is in the frontend-app-learning MFE and frontend-app-account MFE, respectively. The backend implementation of Name Affirmation is in the edx-name-affirmation plugin. The first three repositories are in the openedx GitHub organization and are a part of the Open edX platform. edx-name-affirmation is in the edx GitHub organization but it is installed into the platform.

The problem that we are solving is adding the necessary extensibility mechanism to the platform to make IDV generic such that additional IDV implementations can be added to the platform seamlessly and without disruption to the core.

Use Cases

As an Open edX operator or developer, I do not want to have to navigate through vendor or implementation specific IDV code in the core.
As an Open edX operator or developer, I need to be able to integrate an IDV implementation into the platform with no disruption to other Open edX instances or the core, because I need to verify learners' identities. I need to be able to integrate whichever IDV implementation that suits my needs.
As a learner who has earned a certificate, I need to be able to verify my identity after requesting a name change, because this validates that I am who I say I am. This maintains the value of a certificate and ensures that the certificate is attributable to me.
As a learner who has purchased a verifiable course mode, I need to be able to verify my identity so that I can receive a “verified certificate” when I meet the requirements for earning one.

Proposed Solution

Our proposed solution involves three categories of changes.

Backend Extensibility Mechanisms

Generic Model

In order for the platform to continue to function with respect to the edx-platform verify_student Django application while also enabling integration with other implementations of IDV, the platform needs a way to store generic, implementation-agnostic information about IDV attempts.

We will introduce a new model VerificationAttempt to store this data. This model will hook into the IDVerificationService to expose these attempts across the platform.

The model will likely contain the following fields.

created
modified
user
name
status
expiration_datetime

The status field will have four options - created, pending, approved, and denied. These statuses correspond to implementation-agnostic statuses that are sensible for use in the core.

Hooks

We propose using the hooks extension framework to integrate additional IDV implementations into the edx-platform verify_student Django application.

Filter Hooks

We propose integrating a filter hook into the IDVerificationService. The IDVerificationService serves as an interface to the verify_student Django application and is used throughout the platform. Callers of this service primarily use it to read data from the IDV related models and to get the URL to the IDV flow.

We propose the following filter, which will enable full integration of the IDVerificationService with additional IDV implementations.

url = IDVerificationURLRequested(url)

This filter hook will be used to fetch the URL to the IDV flow. This will allow operators to override the URL to the IDV flow.

Event Hooks

We propose introducing event hooks related to the IDV attempt lifecycle. Taking the EXAM_ATTEMPT_* Open edX events as a model, we propose introducing one Open edX event per status of the generic VerificationAttempt model.

IDV_ATTEMPT_CREATED
IDV_ATTEMPT_PENDING
IDV_ATTEMPT_APPROVED
IDV_ATTEMPT_DENIED

The actual shape of the event data could be determined later.

Frontend Extensibility Mechanisms

We propose using the frontend-plugin-framework to integrate additional IDV implementations into the frontend-app-account MFE.

The current IDV flow is hosted in the frontend-app-account MFE. Although it works well, it is incompatible with Persona’s IDV flow. The Persona application collects all PII, government ID photos, and selfie photos (i.e. portrait photos) directly in the application, but edX’s current IDV flow performs all of those functions and submits the collected photos to Software Secure via the LMS.

For this reason, we need to replace the current IDV flow. In order to do this without modifying the current IDV flow hosted in the frontend-app-account MFE, we propose the addition of a PluginSlot from the frontend-plugin-framework to the frontend-app-account MFE. This PluginSlot would wrap the use of the IdVerificationPage in the id-verification Route. This would allow us to replace the entire IDV component with a new Persona integration component without needing to modify the route or the router.

There are a few other options for where to add a PluginSlot. These are documented in Alternative PluginSlot Locations.

Sample code is shown below for illustrative purposes.

const IdVerificationPageSlot = () => (
  <PluginSlot
    id="id_verification_slot"
    pluginProps={{
      courseId,
    }}
  />
    <IdVerificationPage />
  </PluginSlot>
)
...
subscribe(APP_READY, () => {
  ReactDOM.render(
    ...
    <Route path="/notifications/:courseId" element={<NotificationPreferences />} />
    <Route path="/notifications" element={<NotificationCourses />} />
    <Route path="/id-verification/*" element={<IdVerificationPageSlot />} />
    <Route path="/" element={<AccountSettingsPage />} />
    <Route path="/notfound" element={<NotFoundPage />} />
    ...
  );
});

Refactoring

2U uses IDV on edx.org for the Name Affirmation feature, which verifies certain name changes requested by certificate-bearing learners using IDV. This is because a name change has the potential to change the name displayed on their certificate(s). On the backend, this is implemented by the edx-name-affirmation plugin, and, on the frontend, this is implemented by the NameChange component in the frontend-app-account MFE.

IDV Attempt References

The VerifiedName model has a verification_attempt_id field, which currently stores the id field of the corresponding SoftwareSecurePhotoVerification instance used to verify a particular name. This field poses a problem for the introduction of an additional IDV implementation because the value of this field on any given row could also refer to the id field of a Persona IDV record. It would not be possible to determine whether a given id value refers to a SoftwareSecurePhotoVerification model instance or a Persona IDV record model instance.

We propose amending the IDVerificationAttempt model to add a verification_attempt field, which will be a ForeignKey field to the generic VerificationAttempt model described above.

Support Tools Verified Name Panel

The frontend-app-support-tools contains a VerifiedName panel for displaying a learner’s verified name and the history of previous verified names. In the modal that opens when viewing the history, each VerifiedName is displayed in a table, and the verification_attempt_id field described above is shown in the “IDV Attempt ID” column.

When hovering over an ID in the column, the status of the corresponding IDV attempt (i.e. of the SoftwareSecurePhotoVerification modal instance) is shown. The status is retrieved from the IDVerificationSupportView view of the user_api Django application. This view uses the get_verification_details_by_id method of the IDVerificationService, which queries the SoftwareSecurePhotoVerification, SSOVerification, and ManualVerification models by their id. This works due to inheritance from the abstract base class IDVerificationAttempt.

With the introduction of the VerificationAttempt model, an id cannot uniquely identify a instance of the VerificationAttempt, SoftwareSecurePhotoVerification, SSOVerification, and ManualVerification models. This means that the get_verification_details_by_id method of the IDVerificationService will no longer work, breaking the Support Tools.

As a solution, the VerifiedName panel will be refactored to pull the IDV attempt status from the edx-name-affirmation application. The edx-name-affirmation Django application will refactor its VerifiedNameHistoryView view to pull this data by querying the model associated with either the verification_attempt_id or verification_attempt fields. As a result, the use of the VerifiedNameHistoryView view and the get_verification_details_by_id method of the IDVerificationService can be removed.

Name Change Signal Handler

Currently, the edx-name-affirmation Django application associates VerifiedNames with IDV attempts via the verification_attempt_id field. This association is formed by a task that is triggered by a signal handler that listens for a signal emitted by the SoftwareSecurePhotoVerification model. This means that only Software Secure IDV attempts can change the status of VerifiedNames (e.g. approve or deny them).

We propose triggering this task on the event hooks listed above. The impact of this change is that any form of IDV can change the status of VerifiedNames (e.g. approve or deny them). For example, an instance of the ManualVerification class can be used to approve or deny a VerifiedName.

To handle deletes, the idv_delete_handler will be connected to the VerificationAttempt post_delete signal as it is for the SoftwareSecurePhotoVerification model.

Name Change Modal

We propose modifying the copy in the name change modal to make the instructions more generic. Some IDV implementations do not support non-government IDs.

We propose the following text.

Enter your name as it appears on your identification card.

Other Approaches Considered

Backend

As alternative approaches, we considered Software Secure IDV deprecation, an edx-platform fork, and direct edx-platform modification

IDV Deprecation

The current implementation of IDV using Software Secure has an unclear purpose in the platform. As is, it is our understanding that only 2U uses this integration. Based on relatively old threads in the Open edX Discuss forums, a few operators do use the Software Secure implementation but with modifications to suit their use cases (e.g. bypassing Software Secure to provide the ability for site operators to manually review photos). 2U, as the sole known user of this integration as it is currently implemented, no longer needs this integration, which could indicate deprecation of this integration.

Additional Discuss Forum Posts

Although Software Secure could be deprecated, the deprecation is not necessary to enable integration of additional IDV implementations. In fact, we believe that many, if not all, of the same extensibility mechanisms described above would be required. Additionally, because the Name Affirmation feature relies on Software Secure IDV, removal of Software Secure from the platform would also require us to consider how to remove the Name Affirmation feature. We fear that this will massively expand the scope of this project.

Besides Software Secure, we believe that the concept of “IDV” as a whole should be maintained, at least for now. There are other forms of IDV (i.e. manual, SSO, etc.) that the platform does use. Also, the concept of a “verified certificate” still exists in the platform and is used by the community. We believe that the deprecation of “IDV” as a concept would require far more work than we can afford and isn’t in direct service of our integration with Persona.

We believe that the proposal above will enable us to integrate with Persona without needing to perform any deprecation.

edx-platform Fork

We considered whether it was time for 2U to fork the edx-platform and make the necessary changes to the fork to integrate with Persona. We decided that this was an inappropriate direction. It is a very consequential decision, and the ability to integrate another IDV provider could be accomplished using extensibility mechanisms without resorting to a fork.

edx-platform Modification

We considered whether we could implement the Persona integration in the platform and provide a toggle to select between Software Secure and Persona as the platform IDV provider. However, this was largely a theoretical option. We understand that we must not commit 2U-specific code to the Open edX platform.

Frontend

As alternative approaches, we considered alternative PluginSlot locations, a server-side returned URL, a frontend-app-account MFE fork, and direct frontend-app-account MFE modification.

Alternative PluginSlot Locations

There are a few additional options for where to include a PluginSlot to allow integration with another IDV implementation, all of which relate to the use of the router that the frontend-app-account MFE uses for client-side routing.

Both options have significant issues compared to wrapping the IDVerificationPage component in a PluginSlot, as described in the proposal above.

Adding Routes - PluginSlot as Sibling

The PluginSlot component could be a sibling of the current routes, which would allow us to add an additional route easily, but it would also mean hosting two IDV flows at different URLs.

It would also require us to be able to configure the route that the Name Affirmation feature uses when redirecting a learner to IDV, since the id-verification route would already be reserved for the current Software Secure IDV flow.

<Route path="/notifications/:courseId" element={<NotificationPreferences />} />
<Route path="/notifications" element={<NotificationCourses />} />
<Route path="/id-verification/*" element={<IdVerificationPage />} />
<Route path="/" element={<AccountSettingsPage />} />
<Route path="/notfound" element={<NotFoundPage />} />
<Route path="*" element={<NotFoundPage />} />
<PluginSlot
  id="routes_slot"
  pluginProps={{
    courseId,
  }}
/>

Adding Routes - PluginSlot as Parent

The PluginSlot component could be a parent of the current routes. This option would allow us to have a single IDV flow and to reuse the existing id-verification route. This would result in no necessary changes to the router or the route that the Name Affirmation feature uses. However, this option would make management of the existing non-IDV routes more challenging. It would require duplication and maintenance of the existing non-IDV routes on http://edx.org .

<PluginSlot
  id="routes_slot"
  pluginProps={{
    courseId,
  }}
/>
  <Routes>
    <Route path="/notifications/:courseId" element={<NotificationPreferences />} />
    <Route path="/notifications" element={<NotificationCourses />} />
    <Route path="/id-verification/*" element={<IdVerificationPage />} />
    <Route path="/" element={<AccountSettingsPage />} />
    <Route path="/notfound" element={<NotFoundPage />} />
  <Routes>
</PluginSlot>

Server-Side Returned URL

In this option, a backend API on the LMS would return a Persona URL. The IDV URL is already exposed via the CoursewareMeta view, so a Django setting for the IDV URL could be introduced, which would be returned by the IDVerificationService.get_verify_location method.

However, the current use of the router that the frontend-app-account MFE uses for client-side routing does not allow routing to non-application pages. This would be need to refactored to allow this kind of navigation.

This option could work, but it results in changes to the platform that do not feel very extensible or reusable.

frontend-app-account MFE Fork

This approach is analogous to the edx-platform Fork option but for the frontend. The same judgments apply to this approach.

frontend-app-account MFE Modification

This approach is analogous to the edx-platform Modification option but for the frontend. The same judgments apply to this approach.

Competitive Research

This proposal is concerned with adding the extensibility mechanisms necessary to integrate another form of IDV into the platform, which is an Open edX platform problem, so we did not perform any competitive research.

Implementation Plan

2U will be responsible for implementation and delivery. This project is schedule to start as soon as possible. We estimate around 6-8 weeks for implementation.

Long-Term Ownership/Maintainership

The Cosmonauts team at 2U is the owning team of the IDV feature. The Cosmonauts team would continue to own and maintain the feature, complete with the aforementioned extensibility mechanisms.

Discovery

In a pluggable solution, what IDV features or integrations into the platform must be supported?
Are there any IDV features or integrations that can be ignored or not considered or that are planned to deprecation that we should be aware of?
How does the community use IDV? I asked in a Discuss thread but did not receive very much feedback.
What is the longterm role of IDV in the edx-platform?
Is it acceptable to build out extensibility mechanisms that are sufficient enough only to enable an integration with Persona and that may not support Software Secure or other IDV vendors?
- For example, in the certificates generation logic, we may need to add a filter to “collect IDV attempts” so that we can add in our Persona records. 2U does not gate certificate generation on IDV, but some operators may choose to leverage that code path via the ENABLE_CERTIFICATES_IDV_REQUIREMENT Django setting. This means we may not need to add a filter to enable that certificates behavior.
Is there guidance for where it is acceptable to use a PluginSlot on the frontend? What requirements exist? Can it be inserted anywhere?

Appendix

Uses of IDV on the Open edX Platform

Goal

The goal of this section is to identify dependencies on IDV across the Open edX platform.

Summary

Feature	Notes	Opportunity For Deprecation?

Feature	Notes	Opportunity For Deprecation?
Proctoring Requirements Email	The `id_verification_url` variable is not used in the proctoring requirements email template.	Yes.
PayAndVerifyView Legacy Upgrade and Verify Flow		The PayAndVerifyView likely cannot be deprecated, but the use of the legacy IDV flow could be deprecated.
VerificationStatusAPIView	This is called by the Account MFE, but the data returned by the view does not appear to be used by the application.	Yes.
ReverifyView	This view does not appear to be referenced.	Yes.
Certificates	Certificates still make use of IDV to gate the generation of certificates.	Yes and no. This feels like a pretty core part of the platform, currently. It is gated by the `ENABLE_CERTIFICATES_IDV_REQUIREMENT` feature condition. The use of the `SoftwareSecurePhotoVerification` model could be removed, however.
Learner Dashboard (Legacy)	The IDVerificationService is used to define a set of IDV related variables that are passed to the dashboard template as context but not used.	Yes and no. This dependency could be deprecated on the dashboard course listing, but it would depend on Open edX needs, because Open edX installations may specify separate templates via the filter hook that could use these variables. However, the course verification status could be a required part of the platform.
Support Tools: Program Enrollments	The Program Enrollments support tool references IDV.	Yes. The legacy support tools application could be deprecated in favor of the MFE-based support tools application and the IDV references could removed, as they are not in use in the MFE-based application.
Support Tools: ID Verification Details	The Verified Name history modal references IDV.	No. This dependency cannot be deprecated, because it is used by the MFE-based support tools application to get information about the IDV attempt associated with a verified name. However, IDVerificationStatusView/getUserVerificationStatus and IDVerificationStatusDetailsView/getUserVerificationDetail could be deprecated.
Date Summaries		This dependency could be deprecated. Although it is enabled if the `ENABLE_INTERGRITY_SIGNATURE` feature is not enabled, I do not believe this is a core functionality of the Open edX platform.
Instructor Dashboard: Course Grades Report	This data could be put behind the `ENABLE_INTEGRITY_SIGNATURE` feature.	Maybe. There may be a case to be made to remove references to IDV from this report.
Instructor Dashboard: Data Download via Instructor Analytics	This data could be put behind the `ENABLE_INTEGRITY_SIGNATURE` feature.	Maybe. There may be a case to be made to remove references to IDV from this report.
Course Home Progress Tab (Current)		Maybe. If an approved IDV attempt continues to be a requirement for a certificate, this seems like a useful feature.
Course Home Progress Page (Legacy)		Maybe. If an approved IDV attempt continues to be a requirement for a certificate, this seems like a useful feature.
Courseware API	This API is used by the frontend-app-learning MFE. Further investigation is necessary into the Segment events.	Maybe. If an approved IDV attempt continues to be a requirement for a certificate, this seems like a useful feature.
Track Selection Page		No.
Configuration Dependencies		Some dependencies could be deprecated and other’s could not be. The references to Software Secure settings in the VERIFY_STUDENT edx-platform Django setting and the AUTOMATIC_VERIFY_STUDENT_IDENTITY_FOR_TESTING feature toggle could be deprecated.
Indirect Dependencies - Name Affirmation	The VerifiedName model has an indirect dependency on the SoftwareSecurePhotoVerification model via the verification_attempt_id field.	No. This dependency cannot be deprecated because the Name Affirmation is a feature of the edx-platform, and we will need to maintain references to IDV attempts in the VerifiedName model.
Miscellaneous Dependencies		Maybe.
frontend-app-account MFE	This is where the current IDV flow is implemented.	Yes. This dependency could be deprecated if a frontend extensibility is approved that would enable linking to an IDV flow implementation. This is because this flow is specific to the Software Secure integration.
frontend-app-learning MFE	All references to IDV are gated by the `ENABLE_CERTIFICATES_IDV_REQUIREMENT` Django setting.	No.
Support Tools	See Support Tools: Program Enrollments and Support Tools: ID Verification Details above.	Yes and no. See above.

Unverified Certificates

A certificate is can enter into the unverified status only when the ENABLE_CERTIFICATES_IDV_REQUIREMENT Django setting is turned on. Installations of Open edX may enable or disable this feature.
The ENABLE_CERTIFICATES_IDV_REQUIREMENT Django setting is used during the process of checking whether a “regular” certificate - as opposed to an “allowlist” certificate - can be created for a learner in _can_generate_certificate_common and during the process of creating an unverified certificate in _get_cert_status_common.
edX has tried to deprecate this status in the past, but there was a community need for it at the time. See this Discuss thread. Furthermore, edX deemed it too complex to make this behavior pluggable using Open edX hooks extension framework.

References to IDV in Backend Code

IDVerificationService

The IDVerificationService is an interface with the verify_student application that is used across the platform. You can think of it as a Python API that wrapped in a class. The majority of direct references to IDV occur via the use of this class, so that is where I focused my attention.

Proctoring Requirements Email

The generate_proctoring_requirements_email_context method of the email_helpers module of the edx-platform is used to collect context for proctoring requirement emails. It’s used by the update_enrollment method of the course_enrollment module of the student application, which sends proctoring requirements emails to learners when they enroll in a mode that requires proctoring (e.g. verified) in a course that uses proctoring.

The update_enrollment method uses the IDVerificationService to get the IDV URL, which is passed to the template via the context as the id_verification_url variable.

The following methods of the IDVerificationService are called.

get_verify_location

Note that IDV references were removed in the proctoring template, so the id_verification_url variable is no longer in use in the proctoring requirements email.

This dependency could be deprecated because the proctoring requirements template no longer uses the id_verification_url variable.

PayAndVerifyView

The PayAndVerifyView is an upgrade and verify flow. It was used to enroll in a verified track of a course, which, in the past, required a learner to pay to upgrade and to perform IDV to verify - hence the term “verified certificate”.

The PayAndVerifyView redirects learners to a variety of destinations across the platform depending on the state they are in with respect to track selection, payment status, verification status, and how they entered the PayAndVerifyView flow. These destinations include the learner dashboard, the checkout page, the legacy IDV flow, the frontend-app-payment MFE-based IDV flow, etc.

Legacy IDV Flow

The main concern is the legacy IDV flow. There is a set of conditions that may be met that results in the learner falling through to the legacy IDV flow. The _redirect_if_necessary method redirects the learner elsewhere, but if a learner does not meet these conditions, they will be shown the legacy IDV flow.

It is difficult to understand when and how often this actually occurs and whether it’s of concern to Open edX.

This is the legacy flow. Note that the id-verification application of frontend-app-learner-account MFE replaces the legacy IDV flow.

The PayAndVerifyView uses the IDVerificationService in its _redirect_if_necessary method, which redirects learners to the IDV flow if they have already enrolled and paid for a course. The IDVerificationService is used to get the expiration date of a learner’s IDV attempt in the submitted, approved, or must_retry statuses, if one exists, and to get whether the learner has a pending or approved IDV attempt.

The following methods of the IDVerificationService are called.

The verify_student/start-flow URL slug is used by the edx-enterprise application in the LMS_START_PREMIUM_COURSE_FLOW_URL constant. This URL is used to

redirect the learner to the checkout flow in the HandleConsentEnrollment
redirect the learner to the checkout flow in the CourseEnrollmentView when data sharing consent is not required

The dependency on the legacy IDV flow could be deprecated, but it would depend on Open edX needs.

VerificationStatusAPIView

The VerificationStatusAPIView is used to get a learner’s IDV status. The IDVerificationService is used to get the learner’s IDV status and the expiration of their approved IDV attempt, if one exists.

The following methods of the IDVerificationService are called.

This view is used by the id-verification application of the frontend-app-account MFE in the getExistingIdVerification function. This is used to determine the existingIdVerification attribute of the IdVerificationContext.

However, the existingIdVerification variable does not appear to be used in the frontend-app-account MFE.

This dependency could be deprecated because the existingIdVerification variable is not used anywhere.

ReverifyView

The ReverifyView is used to reverify learners whose IDV attempts have been rejected or expired. The IDVerificationService is used to get the IDV URL.

The following methods of the IDVerificationService are called.

get_verify_location

There are many references to the concept of a “reverify view”, but this dependency could be deprecated because there appear to be no actual references to its URL.

Certificates

The certificates application in the edx-platform handles the generation of course certificates.

Certificate Generation Signal Handler

The listen_for_id_verification_status_changed signal handler listens to the LEARNER_NOW_VERIFIED signal and generates a task to generate a certificate. This signal is emitted when a photo or SSO IDV attempt is approved.

The signal handler uses the IDVerificationService is to get the status of the learner’s most recent IDV attempt. Note that this includes records from the SoftwareSecurePhotoVerification model, which is what we traditionally refer to as IDV, as well as records from the SSOVerification and ManualVerification models.

The following methods of the IDVerificationService are called.

user_status

The expected_verification_status variable is used in a logging statement in the signal handler.

This dependency could be deprecated. The logging statement could be removed or moved elsewhere. I do not see a strong reason to maintain this dependency for a log. Alternatively, the deprecation and removal of the SoftwareSecurePhotoVerification model would also be a reasonable approach.

Certificates Generation Handler

The _id_verification_enforced_and_missing function is used to determine whether a learner is missing IDV and whether IDV is a requirement to earn a certificate. This function is used in a few locations in that module. The function calls the user_is_verified method of the IDVerificationService.

The following methods of the IDVerificationService are called.

[Proposal] Add Extensibility Mechanisms to IDV to Enable Integration of New IDV Vendor Persona

Overview

Problem

Use Cases

Proposed Solution

Backend Extensibility Mechanisms

Generic Model

Hooks

Filter Hooks

Event Hooks

Frontend Extensibility Mechanisms

Refactoring

IDV Attempt References

Support Tools Verified Name Panel

Name Change Signal Handler

Name Change Modal

Other Approaches Considered

Backend

IDV Deprecation

Additional Discuss Forum Posts

edx-platform Fork

edx-platform Modification

Frontend

Alternative PluginSlot Locations

Adding Routes - PluginSlot as Sibling

Adding Routes - PluginSlot as Parent

Server-Side Returned URL

frontend-app-account MFE Fork

frontend-app-account MFE Modification

Competitive Research

Implementation Plan

Long-Term Ownership/Maintainership

Discovery

Appendix

Uses of IDV on the Open edX Platform

Goal

Summary

Unverified Certificates

References to IDV in Backend Code

IDVerificationService

Proctoring Requirements Email

PayAndVerifyView

Legacy IDV Flow

VerificationStatusAPIView

ReverifyView

Certificates

Certificate Generation Signal Handler

Certificates Generation Handler