Course Certificate Generation Logic Improvement

Created by Ryan O'Connell
Last updated: Jun 16, 2021
2 min read

TL;DR

We’re replacing the back-end code (no UX changes) that generates course certificates with a new version to: 

  1. condense the number of conflicting generation logic patterns,

  2. make it easier to troubleshoot/support, and 

  3. to better defend generated certificates against errant revocation.

Problem

As we’ve expanded credentials and programs over the years we’ve added complexity to certificate generation, revocation, and updating - leading to ongoing debt paid by our support team to keep the lights on; to the tune of ~12 issues per month that require engineering assistance to resolve. Issues that require engineering assistance is a lagging indicator of the overall number of issues that the support team troubleshoots and solves independently.

During our investigation and maintenance we learned that course certificate generation and revocation processes were using multiple routes and logic definitions to determine if-and-when a course certificate should be generated. Sometimes leading to unissued, errantly revoked, or delayed certificates.

Course certificate award dates and status’ are underpinning data points for learner records and program certificates. So an issue with a course certificate can cascade into issues with learner records, program certificates, and academic credit issuance as a result.

What is it?

We’ve created a new, unified course certificate generation logic.

The improved logic has been applied to 41 courses since mid-May without issue and now we’ve rolled it out to the full catalog.

This new generation logic is the penultimate release of an ongoing body of work since January that has incrementally improved certificate generation, revocation, allow-listing, program record synchronization, and issue resolution times.

Results

The effect of these efforts has already driven the certificate-related support ticket trend lower than spring 2019 levels (while supporting 16% YoY enrollment growth) and we expect this trend to continue along with decreased time-to-resolution for issues that do arise.

Key talking points for customers

Learners and partners should not notice any change. The only effect is that they should see a reduced time to resolution if they ever encounter a problem related to certificates or credentials.

Internal Info

Prior to this release, there were up to 6 different logic patterns used to calculate course certificate generation. This release has unified all of those patterns into a single logic set as follows.

For a learner to receive a course certificate in the downloadable state (i.e. accessible), the following things must be true at the time the certificate is generated:

  • The user must have an enrollment in the course run

    • The enrollment mode must be eligible for a certificate (i.e. paid)

    • The enrollment does not need to be active

  • The user must have an approved, unexpired, ID verification

  • The user must not have an invalidated/revoked certificate for the course run (see the CertificateInvalidation model)

  • HTML (web) certificates feature must be enabled for the edX instance (which it is for the edX site/org), and also enabled for the course run

  • The user must have passed the course run

  • The user must not be a beta tester in the course run

  • The course run must not be a CCX (custom edX course)

Build Together

Points of Contact for Questions

Engineering: @crice (Deactivated) @Justin Hynes | Product: @Ryan O'Connell

The Team

@Matt Tuchfarber (Deactivated) @TJ Tracy (Deactivated) @Olivia Ruiz-Knott (Deactivated) @Albert (AJ) St. Aubin (Deactivated) @Gabriel Weinberg @Kevin McGrath (Deactivated) @Jennifer A. Akana (Deactivated) @Adam St. Jean