Page Comparison

This document provides motivation, background, and design for creating a more robust and reliable grading subsystem in the edX platform.

...

The importance of grades is increasing as we grow the number of courses offered for credit on the edX platform - edX, learners, and course staff want to have reliable grades that all parties can feel confident are accurate.

• Course Team Use Cases: 

  • I want to be able to determine with confidence whether a learner has demonstrated mastery of the material in a course.

  • I want to modify content without having any impact on grades or certificates (unless specifically desired).

  • I want to hide assignments to maintain integrity and prevent cheating.

• Learner Use Cases:

  • While I work through my course, I want to understand my grades and my progress towards completion.

  • I want to know that any time and anywhere I see my grade on edX.org, it is consistent and accurate.

  • I want to always have a record of my achievements in a course, so that I can share with potential employers or schools.

• edX Staff Use Cases:

  • I want to feel confident that the grades we share with partners are truly accurate.

  • In Support, I want to be able to address any questions a learner has regarding their grades confidently.
    

...

• Why persist?
  • Grade Override - In order to allow course teams to override (and have a final say on) a learner's course grade, we need a table to store the learner's grade.
  • Performance - Although may not be as necessary given that the grading infrastructure is now faster and the grades at the subsection will be persisted, by also persisting the course-level grade, requesting a learner's course grade would be satisfied with a quick database lookup.
• Decouple persistent course-level grades and component-level in-progress grades.
  • The PersistentSubsectionGrade section provides details on saving and updating grades for sub-hierarchical components within a course. 
  • The data model for those grades are specific to component-level grading information and generally consistent throughout all components.
  • The data model for Course-level grades, on the other hand, is dictated by a configurable course-level grading policy.  At this time, the platform is configured to use only a single-type grading policy (WeightedSubsectionsGrader), that provides a course-level grade percentage and letter-value.
• Decouple persistent course grades and certificates.
  • Certificates are dependent on the final course grade - but only to determine whether the grade is a passing grade.  Beyond that, certificates do not depend on the exact grade value and grade value changes.
  • In the future, if we ever support multiple types of certificates that vary by grade, we will need to annotate the certificate record with the certificate type.  However, it still does not need to depend on the exact value of the grade.
  • Currently, the certificates_generatedcertificate table has a column for the learner's course grade.  Once this work is implemented, the grade column in that certificates table will no longer be used.
• Save the course grade in a different SQL table from the certificates table and from the subsection-grades table.
  • Include columns for the course_id, course_version, subtree_edited_on, user_id, and the grade percentage.
  • Should the course letter grade also be persisted?
    • Why yes:
      
      • Performance: All information related to the user's course grade will be persisted with no need for dynamic computation.
      • Robustness: The persisted letter grade will be the value computed at the same time the percentage is computed.  So it would be unaffected by any subsequent course grading policy.
    • Why no:
      • Coupling: By storing secondary fields obtained from the grading policy, the data model is coupled to the artifacts of the configured grading policy.  If in the future, we decide to support other grading policies with additional course-grade values (for example, a grade on social activity or a curve-based grade) then the schema for the course-grades table will need to be updated.  On the other hand, in order to support overriding of all course-level grade data, it seems we would need to anyway.
    • Given the above, we should go ahead and persist the letter grade as well.
    • Overriding course-grade: Since we have multiple course-grade fields (percentage and letter) that can be overridden, the grade override feature can intelligently allow overriding just an individual field, both fields, or just the percentage and have the letter grade automatically recomputed.
• Update the persisted course grade automatically (and asynchronously) whenever any of the subsection grades in the course is updated.
• Notify all subscribers:
  • When a course grade is updated/saved. 
  • When an updated course grade exceeds the passing threshold.  Interested listeners would include:
    
    • Credit eligibility to check whether the user is now credit eligible per course policy.
    • Gating code to check whether new course pre-requisites are now satisfied.
    • Certificate generation code if course policy allows automatic generation of certificates once the course is passed.
  • Note: it's an implementation detail whether the above signals are implemented as separate signal-types or as a single signal-type with a distinguishing field.

Data Model Notes

Note: See Grades Data Model (for published description).

...