This document provides motivation, background, and design for creating a more robust and reliable grading subsystem in the edX platform.
...
| Use Case | Feature Design that satisfies the use case |
|---|---|
| Hide Content to prevent cheating | BlockContentViewPermission |
| Change of Content doesn't affect past grades |
|
| Certificates retain accurate grades | PersistentCourseGrade |
| In Exceptional cases, grades can be overriden | GradeOverrides |
| Event trail enables debugging of grade issues | Eventing/Logging of Grading Changes |
...
- Modifying overall course grade computation policy to account for:
- showing a cumulative percentage based on completion so far while discarding uncompleted assignments
- e.g., If a learner gets a 100% on the 1st homework out of a potential 5 homeworks (yet to be released), the learner doesn't confusingly see only a 20% grade on homework.
- requiring a minimum grade for each assignment type
- e.g., A course author can specify a passing threshold on each exam, requiring each exam to be at least 60%, regardless of what the cumulative exam grade percentage is.
- showing a cumulative percentage based on completion so far while discarding uncompleted assignments
- Consolidating/generalizing ORA's handling of content and grade versions with what's available for other problem blocks.
- Allowing learners to view older locked-down versions of the course after content changes, per Archive: Course Version Locking.
Use Cases
Robust Grades
The importance of grades is increasing as we grow the number of courses offered for credit on the edX platform - edX, learners, and course staff want to have reliable grades that all parties can feel confident are accurate.
Course Team Use Cases:
I want to be able to determine with confidence whether a learner has demonstrated mastery of the material in a course.
I want to modify content without having any impact on grades or certificates (unless specifically desired).
I want to hide assignments to maintain integrity and prevent cheating.
Learner Use Cases:
While I work through my course, I want to understand my grades and my progress towards completion.
I want to know that any time and anywhere I see my grade on edX.org, it is consistent and accurate.
I want to always have a record of my achievements in a course, so that I can share with potential employers or schools.
edX Staff Use Cases:
I want to feel confident that the grades we share with partners are truly accurate.
In Support, I want to be able to address any questions a learner has regarding their grades confidently.
...
Note: #1 and #3 can potentially benefit with improved handling of course versions for better user experience. Learners who already completed problems that are now removed may want to see their previous answers and the content of the removed block, per Archive: Course Version Locking.
Grade Overrides
Implement UI for overriding any of the following to a specific value:
...
To turn on persistent grading in an Open edX instance, go to the "Grades" section :
- Go to /admin/grades/persistentgradesenabledflag/ of your site's Django admin
...
- Click on "Add a Persistent grades enabled flag"
...
- To turn on the feature for all courses
...
- make sure to check both "enabled" and "enabled for all courses."
- If you want to turn on the setting for only a specific course, you will need to create a "Course persistent grades flags" entry in /admin/grades/coursepersistentgradesflag/ for the course and ensure that your main "Persistent grades enabled flag" has "enabled" set to True, but "enabled for all courses" set to False.
There are two waffle switches that is a waffle switch that can be used to modify grading behavior, ideally together:
...
enhance grades performance, but should be enabled only after any Backfilling process is completed:
- grades.assume_zero_if_absent: if there is no grade in the database for a given user, assumes a score of zero for that user in the course or subsection in question. This is a performance optimization, but should be used only if persistent grades have been calculated for all users on the instance. Otherwise, the platform will fail to calculate on-the-fly grades for users whose grades are missing from the database.