Coursera - ORA
- Matthew Harrington
Staff-Graded Assessment
Coursera has learner-facing documentation on resubmitting staff-graded assessments which includes the qualifier “In private courses (such as courses in a Degree program), some assessments may be manually graded by your course staff…”.
This implies that staff-grading is not offered outside of these private and degree-level courses, which would make sense given the workload of staff-grading in a MOOC context.
Staff can configure assignment lockout dates, a period in which submissions cannot be submitted independent of the deadline, as well as limiting available attempts. Course staff can provide additional attempts to learners manually, and Coursera support will not do so, implying that controls related to assignments are left entirely to the course team.
Learners can choose to resubmit their assignments until they hit the attempt limit, even if the assignment has already been graded. So if a learner does poorly, they can try again. Assignments can also have prerequisites, preventing learners from accessing a staff-graded assignment until another assignment is completed.
Coursera reports that having at least one staff-graded assessment drives a 6% increase in student retention on their private courses.
Links:
Peer Assessment
Coursera has peer assessment functionality that is only available on private courses. Instructors provide the learners with instructions on how to complete the assignment, which can include images and attachments for learners to download. Learners can then submit a range of different submission types - including URLs, files, and text. Once uploaded, learners receive a link to the assignment, which can be shared to help promote peer reviews. Learners can also go back and resubmit at any time after submitting until the assignment is due, which deletes their previous score and queues the assignment back up for resubmission. Limitations on this can be configured by the course provider, for example limiting learners to a single submission, or a number of submissions.
Each peer assignment also features a dedicated discussion area, which is quite nice as it means that learners can discuss the assignment without thread proliferation in other areas of the forum. Text-based submissions appear to have built-in plagiarism detection, as the system will automatically prevent a learner from submitting a text submission that is “too” similar to a submission that already exists. There is also a minimum and maximum word count limitation that can be applied to text submissions.
The feature seems very similar to ORA, with a few extra features, but many of the extra features make it extremely easy for learners to cheat at Coursera’s peer assignments. Being able to resubmit after viewing other learners means that the learner can copy submissions from other learners who did better (or actually attempted the assignment). Having a sharable link causes discussion spam and enables cheaters to immediately jump to their own assignment, or those of their friends, and trade good reviews for money or as a quid-pro-quo (good review in exchange for good review). This significantly undermines the integrity of Coursera’s peer assignments, though they have taken a few steps to try and fight this, with strict honour code reminders, plagiarism checking, and other attempts to mitigate the problems introduced by these features.
Grading appears to be similar to ORA, where the learner’s overall grade is based on the sum of the median score of each part of the rubric. There appears to be a 7-day limit on receiving a grade from peer assessment, so long as one learner assesses the submission.
Staff appear to have the ability to manually reset assignments, as well as override their score if they suspect abuse. They may also choose to prevent specific learners from resubmitting. The usability of these tools is unknown, as is whether all of this functionality is available to course staff, or Coursera support alone.
Links:
How to pass any Peer-Graded Assignment of coursera Top 3 trics to pass Peergraded #courserasolutions - This video is actually about how to cheat at Coursera peer-graded assignments, but shows the learner experience of those activities (as well as their issues).
Self Assessment
Coursera does not appear to have a self-assessment feature.