Ideal ORA Features and Functionality

Assuming we were to keep ORA’s multi-functional nature intact, then the feature should be able to support the same activities it does currently - staff-graded assignments, peer-graded assignments, and self-assessment. It would not make the most sense to add or subtract from this list without more significant research. Therefore, we’ll look at generic features applicable to all three of these types of assessment, followed by any specific features for each assessment type.

General Feature Requirements

• Support for a customisable range of submission types, including:

  • In-app text entry

  • File uploads

  • Media uploads

    • Video & Audio, with recording capabilities

  • URL responses

  • Annotation of preset PDF files

  • Code submissions

• Anti-plagiarism checking

  • Compare and reject learner responses that are near-identical to existing submissions

• Blank submission prevention

• AI-generated content analysis (similar to tools like ZeroGPT)

  • Should be optional to allow for assignments where the intention is for learners to use generative AI tools, but with a value (such as generation likelihood) displayed to staff regardless.

• Submission spelling- and grammar-checking

• Inline commenting/annotation on submissions (Google Docs style)

• Digital safety and security features, such as virus protection, malicious link scanning, etc.

• Browsable and searchable (by user, and by content) lists of learner responses for staff

• Support for a wider variety of grading strategies

  • Flat score

  • By Rubric

  • Others (further academic and educator research necessary to define relevant grading strategies)

• Resubmission and submission withdrawal by learners who are unsatisfied with their grade or accidentally submitted early

  • This should be optional and carefully implemented, or possibly just disabled for peer grading, as it enables plagiarism

• Basic syntax and configurable unit tests for code submissions

• Moderation and learner submission management tools for staff

  • Lock/unlock response

  • Delete response

  • Edit response

  • Ban/allow specific learners to submit

• Streamlined UI - as minimal as possible

• Optional notifications for submissions waiting (staff) and submission graded (learners)

  • In-app

  • By email

• Automatic saving of work-in-progress

• Comment-only (non-graded ORAs)

• Discussions tied to ORAs

• Flexible updates to running ORAs (to fix minor issues such as typos after an ORA has begun)

• Deadline-free ORAs (for self-paced courses)

Staff-Graded Assignments

The ideal staff-graded assessment tool could have the following features:

• A clear, full-screen grading interface for staff grading and submission discovery

  • This should not require navigating to the ORA itself

• Support for multiple staff graders, providing an average score between graders

• Grading release date - release all grades on a certain date rather than when graded

Peer-Graded Assignments

The ideal peer-graded assignment tool could have the following features:

• Pre-seeding (staff submission of dummy answers)

  • These should only be offered to learners if an insufficient number of peer submissions exist when a submission is retrieved.

• Grading outlier detection and handling

  • It should be possible for staff to detect grades that are significantly beyond the average grade, and intervene by adjusting or removing a peer grade.

• Grading weighting - granting points to learners based on their contributions to peer grading

  • See Moodle Workshops for an example of this

• A separate grading and annotation tool for learners (with less functionality than the staff grading tool)

• Grading moderation tools - override grade, edit comments, delete comments, etc.

• ORA comment threads

• Anonymous/identifiable peer grading

Self-Assessment

• Prompts with text responses

• Self-grading by rubric