Moodle - Assessment Setup

The core grading object in Moodle is referred to simply as a Grade Item, and this is what relates to each assessment in the course. Grade items are typically:

• Activities

  • Any activity can have a grade item associated with it. Most graded activities create one of these by default, and automatically report grades to the grade item using the Moodle gradebook API, but any activity can have a manually created grade item associated with it, allowing any activity to be manually graded (not to be confused with Manual Grades, which are something else).

• Outcomes

  • As discussed previously, Outcomes are an optional way of tracking learner competency that can be enabled for a course, and serve as part of the calculations for an overall grade in a course. As Outcomes are all measured on some kind of custom-defined scale, they create a grade item in the gradebook in order to store a learner’s progress on that outcome.

• Manual Grades

  • Manual grades are custom-created grade items which can be filled in with a grade by staff. They are not automatically populated, but allow staff to create custom grading activities, such as participation in an in-person event,  or any other arbitrary task that cannot be automatically graded.

All graded content within Moodle exists within Activities, which are typically classified as any page that includes interactive content, as opposed to resources, which are assimilative content. Not all activities are graded by default, and the options available to each activity are largely specific to that activity type, but any activity can technically be graded by assigning a manual grading item to that activity.

Common activity settings can be managed and enforced by site administrators, but the only core common activity setting that seems to exist in the documentation is the activity description, which is displayed to learners and can be optional or mandatory depending on the site settings. As third-party activities (the Moodle equivalent of XBlocks) can be non-localised, the option also exists to only make certain activities available to courses in specific supported languages.

Core automatically graded activity types in Moodle’s core documentation include:

• Assignments

  • Enables teachers to grade and give comments on uploaded files and assignments created on- and offline. For a summary of this activity type, see the ORA feature analysis.

• Forum

  • Allows participants to have asynchronous discussions, and receive grades for their contributions

• H5P Content

  • Enables H5P content created in the Content bank or on http://h5p.com or with the Lumi App to be easily added to a course as an activity.

• Lesson

  • Allows graded and ungraded content to coexist in the same activity. Appearst to act similarly to an Open edX subsection, or the step builder XBlock.

• LTI tool

  • Allows participants to interact with LTI compliant learning resources and activities on other web sites. LTI tools are configured and enabled at the site-level in Moodle and other learning platforms, rather than the course-level (Open edX is near-unique in that regard due to its roots as a multi-organisation MOOC platform, which makes its LTI integrations typically non-standard).

• Quiz

  • The most common graded activity type, this allows basic automatically graded tests, similar to Open edX’s CAPA problems.

• SCORM

  • Enables SCORM packages to be included as course content

• Workshop

  • Enables peer assessment, with grades awarded for grading other learners in-line with other learners in the course. For a full summary of this tool, see the ORA feature analysis.

Administrators have an interface to manage all activities available in courses, as well as the ability to remove them from courses, or make certain activities only available for certain courses. So if, for example, a specialist chemistry tool were to be installed on a site, it would be possible to only provide that tool to courses on request to a site administrator, rather than making that same tool available to all courses. Additional activities can be found on the Moodle Plugins Directory.

Rather than go in depth into each available activity type here, I’m instead going to summarise properties available to grade items, which are used by all graded activities.

Grade Item Properties

• Item name

  • The name given to the item, for example the assignment name.

• Item info (Optional)

  • General information about the grade item, such as a description.

• Grade to pass

  • Specify the minimum grade a learner must achieve to be considered to have passed the item as a literal grade, rather than a percentage. Grades will be highlighted in red for fail and green for pass in the gradebook. If the item’s grade type (see below) is a scale then the grade to pass is the lowest grade that counts as a pass. For example, with the grade scale: “Fail, Accept, Merit, Distinction” the lowest grade that counts as a "pass" is "Accept" therefore the Grade to Pass is set to 2, as it is the second item in the list that goes from negative to positive.

  • Note that this is a significant difference between Open edX grading and Moodle - Moodle has an activity-level concept of “passing” an individual assessment, which Open edX does not.

• Grade type

  • There are 4 grade types:

    • None - No grading possible

    • Value - A numerical value with a maximum and minimum

    • Scale - An item in a list with an assigned point value. Scales are specially defined objects in Moodle that can be reused and inserted in order to do things like automatically grade forum posts according to ratings. See the Scales documentation for more information, they’re interesting and flexible.

    • Text - Feedback only

  • Only value and scale grade types may be aggregated. The grade type for an activity-based grade item is set on the activity settings page.

• Multiplicator

  • Factor by which all grades for the grade item will be multiplied, with a maximum value of the maximum grade. For example, if the multiplicator is 2 and the maximum grade is 100, then all grades less than 50 are doubled, and all grades 50 and above are changed to 100.

• Offset

  • Number that will be added to every grade for this grade item, after the Multiplicator is applied. (Can be a negative number.)

• Hidden

  • Whether this grade item should be hidden from students.

• Hidden until

  • An optional date after which the grade item will no longer be hidden (i.e. a grades release date).

• Locked

  • Whether or not to lock this grade item. A locked grade item doesn't accept changes to its settings or grades from its related module.

• Locked after

  • A date after which the locking will be effective, usually after the grades release date.

Outcomes function almost identically, except they gain two additional properties in addition to those listed above:

• Outcome

  • The course outcome represented by this item

• Linked activity

  • The optional activity module to which this item will be linked

Outcomes are always graded on a scale (the same types of scale that are used as grade types), and are typically used for grade aggregation in multiple areas. An activity can be both graded and linked to an outcome, to essentially provide a grade to both the learning outcome and the assessment itself.

Manual grade items on the other hand have a much more limited set of options:

• Item name

  • The display name of your grade item

• ID Number

  • An arbitrary string of characters used to refer to this grade item in Formulas. If set, it must be unique.

• Grade type

  • The type of grade:

    • None (no grading possible)

    • Value (a numerical value)

    • Scale (an item in a list)

    • Text (arbitrary text).

• Scale

  • Which scale to use for grading: available only when the Scale grade type is selected.

• Maximum grade

  • The maximum grade that can be given (for scales: the number of items in the scale - 1)

• Minimum grade

  • The minimum grade that can be given (for scales: 0)

The following settings are also available for all grade items, and can be given defaults at the site-level:

• Grade display type

  • Real - Grade from minimum to maximum range indicating total points received. Default from 0-100 but may be arbitrary.

  • Percentage - Grade from 0 - 100% indicating the total points received divided by maximum possible amount times 100.

  • Letter- Grade in the form of a letter representing a range of percentages.

• Overall decimal points

  • This specifies the number of decimal points to display for each grade. It has no effect on grade calculations, which are made with an accuracy of 5 decimal places.

Site administrators can also set the default and decide which options of all of the above for each grade type can be seen and modified by course staff (for example, Grade to pass can be set to a fixed number for all grade items with no option to modify it by staff).

Quizzes and Assignments

Activities themselves can contain arbitrary settings that relate to that specific activity. For the sake of brevity, I’ll focus on the most important and comparable assessment types - quizzes and assignments.

Quizzes

Quizzes have the following properties in addition to those generic grade item properties listed above:

• Open the quiz

  • A date/time at which the quiz opens. While closed, learners can see that the quiz exists, but not see any of the questions, only the title, any introductory content, and when the quiz opens.

• Close the quiz

  • When the quiz will revert to being closed for submissions. By default, quizzes have no open or close times, and as such will appear for completion and access at any time the learner has access to the course.

• Time limit

  • How long learners have to answer questions after starting the quiz. A timer is displayed, and when time expires, the action specified for “When time expires..” is performed.

• When time expires..

  • When a learner runs out of time on a timed exam, this will automatically perform one of three options:

    • Open attempts are submitted automatically (This is the default)

    • There is a grace period when open attempts can be submitted, but no more questions answered

      • This enables an additional option allowing learners to answer more questions in the grace period

    • Attempts must be submitted before time expires, or they are not counted.

• Grade category

  • Moodle can group grades by category, similar to Open edX assignment types, allowing for dropping grades and weighting different grade categories for the overall grade

• Attempts allowed

  • How many attempts at the exam a learner is allowed as a whole

• Grading method

  • Highest grade - the final grade is the highest (best) grade in any attempt

  • Average grade - the final grade is the average (simple mean) grade of all attempts

  • First grade - the final grade is the grade earned on the first attempt (other attempts are ignored)

  • Last grade - the final grade is the grade earned on the most recent attempt only

• Navigation Method

  • Learners can be forced to progress through one question at a time sequentially, or allowed to freely answer questions in any order

In addition to these, quizzes have many settings available around reviewing their answers, with the option of making review possible during the quiz (displaying things like feedback and explanations before submitting the quiz, as opposed to their standard model of displaying feedback and explanations after the quiz is submitted as a whole).

Open

There are a huge variety of options available to quizzes as a whole, including a range of features designed to try and prevent cheating and generally restrict access to the quiz, such as:

• Requiring a password (so that only learners with the password can attempt the quiz)

• Requiring a specific network address (so that only on-campus learners in specific rooms can attempt the quiz)

• Enforced delay between attempts (to require learners to revisit a quiz after a week or some other arbitrary time)

• Full-screen pop-up with JavaScript blocking

  • This feature effectively puts some basic in-browser measures in place to try and prevent cheating. It removes all course navigation and attempts to hide all other browser tabs (though this is technically impossible), and has a few other minor features to try and at least make cheating inconvenient.

  • The Safe Exam Browser is a custom browser provided by Moodle which has much more control over learners to prevent cheating. 

Groups and individual users can be given individual overrides for things like open/close dates, and site administrators can set an auto-save period for quiz questions so that unsubmitted answers are not lost (unless they are lost between saves). 

For full details on quiz settings, see the core Moodle documentation.

Assignments

Assignments are how Moodle learners submit work to staff for grading. A full breakdown of Assignments can be found in the ORA Feature Analysis.

The relevant part of this feature for this section, however, is that Assignment Settings feature the following properties:

• Availability settings

  • Control when the assignment is available, time limits, and when the instructors should be reminded to complete their grading. This date appears in their calendar (which is a feature provided by Moodle for instructors) to remind them that they have to complete staff grading on assignments by the chosen date.

Open

• Submission settings

  • These control details about what learners will be submitting (such as a file upload or text written in-browser), as well as whether learners need to submit their assignment or it auto-submits, requirements around submission statements (“I promise that work I submit is my own, etc. etc.”), whether learners get multiple attempts, etc.

• Group assignment settings

  • Configuration options for group assignments, such as whether a single learner in the group submits or the whole group.

• Grading settings

  • Settings for the grading process, like whether learners are anonymised during grading, whether grades are hidden from learners at different stages of the grading workflow, and whether specific learners are assigned to specific staff for grading, as well as what form staff feedback takes.

In addition to these settings being available at the course-level, at the site-level, site administrators can set the default values of all settings, as well as designate some settings as “advanced” to effectively make certain settings less convenient to access (simplifying the setup experience), and locking some settings entirely so that they cannot be modified by non-administrators. Lastly, they can manage any plugins that are applied to assignments, as there are additional features that can be enabled on assignments by using third-party plugins, such as allowing learners to add a comment to their submission for the grader.