Open edX

Within graded subsections, any type of component can be created, with those that return a grade to the LMS being the ones that contribute their grade to the associated assignment type. This includes (but is not limited to):

...

There are many other graded content types that can technically be included, but as this list is effectively infinitely extensible, I’ll draw the line at those listed as fully supported in the edX.org documentation, as they should theoretically be the most complete components that reflect the ideal standard of gradable components for the platform. XBlocks in general can have entirely arbitrary settings, but for a full breakdown of configuration for ORA, see the ORA Feature Analysis.

Problems

CAPA Problems all share the same interface for authoring, as they are fundamentally all simply different implementations of the same component type. A full analysis of the simple problem editor as of Olive can be found in the Problem Editor Feature Analysis document, as well as a look at the new problem editor being worked on by 2U, which is optionally available.

...

Above: The advanced problem editor

General settings

As all problems are created in the same component type, the same settings generally apply to all problems created in both the simple editor and the advanced editor, though the advanced editor can override some settings from within the problem (such as providing a different number of points than the correct answer’s weight):

• Display Name

  • The title of the problem, which is a common field to all components, with the exception that display name is or isn’t displayed to learners almost completely arbitrarily depending on how the component has been built. Problems display their display name, which HTML components do not.

• MatLab API key

  • A long-deprecated and no longer supported setting that is still present in Palm due to only being removed from the newer 2U editor

• Maximum Attempts

  • How many attempts a learner can have at submitting the problem before no longer having the submit option

• Problem Weight

  • How many points correctly answering the question grants the learner

• Randomization

  • This does not affect anything other than Python script variables.

• Show Answer

  • When the learner is allowed to view the correct answer from a list of different problem states that are only explained in the unlinked documentation

• Show Answer: Number of Attempts

  • How many attempts to allow before the show answer button appears, provided the former option is set to “After some number of attempts”. This option appears regardless of whether that option is selected.

• Show Reset Button

  • Whether a button should be shown to allow learners to clear their currently selected or typed responses (this does not reset attempts for obvious reasons).

• Timer Between Attempts

  • A number of seconds that a learner must wait before reattempting the problem. This is always seconds, so a learner who must wait 3 days must be set to wait 259200 seconds.

Basic Problem Editor

The following standard problem types are supported in the basic editor:

...

This has a fairly steep learning curve, especially as the included cheat sheet doesn’t actually have all the options available on it, and it is far from ideal. The shortcomings of this approach are discussed in far more depth in the Problem Editor Feature Analysis, so I won’t spend too much longer on that here.

Advanced Problems

Custom JavaScript Grading problems and Math Expression Input Problems (as well as Custom Python-evaluated Input Problems, which are also present but only ‘partially supported’) are effectively different implementations of advanced OLX problems, making use of <script> tags to inject arbitrary code into problem components. This is how tools like the chemical equation problem exist in their under-maintained state, with custom tags like <chemicalequationinput> being defined to provide input fields in order to provide an input to a custom grader. Somewhat entertainingly, custom JavaScript grading problems appear to make use of Python grading (the example uses Python to import the grading from the JS), so JavaScript Grading effectively requires Python grading problems to exist, despite the former being ‘fully supported’ and the latter being ‘partially supported’.

...

• Partial Credit
  

  • Partial credit awards a lower point value for correctness, but is not fully implemented in reporting, as all reports will report that the learner got the question correct and received 1 point, even if that is not the case.

• Tooltips
  

• Randomised answer pools for multiple choice problems

...

Content Libraries

The task of problem banking in Open edX is performed by Content Libraries, which also support non-problem content. A full analysis of this feature from the perspective of focusing purely on randomised content can be found in the Randomized Content Feature Analysis. The short version is that despite the fact that content libraries have been extended for non-problem content, the XBlock for pulling content out of these libraries is clunky and not fit-for-purpose, and needs a complete replacement that both performs the same function and enables the other uses of content libraries.

LTI

LTI content can return a grade, and is handled by the LTI consumer XBlock. While there are many graded XBlocks that exist, most LMSes included an external tool feature to get grades from an LTI assessment, so LTI deserves a quick mention.

...