Problem Editor - Current State, Gaps, and Opportunities
As of Palm, the current problem editor is too complex for many course authors, unintuitive to use, and lacking in functionality. It is technically fine, but the barriers to using the editor and typing markup formatting syntax result in misuse, issues, and frustration. Many features are unknown to the majority of course authors, such as option randomization and the ability to include (some) HTML elements and formatting, and there are no WYSIWYG formatting options to simplify the process of using them.
Despite supporting the necessary inputs for cloze tests, such as dropdowns, it is only possible to construct a true cloze question using OLX, or by injecting custom CSS into the problem editor.
The current problem editor has numerous issues with its authoring and configuration experience:
Configuration
Configuring problems is done by typing punctuation elements to define input types and other problem elements, for example (x) indicates a correct multiple choice input option. When inserting a problem, authors are given a list of options for what template to insert into the problem editor:
These essentially all insert the same thing (the simple problem editor) with different templated content that handles some of the features of the editor. This leads to a faulty user assumption that A) These are different features and B) There can be no combination of elements, conversion between problem types by changing the syntax, or even things like hints and feedback without first selecting the template that includes hints and feedback. Neither of these are true.
Once in the editor, there is minimal guidance given besides a cheat-sheet which does not include all the markup that is possible in the editor, and cannot be closed by advanced users who do not need the cheat sheet.
The cheatsheet refers to markdown, which is used for some basic formatting syntax, but the problem input syntax is not part of the markdown schema, and due to the symbols involved, the vast majority of markdown is not available in the editor (for example, as [ ] denotes a checkbox and ( ) denotes a multiple choice input, you cannot include a link via the standard markdown syntax of [link text](link address). Worse still, should certain symbols be used in certain elements, the problem can cease to function entirely and cause errors, or cause the markup not to function to prevent the error. This occurs regularly in some academic subjects such as mathematics (which routinely use mathematical symbols), and law, which uses square brackets to denote references.
Elements cannot be used inline without custom CSS injection, which makes cloze problems impossible to insert in the editor for most users, and some syntax is extremely complex and unintuitive to use. For an example of this, here is a checkbox problem that displays one set of feedback only if certain items are selected together:
[x] a correct answer[ ] an incorrect answer[ ] an incorrect answer [x] a correct answer
{{((A B D))Specific feedback}}
There is no labelling that indicates options by letter - A, B, and D are concepts that only exist within the feedback setup. Options are space-separated, which is not a common way to format a range of options, and due to the limitations of using punctuation to mark this feedback, multiple symbols are used to mark the feedback that are easy to mistake (double curled braces into double parentheses). While not a huge issue with this level of setup, with more complex problems this is rapidly an issue, and if problem syntax is wrong it will fail to behave correctly with no indication that what has been written is incorrect.
Options along the top of the editor allow the user to quickly insert templated content with the correct syntax, but only in large blocks (for example, clicking “multiple choice” inserts three multiple choice input options with label text), and there is no distinction between multiple inputs of the same type. More recent features like hints and adaptive feedback have never made it into the UI, and as such there is no button or information on the cheat sheet about them.
Important features also appear to have been stripped from the problem editor between releases without being documented in release notes. The key example for this is in option order randomization, which was once possible by entering an exclamation mark (!) into a multiple choice or checkbox problem input (for example (x!)to enable randomization and (x@) to lock an answer in place). This feature was never fully documented to my knowledge, and is not present in recent overhauls of the problem editor, leading me to believe (without evidence) that this deprecation may not have actually been intentional, and was likely removed to fix errors caused by entering symbols in the problem editor without being aware that the feature existed. It is entirely possible there are other features of the editor that are lost or broken, because they were never documented or included in the UI.
On the settings side, multiple settings exist that only pertain to the creation of “advanced” or OLX problems, such as having a setting for a Mathlab API key (which poses a potential risk to the security of that API key as it is presumably stored in course export files rather than being defined at a course level in the platform), and support for randomization, which only applies to the randomization of Python variables, and not problem editor elements.
Community Opinions & Solutions
2U are also creating an entirely new problem editor for themselves as the old one is so poor. It currently lacks flexibility and some functionality, but the user experience is undeniably better than the old editor. As this is the primary reasonable alternative, this will be discussed in more depth in criteria 4.
2U have deprecated the Mathlab field in their version of the problem editor, and it stopped being useful a long time ago as Mathlab stopped issuing API keys.
OpenCraft’s Problem Builder XBlock effectively provides an alternative way to implement problems with more functionality and flexibility than the standard problem editor, but has some major UX issues and lacks some of the core problem editor functionality (as it wasn’t designed to fully replace it). This XBlock likely would not exist if the standard problem editor had comparable features.