Text Input Problem Comparator Research
- Matthew Harrington
Analytics
Canvas has no short text input question apart from the free text field option for its Fill in the blank questions (a Cloze Problem Analysis & Requirements question type) but this field has a number of options of note for how it evaluates inputs that are worth mimicking:
Contains: Allows for the text to exist anywhere in student responses. Close Enough: Uses Levenshtein Distance to determine if the response is close enough to the correct answer. Levenshtein Distance is the number of single-character edits needed to change one word to another.
Exact Match: Requires case and spelling to match a single answer exactly.
Specify Correct Answers: Allows a list of multiple allowed correct answers.
Regular Expression Match: Evaluates the input with regex.
Canvas also has the Essay question type for longer text entry with the following settings of note:
Word Limit: a minimum and maximum word count for responses.
Rich-Content Editor: Whether learners get a WYSIWYG editor for responses.
Spell-Check: Whether to automatically spell-check learners as they type.
Word-Count: Whether to display an automatic word counter to learners.
Skilljar has the minimum viable version of this question type (as typically is the case), titled Fill in the Blank. It allows a WYSIWYG question stem, with all allowed answers and a case sensitivity checkbox.
Moodle has a question type labelled as Short-Answer questions. In addition to very common settings, it has the following settings of note:
Wildcards: You can use the asterisk character (*) as a wildcard to match any series of characters. For example, use ran*ing to match any word or phrase starting with ran and ending with ing, or fire*water to match any answer starting with fire and ending with water. This matches “fire and water”, “fire & water”, but also “fire but definitely not water”.
Answer scores: Answers can have different point values and are evaluated top to bottom. So for their example question of “What does a rocket burn?” they recommend the following:
oxygen*fuel with a score 100%
*fuel* with a score 50%
*oxygen* with a score 50%
*air* with a score 40%
* with a score of 0%
The final wildcard essentially gives them a slightly clunky “else”.
Per-answer feedback: Staff can enter as many potential answers as possible with different scores, and give each of those answers specific feedback.
Penalty factor: A common practice for Moodle questions, a percentage of points available can be deducted for each incorrect attempt, reducing the maximum available grade.