Text Input Problem Comparator Research

• Canvas has no short text input question apart from the free text field option for its Fill in the blank questions (a Cloze Problem Analysis & Requirements question type) but this field has a number of options of note for how it evaluates inputs that are worth mimicking:

  • Contains: Allows for the text to exist anywhere in student responses. Close Enough: Uses Levenshtein Distance to determine if the response is close enough to the correct answer. Levenshtein Distance is the number of single-character edits needed to change one word to another.

  • Exact Match: Requires case and spelling to match a single answer exactly.

  • Specify Correct Answers: Allows a list of multiple allowed correct answers.

  • Regular Expression Match: Evaluates the input with regex.

• Canvas also has the Essay question type for longer text entry with the following settings of note:

  • Word Limit: a minimum and maximum word count for responses.

  • Rich-Content Editor: Whether learners get a WYSIWYG editor for responses.

  • Spell-Check: Whether to automatically spell-check learners as they type.

  • Word-Count: Whether to display an automatic word counter to learners.

• Skilljar has the minimum viable version of this question type (as typically is the case), titled Fill in the Blank. It allows a WYSIWYG question stem, with all allowed answers and a case sensitivity checkbox.

• Moodle has a question type labelled as Short-Answer questions. In addition to very common settings, it has the following settings of note:

  • Wildcards: You can use the asterisk character (*) as a wildcard to match any series of characters. For example, use ran*ing to match any word or phrase starting with ran and ending with ing, or fire*water to match any answer starting with fire and ending with water. This matches “fire and water”, “fire & water”, but also “fire but definitely not water”.

  • Answer scores: Answers can have different point values and are evaluated top to bottom. So for their example question of “What does a rocket burn?” they recommend the following:

    • oxygen*fuel with a score 100%

    • *fuel* with a score 50%

    • *oxygen* with a score 50%

    • *air* with a score 40%

    • * with a score of 0%

      • The final wildcard essentially gives them a slightly clunky “else”.

  • Per-answer feedback: Staff can enter as many potential answers as possible with different scores, and give each of those answers specific feedback.

Open

Penalty factor: A common practice for Moodle questions, a percentage of points available can be deducted for each incorrect attempt, reducing the maximum available grade.