Canvas - Problem Editor
- Matthew Harrington
Analytics
Canvas, similar to Moodle and other platforms, structures its question content into quizzes, and you cannot create questions as part of course page content, putting a hard line between interactivity and questions unless integrations with third-party tools are used. They have two versions of quizzes, somewhat confusingly named “quizzes” and “new quizzes”. This research will focus entirely on new quizzes, as they represent both the current state of quizzes at the time of writing, and Instructure’s focus for quiz tools.
Quizzes and their questions are tied explicitly to learning outcomes which are defined at the course and institution level, which is a nice feature for ensuring academic standards are maintained and the purpose of the quiz is clear to staff and learners, as well as making them more discoverable in question banks. The following question types are available for quizzes:
Categorization - Requires learners to drag and drop to sort items into predefined categories. Can have any number of items and categories.
Essay - Requires learners to enter their response as a long-form essay. Word count requirements can be enforced, with rich content editing, spell checking and other basic word processing features. Grading notes can be provided to instructors, and all essays must be graded by instructors.
File Upload - Simply allows learners to upload a specified number of files in response to a question prompt. Uploaded files must be graded by an instructor.
Fill in the Blank - A cloze-style question in which the instructor writes the question and marks the words that will be replaced by `backticks`. Once marked, the author may pick from multiple types of input to use to fill in the blanks. This problem type is extremely fully-featured, so reviewing the documentation for this question type is recommended if you’re interested in how this is implemented, else this will take multiple pages.
Formula - Requires learners to enter a value that answers a generated question based on a formula provided by the instructor. Any variables in the question are denoted by using `backticks`, and the variables can be assigned value constraints in the editor (for example if the formula is “10*x”, you can define x as any number between 1 and 512.7, with a limit of 1 decimal place).
Hot Spot - Hot spot questions allow instructors to upload an image in .BMP, .PNG, .JPG, or .GIF format, and require learners to select a specific region on the image. Instructors can do this by drawing a square, circle, or polygon around the area of the image they want to use as a hot spot.
Matching - Presents a static list of items in the left column, and a dropdown list in the right. All the options are listed in the dropdown, and the learner has to select the correct one that correlates to the item on the left.
Multiple Answer - Requires the learner to select a predefined number of options that answer the question using checkboxes. Supports answer shuffling and is otherwise functionally identical to Open edX checkboxes, with the exception of support for partial credit in the interface.
Multiple Choice - Requires the learner to select only one answer from a list of options using radio buttons. Answers can be shuffled, and partial credit can be awarded for some answers.
Numeric - Requires the learner to enter a numerical response, with a small range of options for variance, such as:
Exact (only the specific number will be accepted)
Margin of error (the specific number +/- an amount or percentage)
Within range (between two numbers)
Precise (the answer to a specific level of precision, such as two decimal places or significant digits)
Ordering - The learner must sort items in a list in a specific order, such as the order of planets in the solar system. The top and bottom of the list are given labels to identify them, such as “Closest” and “Furthest”, or “First” and “Last”.
True or False - The learner must select whether a question is true or false. Functions as a limited multiple choice question.
Interestingly, there appears to be no automatically-evaluated short text input question type. When created, the editor also has the option to immediately save the question to a question bank, and when creating a quiz, instructors can select questions from the bank of existing questions that were saved to the bank. Questions do not have to be saved to the bank to be used, it is simply an option for reuse. Every single question has the option of whether to include a calculator on any quiz pages on which the question appears, which seems like an extremely clunky implementation option, suggesting some level of inflexibility in quiz structure.
In general, Canvas seems to heavily emphasise a small set of interactive problem types that have very little overlap. Exceptions to this are True and False, which again, could just be multiple choice, and arguably Categorization and Matching, though there are uses for Matching which would be awkward to implement as Categorization and vice-versa.
Canvas does not have syntax-based question creation, instead relying on an entirely UI-led approach:
The question area allows for rich content in a WYSIWYG HTML editor, but there is no flexibility in answers. For example, in the editor above, it’s not possible to create a multiple choice question where the answer choices are images. Each question does exactly as it is designed to do, and no more or less than that.
Links: