Multiple Choice Problems
- Matthew Harrington
Analytics
Multiple choice questions (or MCQs) typically come in two forms - single-select (radio buttons) and multi-select (checkboxes). They are near functionally-identical, but differ primarily in the form of the input expected from the learner. Multiple choice questions are a staple of basic assessments that are included in every LMS on the market. More complex features add onto the functionality, but at its core, many platforms leave them at their most basic level - a text question with a single correct text answer that is selected using radio buttons.
Usage
Multiple choice questions are the most used problems on edX.org, forming more than half of the problems used on the platform, with nearly three times the number of instances of the next most popular problem.
If considering multiple choice to encompass both multi-select and single-select problems (as some platforms do), then multiple choice encompasses a whopping 77.7% of all problems.
Further still, if considering dropdowns to be alternatively displayed implementations of multiple choice, which can quite easily be argued, that number increases to 82.4%.
It’s not unreasonable to say that multiple choice in any of these three interpretations form the most significantly used and important form of basic problem on edX.org.
Multiple choice questions are widely used across all platform contexts, and over-analysis of this is not necessary or a good use of time. Anyone who has ever created a course on an LMS has at one time or another used this type of problem.
Pedagogic Analysis
MCQs are so firmly established as an assessment method that their presence and reliability is generally accepted by most. There are and have always been many concerns that they are too efficient and possible to succeed via pure guesswork, but the reliability at which it allows questions to be easily asked and answered means they see ubiquitous use across all learning use-cases.
This has advanced to the point where many educators are now attempting to use the construction of multiple-choice questions by learners as a learning activity (i.e. peer-constructed questions), which may be a potential route for expanding learning functionality in the future due to generally positive results from these tests as a peer-driven activity.
Also of note is that while MCQs are accepted, they are generally regarded as a way to augment and diversify assessment, with pure multiple-choice only quizzes being inadequate for higher-stakes learner testing. Most interestingly, learners typically view multiple choice questions as a problem solving exercise, rather than a question of comprehension - rather than answering the question, they typically examine the options and evaluate which is most likely to be the correct answer, or deduce the answer by ruling out obviously incorrect distractors. This means that their value for formative assessment is high, but significantly lower for summative assessments.
This elimination process means that distractor construction is vitally important for the effectiveness of MCQs, a skill that is difficult to learn for many new course authors, particularly those from a non-academic background. One way of solving this is to guide authors towards creating MCQs with only 3 options - this way less distractors need to be written, which typically means those distractors are higher quality, and the number of learners correctly answering does not typically significantly change (other than making it harder to solve as a deduction-based puzzle), despite the increased likelihood of learners being able to randomly guess the answer.