Randomized Content

Open edX

The Randomized Content Block is the component used to access a random selection of content from a content library.

Content Libraries are course-like objects that contain different components:

HTML
Problems
Video
XBlocks (whatever is installed)

The Randomized Content Block is only able to access randomized content from the library, not specific content, but it can select a subset of problem types. It has the following controls:

Count - How many items to select from the content library
Display name
Library - Which specific library to pull from
Problem type - A preset list of problem types to pull from the library
Show Reset Button

Notably, the Randomized Content Block is only able to filter down to specific problem types, not XBlocks, or other content like HTML or Videos from Libraries.

The ability to provide random questions from a bank is a table-stakes feature of any LMS, and lacking it would put Open edX at a significant competitive disadvantage. So from that perspective, yes, this capability is required.

However, very few platforms seem to provide functionality for randomised course content that is not backed by some form of machine learning algorithm. This makes Open edX more valuable for education research and education-led research, but means that author education and research would be necessary around why they should use randomised non-problem content for this to be a useful feature.

In my own experience, I have only seen randomised content used for a small number of generative activities, for example providing learners with a randomised input and then setting an assignment for them to create an output. Even then, it caused confusion when learners weren’t made aware that they were being given randomised content, as it became immediately obvious in learner discussions that they were seeing different content. It increased the burden of creating a wider range of content without any really measurable impact on learner satisfaction or outcomes unless it was specifically designed with a high level of academic knowledge and rigour for the purposes specifically of using the activity for research.

Pedagogic Summary

Reuse of question banks is standard at academic institutions to the point where there are large websites dedicated to collecting and selling question bank questions. Without question randomization, however, individual courses require greater effort for creation and can be cheated more easily at an individual level, as having randomised questions means that cheating within a single course is as simple as sharing a single answer sheet. This is interesting (to me at least), as question randomization is both an enabler for and a deterrent to widespread cheating.

In a brief search, I was unable to find any pedagogic research linked to providing randomised content (rather than questions), though it is likely that this may have been used as a tool in studies rather than being the primary subject of a study, as it inherently supports using different types of content for comparison. This means that, for example, it’s possible that a study on racial bias in the presentation of educational content could use content randomization to provide the same video content with different presenters to assess learner comprehension of the subject and their impressions of the speaker, and this wouldn’t show up in my very brief, cursory search as it was not the subject of the paper.

Most discussion of randomised content is tied to semi-randomly providing content that is determined through algorithmically derived weights, for example providing random content based on a learner’s interests or knowledge to construct algorithmically determined “random” learning journeys, rather than fully random content from a subset the way that Open edX does it. The more sophisticated this approach becomes, the less random they actually are, deriving their content from increasingly precise measures of the most appropriate content to provide via machine learning.

Links: