Randomized Content - Current State, Gaps, and Opportunities

The ability to provide random problems from a bank is a table-stakes feature for any LMS, and content reuse increasingly has become a major area of interest for academic institutions, as well as other less-traditional educators. This all means that there should be no doubt that content libraries should be a core feature, and by extension the methods of extracting content from those libraries become core features too.

While content randomization doesn’t see much use, this is an area in which Open edX is equipped beyond the capabilities of most competitor LMSes, and that makes this somewhat self-reinforcing (it’s not supported as a feature because it’s not used, which means it doesn’t get supported as a feature so it never gets used…). Randomised problems are by far the most common use-case, and the most important part of this feature, but in its current state, the feature is nowhere near flexible enough to serve its purpose.

Content libraries are already undergoing significant reworking and overhauls, so a new tool (or significant redevelopment of the existing tool) will be required to ensure the new capabilities of content libraries are used to their utmost potential. It will not be feasible to simply plug the existing component into the new libraries and expect them to be sufficient, and for that reason this feature has been marked as worthy of investment. The exact scope will depend entirely on the features and functionality of the new content libraries.

General Summary

Overall, the Randomized Content Block is misnamed - it is not suitable or designed for content, only problems. It has been neglected while content libraries have developed, and as the only native way to pull content from content libraries, it is inadequate.

Randomized Content Blocks have significant usability issues. Functionally, they are not built to serve the needs of users of content libraries, but are the only way to access content from libraries. This means that the experience of using these for HTML, Video, or XBlocks is underwhelming at best. Content libraries have essentially been developed and improved significantly, while the method for accessing library content (this component) has been neglected. This is likely due to work on content libraries still being ongoing.

Randomized content blocks can also cause significant issues for course staff performing things like checks on specific learner progress, regrading, and other learner maintenance activities due to requiring a more complex process for accessing controls over specific problems served to the learner.

One of the biggest issues with Randomized Content Blocks for assessment is their lack of intelligence. All a Randomized Content Block does is deliver a random selection of content from the library. If you insert multiple blocks from the same library, you cannot guarantee that a learner will not receive duplicate problems, meaning you cannot do something like insert a Randomized Content Block to pull in a problem, deliver a piece of content or pose a specific problem, and then insert a Randomized Content Block to pull in a second problem without risk of problem duplication. No knowledge is kept that the learner has already seen that problem on that course, or even that same page, unless it is within the same Randomized Content Block. This significantly harms the flexibility of the component.

Configuration

Randomized Content Blocks are configured in Studio, which is the correct location as a content block. They have the following issues:

• The Randomized Content Block is only designed for problems, when content libraries can provide more than problems.

• The majority problem types listed are not problem types that can be authored through the simple problem editor, which leads to author confusion.

• There is no flexibility as to which problem types can be selected. You can tell it to select between all multiple choice problems, but not, say, 3 multiple choice and 2 text input problems.

• Selection of the content library is from a dropdown list of all content libraries, so on a site with 200 content libraries, the dropdown has to load and display a massive scrolling list of all 200 libraries. This is less of an issue for staff than it is for admins, but is still an issue for organisations with significant numbers of content libraries at scale.

• As mentioned previously, it is not feasible to use multiple Randomized Content Blocks accessing the same library in the same course, as content will be repeated for learners.

• While not strictly configuration, when accessing staff tools in the LMS to manage learners, the inherent awkwardness of resetting and regrading learners for problems is amplified by the content being randomized - where a normal problem can be accessed from staff debug info, randomized content requires viewing that content as the learner in order to access the problems that were actually provided to that learner. This is already a suboptimal workflow, with its issues amplified by the randomization process.

Community Opinions

• Updating courses using Randomized Content Blocks for library content is extremely manual and time-consuming at scale, making them unsuitable for the most common use-cases for creating centralised content - link 1 link 2

Feature Improvement Recommendations

• Build a new component from scratch designed to pull content from content libraries, with randomization as a feature of that component
  OR

• Build content reuse into other components in a more coordinated way. Allow content library problems to be accessed from the problem editor, allow content library videos to be accessed from the video component, etc.
  OR

• Focus this tool on being a Randomized Problem Block, and rename it. Ignore all content besides problems, filter down available content libraries to those specifically marked as problem banks through tagging or categorisation, and improve this tool’s problem-oriented capabilities, while providing a different method for adding randomized or sequential content.

Regardless, the capabilities and needs of the component that is designed to draw from the new content library functionality should be re-scoped and re-designed to match those of the new libraries, and this method needs significant reworking and review at best, and removal at worst.