Annotation Problem - Current State, Gaps, and Opportunities

The current annotation problem is good at what it does, if what it does is truly necessary, which presumably it is if it is used by institutions. Research should be conducted with the institutions that use this plugin to identify gaps that those institutions need, as this is a relatively unique problem type.

The biggest issue this question type has is that the current implementation of these problems requires authoring the problem in XML. There is no UI-based way to configure these problems, making them unsuitable for use by many course authors, and thus the feature is underused.

Not only this, but an annotation problem actually requires two components to be configured - one with the instructions, another with the actual input. This does not match any other components, and is an extremely subpar authoring experience.

I am fairly certain that this would see more use in academic contexts if there was a suitable authoring UI, so its lack of usage is likely not indicative of lack of value.

Current Gaps & Potential Improvements

• The current implementation is extremely heavyweight, and isn’t suitable for use alongside other components. This may be fine, because a lighter implementation of this can be easily accomplished using a regular CAPA problem, and that’s the question here - does this XBlock truly add value over producing something similar using basic problems?

• This is a very clear example of a tool that does not need to be in the core platform, as it is only usable by specific subjects. That does not mean it doesn’t have value, especially if expanded, just that it is not core due to not being suitable for all use cases.

  • I could, for example, see this being used for code analysis questions if the XBlock were extended with support for line numbering and syntax highlighting, to ask learners questions about specific lines of code. Right now it only works for text analysis.

• The point of highlighting text in the passage is to provide the passage as wider context for the question. Yet when there are multiple questions about different passages of the text, you lose some of that context.

• I do not know how this problem type affects the accessibility of materials. It is likely that accessibility testing would be required.