Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 3 Next »

Overview

The Uni Digital initiative (Spanish universities consortium) is interested and motivated to contribute the ability for problems with multiple attempts to support alternative grading strategies.

In problems that allow multiple submissions, if the learner is allowed to submit more than one attempt, the current scoring function only takes the score of the last attempt. 

With this modification, course creators will be able to select the desired grading strategy for the problem.

Strategies would include:

  • Highest score

  • Last score (this is what the platform currently does)

  • First score

  • Average score

Problem

  • The current single strategy of grading the last attempt could in some cases discourage learners from exploring a problem in more depth or punish them from trying.  It could also make it easier for learners to use the multiple attempts to guess the  correct answer without any negative consequence

  • A number of universities currently using Open edX for MOOCs and interested in leveraging the platform also for in-campus purposes believe this would help their course creators provide a better range of options when building course gradable components

  • Other LMS systems have this as an option by default with their features: Moodle, Canvas and Sakai for example

Use cases:

  • As an instructor, I want learners to engage with problems risk-free, that is to be able to submit an answer and use up to the last available attempt without fearing they will get a score that is lower than the one they already have. This would apply when it is particularly difficult to provide a response,  perhaps because multiple input boxes need to be filled,  I want the learner to know that if they provided some response that got, for example, a partial credit,  they can try again and try to provide a better one without risking the loss of the score they already obtained. In this case, the best grading strategy would be “highest score”

  • As an instructor, I want learners to engage with problems and reward a learner that gets the answer right in their first attempt, as opposed to one learner that needs to try 4 times to get it right. In this case my grading strategy of choice would be “Average score”

  • As an instructor, I want learners to engage with problems the way they always have. In this case my grading strategy of choice would be the default “last score”

  • As an instructor, I want to measure the learner's ability to get the response right in one attempt, but I also want them to engage with problems multiple times, and be able to explore other alternative answers, see the feedback, etc, basically use the problem as a learning activity after they submitted the first response and received a grade. In this case my grading strategy of choice would be “first score”.

Proposed solution:

The proposal is to add the “Grading strategy” as a new setting to the ProblemXblock and change the behaviour of the submit_problem ↗ method to accommodate the proposed grading strategies.

This change will give course creators some more flexibility when creating and configuring graded components (problems), it will level up the capabilities of the platform against other widely used open source LMS Solutions,  and most importantly it will provide improved ways for Learners in Open edX to engage with the problem components in a way that supports their learning.

Proposed visuals:

Current

proposed

New problem editor

Legacy settings in Studio

Student view

 There is a roadmap issue for this feature here: https://github.com/openedx/platform-roadmap/issues/261

Other approaches considered:

We have also considered forking the Xblock and building a new version with these changes, but since they are minor improvements and additions to the capability of the current component, it feels like a sub-optimal approach not to contribute the changes to the official component.

Competitive research:

  • How do Canvas/Moodle/Coursera solve this problem?

Moodle includes this option:

sakai has this option too:

Proposed plan for any relevant usability/UX testing

TBD

Plan for long-term ownership/maintainership

edunext is committed to building and contributing this work as part of the Uni Digital (Spanish government) project. As part of that commitment, edunext would commit to maintain the feature for a minimum of 2 years and after that, either find a suitable maintainer to hand it over to, or to follow the deprecation procedure in case the feature has any inconveniences or its maintenance is a burden that no one can carry.

Open questions for rollout/releases

TBD

  • No labels