Toggle rubric visibility for learners
As it may be helpful for learners doing an ORA assignment, add the ability to toggle visibility of the ORA rubric upfront to learners before submitting their ORA assignment.
This should support peer-graded ORA, as well as staff-graded ORA.
Rubric display must be a toggle and still allow to hide it.
Rubric should be hidden by default.
We’ll create a collapsible div inside the “Your Response” step of every ORA.
It’s default state is collapsed.
It should be titled “What will this assignment be graded on?”
It should contain every criterion for the item’s rubric, and each criterion should list all options therein.
Because (as noted below) we’re using the same rubric template that is used in other steps, each criterion should also be collapsible.
Inside this div, we’ll render the same rubric template that we show in, for example, the “Learn to Assess Responses Step”.
The main difference: don’t render the radio buttons in this new div. Add a boolean flag to the template/context to control when these buttons should be rendered.
Don’t create any new CSS styles unless necessary.
This new rubric div should be placed before the first prompt in the Response step.
We want an option when editing the block in studio to control whether this rubric is visible to learners in the Response step.
This should be stored in a new XBlock field, probably scoped to “settings”. Call it something like show_rubric_during_response. It’s default value should be False, that is, this is not rendered unless toggled to true in the ORA studio settings by course staff.
You’ll have to update the studio template to include this option, too.
NOTE: Feedback below from will be considered for a future version of the learner-facing rubric.
Steps to Reproduce
Reason for Variance
User Impact Summary
For future efforts - please see my manually created chart which shows a single criterion in reference mode. This is in situations where the course team chose not to use the ORA “training step” and they created it by themselves just before the ORA.
Bottom line: Our priority is to show something to learners preparing to submit their ORA work. If that means showing them the rubric used as a tool for now, that’s certainly ok.
Once a rubric gets longer than 1 or 2 criteria, it will be necessary to make it easier to read as a reference. Many ORA rubrics I’ve seen typically contain 2-4 criteria.
A prompt like “What will this assignment be graded on?” would be great when a learner toggles the rubric before submitting their response.
When used as a reference, the rubric should take a form different from how we show it today. I recommend summarizing the entire rubric in one chart so that it can be scanned across criteria and point values. (See example below)
When used as a tool, to actively grade an assignment (self, peer, staff), I think our current layout helps focus attention appropriately. It takes each criterion and itemizes each band descriptor with point values.
We could certainly improve this by including the criterion name above each prompt. Right now we only show a sentence which explains what the evaluator needs to do. It is missing a label which would provide better context and organization.
It sometimes surprises educators when they don’t see the name of the criteria listed while it’s being used as a tool, too.
On the authoring side, we can talk more about what things should be labelled. I do get raised eyebrows from folks who are new to building ORAs, even from folks who are proficient at rubric writing.
CC Let me know if you need more input on this.
Here are the notes captured from our meeting with some commentary:
Both Ben P and Kevin M agreed that it would be useful for all users to be able to see some rubric or guidance on how an ORA assignment type will be graded.
There was some discussion surrounding the situations where the rubrics are already displayed such as for “peer” or “self” assessment learners. Staff members see the rubric when grading an ORA but, currently the Learner does not for staff graded ORAs.
There was additional discussion about possibly wanting to adjust wording and messaging, such as “What will we be graded on?” as opposed to “Rubric”. Also, we discussed prior ORAs will not show the rubric by default, but newly created ORAs display of rubric will be decided by the team.
Team will meet to discuss next steps.
can you add the notes you took from our meeting with Ben and Kevin as a comment on this ticket?
Just moved into “to be triaged” so I can take a look, and talk to