[BD-05] ORA v3: Usability + Moderation

Owned by Michelle Philbrick (Do Not Use) (Deactivated)
Last updated: May 16, 2022 by Wes Mason [he / him / his] (Deactivated)Version comment
12 min read

 

Blended Project ID

BD-05

edX Product Lead

@Marco Morales (Deactivated)

edX Technical Primary

@Justin Lapierre (Deactivated)

Provider

OpenCraft

Provider Lead

@Giovanni Cimolin da Silva

Calendly Scheduling Link

Slack Channel

#ora-dev,

Meeting Cadence / Preference

Scheduled bi-weekly project team syncs

Action Item Tracking

Use slack first, can also add action items on project status pages or provider status page

Escalation Path

https://openedx.atlassian.net/wiki/spaces/COMM/pages/1529676506 Escalation path outlined here

Relevant Discovery Documents

Discovery doc

Status Update Frequency

Bi-Weekly

Blended Project Epic (Tracks pull requests)

BLENDED-91

Target Completion Date

TBD 2020

~Dev Hours Per Sprint Expected

27 hours this sprint

Total Hours Used

88% - 2/8

Open Action Items

Status Updates

Time Frame

Updates & Actions

~Hours

Time Frame

Updates & Actions

~Hours

June 15 - June 29

Work done this sprint:

  • TNL-7273 Change ORA Report Download to use Usernames instead of anonymous IDs

Ongoing work from this sprint:

  • TNL-7274 Add a Grade Status Summary Message

  • TNL-7272 Separate “Assessment Steps” and “Schedule” from Settings Tab

New work for next sprint:

  • TNL-7282 Support Flexible Peer Grading Averaging for Learners delayed / in Peer Grading step (Postponed due to LSE tickets pushed in)

  • TNL-7281 Change ORA Report to Include Problem Name and Location

I've also looked at React integration points for both LMS and Studio, and, as discussed in the meeting, I'll send an email following up on that.

How is the budget coming along?
We've used 75.3 hours of the 331 hour budget.

From these, around 5 hours were used to onboard developers in the project.

 

22.7% - 6/29

June 29 - July 13

Work done this sprint:

  • TNL-7281 Change ORA Report to Include Problem Name and Location

Ongoing work from this sprint:

  • TNL-7274 Add a Grade Status Summary Message

  • TNL-7272 Separate “Assessment Steps” and “Schedule” from Settings Tab

New work for next sprint:

  • TNL-7282 Support Flexible Peer Grading Averaging for Learners delayed / in Peer Grading step

  • TNL-7308 Add ORA Zipped File Download for Submission Text + Attached Files

  • TNL-7311 Rewrite Open Responses Dashboard in React

  • TNL-7310 Discovery: pluggable ORA Submission Editor

We've used 105.8 hours of the the 331 hour budget.

32% - 7/13

July 13 - July 27

Tasks from last sprint waiting for edX review:

  • BB-2504 ORA v3 - Add a Grade Status Summary Message

Tasks from current sprint still in progress:

  • BB-2502 ORA v3 - Separate “Assessment Steps” and “Schedule” from Settings Tab

    • This is almost done and should be ready for edX review soon.

  • BB-2505 ORA v3 - Rewrite Open Responses Dashboard in React

  •  BB-2620 ORA v3 - Discovery: pluggable ORA Submission Editor

New tasks:

  • BB-2651 ORA v3 - Add ORA Zipped File Download for Submission Text + Attached Files

  • BB-2621 ORA v3 - Support Flexible Peer Grading Averaging for Learners delayed / in Peer Grading step Story

  • BB-2731 ORA v3 - Rewrite Studio block selector using React

  • BB-2732 ORA v3 - Make peer dependent steps skipable 

We've used 118.8 hours of the the 331 hour budget.

Some work from last sprint was postponed because of pending LSE tasks, so we’re ramping up the capacity on this epic in this sprint.

July 27 - August 10

Tasks from last sprint waiting for edX review:

  • BB-2502 ORA v3 - Separate “Assessment Steps” and “Schedule” from Settings Tab

    • This is almost done and should be ready for edX review soon.

  • BB-2505 ORA v3 - Rewrite Open Responses Dashboard in React

    • Paused

  • BB-2651 ORA v3 - Add ORA Zipped File Download for Submission Text + Attached Files

  •  BB-2620 ORA v3 - Discovery: pluggable ORA Submission Editor

Tasks in progress:

  • BB-2621 ORA v3 - Support Flexible Peer Grading Averaging for Learners delayed / in Peer Grading step Story

New tasks:

As per @Marco Morales (Deactivated)'s request, we’re pausing this epic for 1 to 2 sprints until the TNL and OpenCraft team have more availability.

This pause will allow both teams to work on less projects at the same time.

We've used 184.3 hours of the the 321 hour budget.

August 10 - August 24

Tasks waiting for edX review:

  • BB-2502 ORA v3 - Separate “Assessment Steps” and “Schedule” from Settings Tab

  • BB-2651 ORA v3 - Add ORA Zipped File Download for Submission Text + Attached Files

  •  BB-2620 ORA v3 - Discovery: pluggable ORA Submission Editor

New tasks:

As per @Marco Morales (Deactivated)'s request, we’re pausing this epic for 1 to 2 sprints until the TNL and OpenCraft team have more availability.

This pause will allow both teams to work on less projects at the same time.

We've used 207 hours of the the 321 hour budget (we might recoup some hours after next sprint).

September 21 - October 5

What's left to do? (Is it on track to be done before the deadline and within the hours budget?)

  1. Studio changes: No need to rewrite block selector into React component, just implement functionality in Backbone and the current mechanisms. 

  2. Pluggable editor: blocked on upstream.

  3. LMS Instructor dashboard rework: build new MFE.

New tasks:

Following sprint:

Tasks in progress:

Tasks waiting for upstream review:

We've used 262.4 hours of the the 321 hour budget.

October 5 - October 19

New tasks:

Following sprint:

  • Add integration of from LMS to new MFE

  • Update documentation

Tasks in progress:

Tasks blocked on edX:

Tasks waiting for upstream review:

We've used 301.7 hours of the the 321 hour budget.

October 20 - November 2

@Marco Morales (Deactivated) Doing some adjustments to the budget:

  • Total hours spent: 321h

  • Hours spent by newcomers above the estimates: 69h - we’ll absorb these hours.

  • Underestimated tasks - time difference: 10h

  • Scope changes/injections: 51h

    • 23h of work done related to React embedding, approach changed and now we’ll use an MFE

    • 12h to set up MFE

    • 12h ORA conversion to pluggable app

Considering those:

  1. The total amount of project hours used is 321h - 69h = 252h.

  2. We’ll need to add 61h to the current budget to finish all items in the project.

New tasks: no new tasks, all tasks are now blocked and waiting for reviews.

Tasks in progress:

Tasks blocked on edX:

  • https://openedx.atlassian.net/browse/TNL-7573

    • Instructor dashboard v2 (in MFE)

    • Student grade override views (in MFE)

    • UX dependency on this work as well

  • Implement pluggable ORA Submission Editor

  • Add integration of from LMS to new MFE

  • Update documentation

Tasks waiting for upstream review:

We've used 252 hours of the the 321 hour budget.

See note in the Update about budget calculation change.

November 2 - November 16

New tasks:

Tasks blocked on edX:

  • https://openedx.atlassian.net/browse/TNL-7573 - ready for review

    • Instructor dashboard v2 (in MFE)

    • Student grade override views (in MFE)

    • UX dependency on this work as well

    • Add integration of from LMS to new MFE

  • Implement pluggable ORA Submission Editor

Waiting:

Tasks waiting for upstream review:

We've used 270 hours of the the 321 + 61 hour budget.

December 14 - December 29

There’s a 2 sprint gap of updates here, this update will cover all items done in the last two sprints, as well as work planned for the next sprints and blockers.

Tasks completed and deployed to production:

Tickets done that unblock work:

Tickets in the current sprint & next sprint (until Jan 10):

Starting in the current sprint, as @Justin Lapierre (Deactivated) and @Mat Carter (Deactivated) request, we’ll use the EDUCATOR board to track tickets and work.

Pending discussions:

  • MFE API architecture? How to build ORA (mix of XBlock and Django app) APIs for new MFE and ESG?

    • Discussion started by @Ben Warzeski (Deactivated) on Slack, continuing on public channel and ticket

    • Decision here impacts the work being done on the MFE

New work scope (needs budget evaluation, estimates coming):

We've used 290 hours of the the 382 hour budget.

December 29 - January 11

?

 

January 11 - January 25

?

 

January 25 - February 8

I was on vacation and the two previous entries were not written. I’ll include a recap of tasks done and budget status on this update.

Tasks ready for edX review:

Tasks in progress:

Next sprints:

https://openedx.atlassian.net/browse/EDUCATOR-5363 will unblock us to implement the remaining work in the ORA repo.

We've used 339 hours of the the 382 + 20 hour budget.

The extra 20 hours came from a scope increase discussed in the last two meetings. See the notes and it’s comments below to get the full context.

Note: Other tracking docs have 416 hours as max budget, calculation for now to be based on that max for now. (339/416: 81.4% - 1/29)

February 8 - February 22

Waiting edX review:

Blocked waiting on requirements:

Done:

  •  Read discovery and test React based views on Xblock

Next sprint: 

  • Implement instructor dashboard waiting step - 8h timebox for sprint 240

Tickets left:

  • Update documentation



We've used 359 hours of the the 406 hour budget.

February 22 - March 8

Waiting edX review:

Done:

Next sprint: 

Tickets left:

  • Update documentation

We've used 372 hours of the the 406 hour budget.

Meeting Notes

Date

Attendees

Agenda

Notes, decisions and action items

Date

Attendees

Agenda

Notes, decisions and action items

2020-02-04

@Giovanni Cimolin da Silva @Marco Morales (Deactivated) @Sapana Thomas (Deactivated) @Justin Lapierre (Deactivated)

Status update

  • @Giovanni Cimolin da Silva Presented a demo of the ORA pluggable submission editor (simple editor, tinymce and codemirror)

    • Public sandbox linked on ticket

    • Ready for edX’s technical and product review.

  • make sass → webpack conversion in internal review, should land soon

  • Waiting on edX product requirements for ZIP file

  • Work unblocked on instructor dashboard: a react integration is now possible and the waiting step details will be delivered within the next two sprints

  • Documentation tasks: we’ll hold off on those until we finish all the implementation tickets, to reduce the ticket overhead and unecessary budget usage.

Project tracking/management

@Marco Morales (Deactivated) Previewed the new project tracking spreadsheet that will be used to track BD projects, analyze budget usage and provider capacity.

@Sapana Thomas (Deactivated)@Marco Morales (Deactivated) ZIP file structure requirements for implementation.
@Justin Lapierre (Deactivated) Review ORA pluggable submission editor
@Sapana Thomas (Deactivated)@Marco Morales (Deactivated) Product review on pluggable submission editor sandbox.

Jan 20, 2021

@Marco Morales (Deactivated) @Justin Lapierre (Deactivated) @Sapana Thomas (Deactivated) @Mat Carter (Deactivated) @Giovanni Cimolin da Silva @Ben Warzeski (Deactivated)

Status Update

 

Instructor dashboard implementation:

Discussed two possible implementations (outlined in the last meeting) and decided to follow the React approach, since it’ll be more “future-proof” and allow us to use the newer Paragon components.

The feature delivery depends on laying the groundwork and getting react working in the ORA instructor views, which is where there a risk of budget extensions (depending on the complexity of the implementation). @Ben Warzeski (Deactivated)'s discovery will help guide the implementation.

Pluggable ORA submission editor

Work is going through cleanup and getting tests/quality fixed.

OpenCraft’s team should configure TinyMCE following the guidelines from https://www.figma.com/file/LQIDop57o1FMsMiIsH5AhV/TinyMCE-Implementations?node-id=0%3A1

  • Close to demo / merge for pluggable editor

  • Next sprint: Downloaded file improvements, budget shift of 12 hours

  • EDUCATOR-5482 view access fixed! (same as remaining scope for Waiting column updates in ORA Dashboard)

  • Ben’s ongoing discovery on ORA xblock for reac twill be helpful for Giovanni’s upcoming React conversion work.

@Giovanni Cimolin da SilvaEmail Gabriel about new budget extension
@Giovanni Cimolin da SilvaCreate mock zip file with new structure and attach to https://openedx.atlassian.net/browse/EDUCATOR-5363
@Giovanni Cimolin da Silva Sandbox for ORA pluggable editor
@Giovanni Cimolin da SilvaCommunicate TinyMCE settings to assignee
@Giovanni Cimolin da Silva Create and schedule 2 tickets from new scope

 

Jan 13, 2021

@Marco Morales (Deactivated) @Justin Lapierre (Deactivated) @Sapana Thomas (Deactivated) @Mat Carter (Deactivated) @Giovanni Cimolin da Silva

Discussing remaining scope:

Instructor dashboard (waiting responses dashboard)

We have two possible paths moving forward:

  1. Use the remaining budget to implement it on the old interface (dropping react work to accomodate) and work on React integration in a future project or budget extension

    1. The task itself will take ~30 hours

  2. Continue React based implementation, implementing the groundwork to improve future implementations of the platform. Remaining budget (~60 hours) might not be enough to finish it.

    1. There’s a lot of risks in the delivery timeline if taking this path, it will be a solution supported for years to come.

    2. We need to get engineering “stakeholders” for MFEs and XBlocks in the discussion and ADRs.

    3. @Ben Warzeski (Deactivated) is doing a discovery related to this in the current sprint.

Pluggable ORA submission editor

Proof-of-concept implementation ready, wiPluggable ORA submission editorth CodeMirror implementation done.

A product issue came up during the implementation phase: ORA problems can have more than one prompt. During the meeting, we discussed first shipping the problem-wide pluggable editor setting, and then refine it in a follow up ticket.

Extra work from @Justin Lapierre (Deactivated) (estimates):

@Justin Lapierre (Deactivated) Check @Ben Warzeski (Deactivated)'s discovery after it’s delivered to help decide between approaches 1 or 2 for the instructor dashboard task.
@Giovanni Cimolin da SilvaEstimate EDUCATOR-5482 once have access to it.
@Justin Lapierre (Deactivated) Approved budget extension for the two injected tasks and ping @Marco Morales (Deactivated) and @Giovanni Cimolin da Silva to wrap up the budget discussion.
@Giovanni Cimolin da Silva Clean up ORA epic, add descriptions to all tickets missing it.
@Giovanni Cimolin da SilvaCreate follow up to Submission editor ticket.

Nov 18, 2020

@Kshitij Sobti , @Justin Lapierre (Deactivated) , @Marco Morales (Deactivated)

  • Weekly Status

  •  

  • Status

    • TNL-7577 (OpenCraft) - Reviewed and approved, pending deployment

    • TNL-7337 (OpenCraft) - PR is not attached to the ticket, requested from OpenCraft

    • TNL-7282 (OpenCraft) - PR is not attached to the ticket, requested from OpenCraft

    • TNL-7578 (OpenCraft) - PR is not attached to the ticket, requested from OpenCraft

    • TNL-7653 (OpenCraft) - Bug was found after rebasing, currently being worked

    • Requesting if it's possible for them to provide short video demos of work, still in discussion

  • Recapping UX Decision - will use table layout for Instructor Dashboard, explore use of cards in future effort (not in scope for this project)

  •  

Nov 10, 2020

@Giovanni Cimolin da Silva , @Marco Morales (Deactivated) , , @Mat Carter (Deactivated) , @Sapana Thomas (Deactivated) , @Justin Lapierre (Deactivated)

  • Blockers

  • ORA Dashboard UX

  • TinyMCE Update

 

Oct 28, 2020

@Giovanni Cimolin da Silva , @Marco Morales (Deactivated) , , @Mat Carter (Deactivated) , @Sapana Thomas (Deactivated)

  • Reviewed latest tickets and status

  • New ticket “Convert ORA to pluggable app” (in review)

  • Discussed Code Editor / Pluggable Editors

    • for now, Code Editor only as proof of concept, not ready for full release

    • Code editor POC in separate PR

@Marco Morales (Deactivated) to send draft of updated partner portal post for upcoming / recent changes
@Mat Carter (Deactivated) (or Justin) to review new “Convert ORA to pluggable app” PR (TNL-7653)

 

Oct 14, 2020

@Giovanni Cimolin da Silva , @Marco Morales (Deactivated) , @Justin Lapierre (Deactivated) , @Mat Carter (Deactivated)

  • Budget allocation

  • Open ticket issues

  • Minor Note on TinyMCE -

    • could enable future in-context comments

    • UX team might have TinyMCE theme to share back to this group (TBD)

  • LMS to New MFE Link: Banner in ora dashboard will be how people can get to new dashboard.

@Marco Morales (Deactivated) to send note to OC leads about budget hours shifts for visibility (65 to ORA, plus other BD-29 shifts)
@Giovanni Cimolin da Silvato create documentation task to flag what needs updating.
@Justin Lapierre (Deactivated) to connect with Ben P on documentation ask / next steps.

Oct 14, 2020

@Giovanni Cimolin da Silva , @Marco Morales (Deactivated) , @Justin Lapierre (Deactivated) , @Mat Carter (Deactivated)

  • Status Updates

  • Blocking edX Items

@Marco Morales (Deactivated) to schedule time for product review on remaining tickets!
@Marco Morales (Deactivated) to review permissions, still an issue on TNL project for mentions. (FYI @Michelle Philbrick (Do Not Use) (Deactivated) looks like this is still an issue but it is more specifically scoped to mentions not being possible.)

Sep 16, 2020

@Giovanni Cimolin da Silva , @Marco Morales (Deactivated)

  • Status Updates

  • Product Reviews

  • Blocking edX Items

    • TNL-7573

    • Reviews for TNL-7310, 7282, 7337

Sep 16, 2020

@Giovanni Cimolin da Silva , @Marco Morales (Deactivated) , @Justin Lapierre (Deactivated) , @Mat Carter (Deactivated)

  • Status Updates

  • Upcoming Efforts / Changes

Status

  • Pluggable ORA submission editor needs feedback

  • Zipped File Download delivered/completed

  • new ORA component category section on unit pages for authoring in progress soon

Sep 2, 2020

@Giovanni Cimolin da Silva , @Marco Morales (Deactivated) , @Justin Lapierre (Deactivated) , @Mat Carter (Deactivated)

Marco never came back to put notes from our meeting as intended

 

Aug 19, 2020

@Giovanni Cimolin da Silva , @David St. Germain (Deactivated) , @Marco Morales (Deactivated) , @Justin Lapierre (Deactivated) , @Mat Carter (Deactivated)

  • Transition for DSG - owner needed,

  • Discussed status update

  • Core Committer update / status

Aug 5, 2020

@Giovanni Cimolin da Silva , @David St. Germain (Deactivated) , @Marco Morales (Deactivated) , @Justin Lapierre (Deactivated)

Milestone Update

  • A - In Backlog

  • B - In Backlog

  • C - In Code Review

    • Complicated change, moved a fair amount of code

  • D - Deployed and Delivered

  • F - In Backlog

    • depends on React rewrite

  • H - In Progress (code review soon)

  • I - Deployed and Delivered

  • J - Deployed and Delivered

  • K - Deployed and Delivered

  • L - In Code Review (sandbox for testing)

  • Q - Discovery started, pluggable path forward defined. Needs feedback for next steps

  •  

 

Jul 15, 2020

@Giovanni Cimolin da Silva , @David St. Germain (Deactivated) , @Marco Morales (Deactivated)

  • Graded status message merged but not yet deployed, PR to be created.

  • Marco to review wireframe needs on ora-dev project.

  • React instructor dashboard views - discussion on Discuss.openedx.com

  • Open question here about ORA dashboard packaging / approach.

  • Pluggable ORA Submission Editor - discussion to be added to the JIRA ticket

    • discussed - no plan to migrate ORA xBlock to React for the time being. sounds like this unblocks some work / simplifies things.

    • discovery plan for pluggable ORA editors coming soon (Tiny MCE, code editor, simple text area etc)

  • Current Sprint:

    • ORA Zipped File Submissions - folder structure in progress and will be posted on ticket to be reviewed.

    • Support for Flexible Peer Grading

  • Next Sprint:

    • Studio Rework for Components delayed a bit given other priorities.

    •  

Relevant Jira Stories, Tasks, Bugs

This section links to all the development stories, tasks, and bugs connected to the single (or multiple) epics used to track the work on the Open edX JIRA site.

key summary type created updated assignee reporter priority status resolution
Refresh

Relevant Pull Requests linked to Blended Project Epic

This section will link to the edX internal Blended project epic, and contains all Pull Requests related to this Blended Project.

key summary type created updated due assignee reporter priority status resolution
Refresh