Modular Learning: Data and Analytics Strategy and Product Requirements
Overview
Modular learning data needs summary
With the introduction of Learning Pathways (LPs) and Competency-Based Pathways to the Open edX platform a wide variety of personas including:
Authors
Instructional designers
Course team members
Admins
Learners
… all need access to the different data about their pathways, so they can easily:
Manage and keep track of pathway and competency-related activity, progress, and completion
Report out on key pathway and competency-related metrics
Identify ways to improve pathways for learners
Navigate pathways and competency frameworks and requirements as a learner
Track progress toward competencies and pathway completion as a learner
Purpose
The purpose of this document is to capture the myriad pathway and competency-related data needs for the Open edX platform all in one place, to align on what data belongs where (in Studio, in the LMS, in an Aspects Dashboard), to prioritize these needs, and to ultimately ensure that we think about the data component of each release holistically and plan to deliver value to users iteratively.
Scope and Definitions
When we talk about modular learning data and analytics, what do we mean?
What types of data are there? (Definitions with examples…)
Use Cases
Global management of modular learning components
As an administrator, I want to…
See learner enrollment, completion, and certificate information for the learners in my pathways, groups (not available for Willow release), and items (courses for Willow), so I can report this information to leadership and/or external funders.
[From Phase 1: Product Proposal: Modular Learning Paths] View a catalogue of all learning paths in the Django Admin (for Phase 1 MVP), so I can manage each learning path. (Note: In Phase 2: Learning Pathways: Studio, the requirements mention that this functionality must exist in Studio at least).
As an LP Administrator, I want to be able to get a sense of where my LP learners are and what they’re working on right now, so that I can better manage my pathways and/or my team in charge of selected pathways.
[From Phase 1: Product Proposal: Modular Learning Paths] Track progress across learners and paths so I can monitor completion rates.
Content performance and improvement
As an LP Administrator, I need a way of identifying learner dropoff, friction points and bottlenecks in my pathways, groups (not available for Willow release), and pathway items, so that I can devote resources towards revising and improving the right pathway elements.
As an LP Administrator, I want to know how long it took each learner and how long on average it takes each learner to complete a pathway so that I have this information to make decisions about pathway scheduling and decisions about future pathways.
Learner management and intervention
As an LP administrator, I want to be able to know which learners have enrolled in my pathway, have engaged with that pathway, have not yet completed the pathway, and have not engaged with the content in the last seven or more days.
As an instructor or course team member, I want to be able to view enrollment information, activity, performance, and engagement statistics for an individual learner, so that I can provide them individualized support.
As an instructor, I want to know how a learner is currently doing in each course on a pathway, so that if a learner is close to the passing threshold, I can tailor my outreach for them with this information in mind. (Example: the course or pathway end date is fast-approaching, but they only have a few percentage points to earn to pass that final course in the pathway).
[Nice to have, might be instance-specific] As an instructor, I want to know about the number of discussion forum posts each learner is leaving in each course in a Pathway and how many reactions they have, so that I can use this metric as one facet of determining learner engagement/involvement with their pathway.
Learner-facing data
As a learner, I want to…
[From Phase 1: Product Proposal: Modular Learning Paths] Track my progress across each course in a learning path and in the overall learning path through a status indicator (e.g. Not Started, In Progress, Completed), so I can keep track of where I am.
LMS and Studio
Currently being worked on. More detail to be added.
Aspects
See the following pages for Aspects Requirements:
Open Questions
If in the future we decide we want to grant learners credit or unlock progression on a pathway based on prior learning or skill assessments, how/where is that data tracked? What data do we need to capture in this case? (From Pathways Author Requirements Figma file - listed as a Medium-priority use case).