Phase 1: Aspects x Modular Learning
Overview
Phase 1 of Aspects x Learning Pathways represents the initial work required to allow Pathway Administrators and Course Teams to monitor and manage their pathways and the learners enrolled in those pathways. In Phase 1, we’ll be adding one new dashboard to the suite of dashboards available out-of-the-box with the Aspects plugin and we’ll be expanding one existing dashboard (the Individual Learner Dashboard) to include pathways data.
Phase 1: Aspects x Modular Learning Epics
Epic 1: View data for a single pathway
Dashboard Overview
Who is my audience?
Pathway administrators and course teams
What are their business goals and objectives?
I want to be able to keep track of the pathways we’re currently running
I want to be able to pull reports on high-level pathway metrics for leadership and other pathway stakeholders
I want to be able to manage my current pathways and make decisions about item content updates, items included in a pathway, figuring out where and when to nudge and/or check in with learners, etc.
What specific questions do they need answers to?
What does enrollment look like for my pathway?
How many learners have completed my pathway?
Where are my learners getting stuck? Dropping off? (We aim to answer this with the Item Completion Rate chart and by surfacing the item with the lowest completion rate).
How many learners have earned what certificates for my pathway?
How often will the dashboard be reviewed?
Occasionally: For some, managing pathways day in and out, I’d guess daily or weekly. For others (higher level leadership, course/pathway teams) I’d guess monthly or quarterly.
Dashboard Requirements
I want to be able to view information for all Pathways I have access to and be able to filter this Pathway Dashboard by Pathway(s) (shows all runs of a pathway) or by Pathway Run(s). By default, this dashboard is not filtered in Superset (in the future, we may embed this dashboard somewhere within the LMS - for which it will be filtered by a single pathway or pathway run, but not for Willow.
All of the tables, charts, and numbers on the dashboard update based on what pathway or pathway run(s) are currently filtered.
This dashboard will live in Superset for the Willow release.
Pathway Overview
Pathway Summary (table) that shows the following information for each filtered Pathway/Pathway Run
Pathway
Pathway Run
Organization
N of learners currently enrolled
N of learners that have passed all pathway courses to date
Pass rate (%) ((N of learners who have passed every course in the pathway/ N of learners enrolled in pathway)x100)
Average time from start to pass all courses in the pathway of those who passed (in weeks)
N of pathway items
Item with highest pass rate
Item with lowest pass rate
N of pathway certificates earned
Note: User can sort this table by any column (except for Item with highest or lowest completion rate columns)
Enrollment & Completions
Pathway Enrollment and Completion
N Enrolled (big number) ← number of learners enrolled in all currently filtered Pathway(s) (if the same learner is enrolled in multiple pathways, I’d count them once regardless of the number of pathways they’re enrolled in).
N Passed (big number) ← number of learners who have passed all courses within currently filtered Pathway(s)
Cumulative Pathway Enrollments Over Time (area chart)
Y-axis is N Enrollments; X-axis is Time
Users can update the time grain and time range of chart
Pathway Item Completion
Pass Funnel (funnel chart) showing the number of learners who have completed (passed) 1 item, 2 items, 3 items, etc. ← This chart only renders if filtered on a Pathway or Pathway Run (if not possible, we should add helper text to nudge dashboard users to filter by a pathway/run to use this funnel)
Item Pass Rate (bar chart) showing the pass rate ((N of learners who have passed the item in the pathway/ N of learners enrolled in pathway)x100) learners who started each pathway item
Item Completion
Pathway name
Pathway run
Item name
N learners passed
Pass rate % ((N of learners who have passed the course/ N of learners enrolled in course)x100)
Average time from start to (pass) (in weeks)
Link to Course Dashboard if item is a course (opens in new tab)
Learner Behavior
Progress and Activity
Progress (histogram) shows % of courses passed of pathway along the x-axis and N learners along y-axis (by default shows Pathway progress can be filtered by a Pathway, Run, or Pathway Item) ← (If a single learner is enrolled in multiple pathways, I want to see that learner’s progress represented for each pathway run they are currently enrolled in).
Recent Pathway Activity (bar chart) that shows the following information for the pathway:
N passed learners
N incomplete learners active within 7 days
N incomplete learners inactive for 7 or more days
Recent Item Activity (bar chart) that shows the following information for each pathway item, where each bar represents a pathway item (course)):
N of passed learners
N of incomplete learners active within 7 days
N of incomplete learners inactive for 7+ days
Assignment Engagement and Performance
Can filter the following item below by time range:
Assignment Engagement (bar graph that plots graded subsections (in order, when applicable) on the x-axis and number of learners that engaged with that assignment on y-axis)
Average Assignment Score (bar graph that plots graded subsections (in order, when applicable) on the x-axis and the average score for that assignment among learners that engaged with that assignment on y-axis)
—----
Enrolled Learners Overview (table) ← Only to show this section if PII is enabled for the instance
Learner Name <link to individual learner dashboard>
Learner Username
Learner Email
Pathway Run
Enrollment Date
Last Active Date
Pathway Status (Not Started, Started, Passed)
Pathway Performance Progress (N of pathway items passed out of N items in pathway)
Pathway Credentials Earned (list the pathway credentials the learner has earned and the credentials they earned from items within the pathway)
Note: If the learner is enrolled in multiple pathways among the pathways currently filtered by the dashboard, this table shows 1 row per filtered pathway, so the same learner could appear in multiple rows of this table.
User can filter by pathway, run, status, last active date range and can cross filter using any of the above Progress and Activity or Content Engagement Summary charts
User can sort this table by any column (by default this table is sorted by last active date (most recent to the top, least recent to the bottom)
—----
Outcomes
Certificates Earned (bar chart) where the x-axis shows the name of each certificate available for the currently filtered pathways and for the courses within the currently filtered pathways and the y-axis shows the number of certificates earned
--
Epic 2: Add pathways data to individual learner dashboard
Dashboard Overview
Who is my audience?
Pathway and Course Instructors
What are their business goals and objectives?
I want to be able to keep track of my learner’s progress, success, engagement and completion for their pathways and courses.
What specific questions do they need answers to?
I want to be able to see what courses the learner has enrolled in
I want to be able to see what pathways the learner has enrolled in
I want to be able to see what pathway items they’ve started, they’ve not started, and they’ve completed.
I want to be able to see what percent done they are with the pathway items that they need to do.
I want to be able to see what certificates they’ve earned to date.
I want to be able to see what graded subsections they’ve started, not started, and completed.
I want to be able to see how long it took the learner to complete each pathway item and the pathway as a whole.
I want to be able to see when they enrolled, when they started and completed each assignment and each pathway item and each pathway and earned each certificate.
How often will the dashboard be reviewed?
This very much depends on how often I need to touch base with my learners, monthly, weekly, etc.
Dashboard Requirements
From the Instructor Dashboard Reports tab, this is filtered by the specific course that the instructor is looking at in the Instructor Dashboard.
From Superset, this is not yet filtered by anything by default, but the user can filter this dashboard by:
Learner
A single pathway, pathway run, course, or course run
By one or more pathways, pathway runs, courses, or course runs
By learning context (show me only pathways data; show me only course data)
By status (Started, Not Started, Completed)
All tables and charts throughout the dashboard filter based on these dashboard filters.
Embedded Individual Learner Dashboard tab does not render at all if all PII is turned off on the instance.
Overview
Update Learner Information (table)
Name
Username
Email address
Add Executive Summary with helper text: The following summary presents individual learner information based on the filters that are currently applied to the dashboard.
Learner is currently enrolled in N standalone and N pathway courses
Learner is currently enrolled in N pathways (is it possible to show this only if the instance has 1 or more pathways? Assuming no, but figured I’d ask)
Learner has passed N courses and N pathways
Account created on
Last active on
Learner has earned: N certificates
Learner has mastered: N competencies (is it possible to show this only if the instance is using competencies?)
Enrollment (table)
Learning Context: Course or Pathway
Course or Pathway
Course or Pathway Run
Enrollment Date
Last Visited (date)
Status (Started, Not Started, Passed)
Current Course Grade (only available for courses, not pathways)
Time to Pass: # weeks from start to pass (if passed)
For Pathways: From first course content engagement to when all courses in the pathway have passed
For Courses: From first course content engagement to meeting passing grade
Note: This table is sorted by Status (Z-A Started, Not Started, Completed) by default
Activity and Assessment
Add Activity Summary (bar chart) that shows the following with the helper text: Please note: The following chart presents individual learner information based on the filters that have been applied to the dashboard.
N of not yet passed enrollable items the learner was inactive in for 7 or more days
N of not yet passed enrollable items the learner was active in within the last 7 days
N of enrollable items the learner earned a passing grade in
(Clicking on one of these bars filters the learner’s current enrollment table)
Assignment Engagement (table)
Course Name
Course Run Name
Assignment Name
Assignment Type
Status: Not Started, Started, Attempted All (All problems attempted)
Assignment % Done (Note: Is this a metric we can measure? If not, we can omit for this release)
Current Grade: Assignment Grade
Competencies Assessed (if any) (List of competencies)
Updated when filtered by a Course Run, Course, Pathway, Pathway Run or Competency (can filter by clicking on a Competency row in the Competency Snapshot table above).
Achievement
Competency Snapshot (Only show if on a competency-based instance)
Competency Name
Competency Status: Not Started, Partially Attempted, Attempted Not Demonstrated, Demonstrated
Sorted by Status by default (A-Z)
Certificates
List of Certificates Earned & In Progress
Certificate Name
Certificate Status (Earned, In Progress)
Date Earned
Sorted by Most Recent to Least Recent
Content Engagement
Existing Pages, Problems, Videos tabs are accessible at the bottom of the above content
Note: I’d like to think through how to improve upon the engagement charts at the top of each tab so that it better shows learner drop off.
Epic 3: Add Competency Dashboard
A new Competency dashboard exists in Superset that can be filtered by Organization, Competency, Course, or Pathway. All of the following charts and tables should update based on how the dashboard has been filtered.
Status Snapshot
Status Breakdown charts that show:
Competency Status Count (stacked bar chart) N of learners who have Not Started, Partially Attempted, Attempted Not Demonstrated, Demonstrated each competency on the Y-axis and the Competency (of the currently filtered competencies) on the X-axis
Competency Status Percentage (grouped bar chart) % of learners enrolled in some element to assess each competency who have Not Started, Partially Attempted, Attempted Not Demonstrated, Demonstrated each competency on the Y-axis and the Competency (of the currently filtered competencies) on the X-axis
Learner Attainment
For instances where PII is turned on, I want to show a table that shows me data about individual learners' competency mastery. The dashboard should be designed in such a way that this section is completely hidden or this table is missing and the section is unobtrusive when PII is turned off. In addition to the dashboard filters, the table can be filtered by Status and sorted by any column, and it should show:
Name
Username
Email address
Competency
Competency Status
Mastery
Competencies Mastered Over Time (area chart) where the X-axis label is time and the Y-axis label is N Competencies Mastered of the currently filtered competencies (user can update time grain of chart)
Time to Mastery (box plot chart - need to see both to decide) shows the time from Started to Demonstrated (in weeks) on the Y axis and Competency on the X-axis
Epic 4: Update to what user can see when PII is disabled in Aspects
As an instance operator, I want to hide the Individual Learner Dashboard from the Instructor Dashboard tabs shown on the Reports tab of the Instructor Dashboard, so that my course teams aren’t presented with Dashboards that they can’t use on the Instructor Dashboard.
Not in scope for Willow Release
Improvements to the Individual Learner Dashboard
Add Item Interaction Progress Summary vertical bar chart that shows the current % done for each currently filtered course(s) and/or pathway(s)
Item Interaction Progress Over Time line chart that shows the % done over time for each currently filtered enrolled course and/or pathway
Note: Can update time grain and/or time range of chart
Note: Can click on a single line in this chart to view only the information for that line (course or pathway)