Learner Progress Tracking Personas & Needs
- Matthew Harrington
Analytics
Organisation Administration
As an organisation administrator interested in the overall effectiveness of courses across my learning platform, I have the following needs related to learner progress tracking:
Tracking on average progress of members of courses in order to identify outliers
Scenario: Course A and Course B are intended to be comparable in terms of effort. Participants started their courses at the same time. Course A participants are on average 20% of the way through their course, while Course B participants are on average 50% of the way through their course, which indicates a discrepancy between the two courses, whether between users or between course content.
On demand reporting for learner progress from different groups of users
Scenario: Training is sold to a partner organisation. That partner claims that none of their users have started their training in an attempt to claim a refund or discount. Progress data will reveal whether this is true.
Integration of progress reporting into other organisational systems via API
Scenario: The LMS is one learning platform among a stable of common platforms integrated into university systems or an LXP. Learner progress metrics need to be reported to centralised dashboards in order to be displayed alongside courses on other platforms.
Organisation administrators on the whole require aggregate-level data, which typically cascades upwards from course-level data available to course staff. They are less likely to care about the data of specific learners on a day-to-day basis, and typically require dashboards, reports, and system integrations with BI platforms and other tools.
Course Staff
As a member of staff concerned with the success of a specific course, I have the following needs related to learner progress tracking:
A report showing the average progress of learners in the course
Scenario: An opportunity to create some ad hoc content such as a live session to improve the course experience has arisen, but the instructor wants to make sure the majority of learners will be able to understand the content to ensure they aren’t discouraged and that the content is relevant to what they are currently studying. The instructor is able to use this data to plan the session so that it is relevant to the majority of learners, despite not being strictly bound to the course schedule.
The ability to review the progress of specific learners, measured against the average progress of the course
Scenario: Learner A has completed 25% of an instructor-paced course. The rest of the learners in that course have on average reached 42%. The instructor is able to intervene to help ensure that the learner is not struggling or at risk of dropping out.
The ability to review which content learners have and have not completed
Scenario: Analysis has identified through a range of markers and indicators that a group of 15 learners have dropped out, or are likely to, and are no longer participating in a self-paced course. Reviewing their progress, all 15 reached an especially difficult subsection of the content before stopping, which may indicate the difficulty of the content is causing problems for learners.
Reports showing how long it is taking for learners to complete each item of content or collection of content on average
Scenario: Estimates for a piece of content say that it should take learners 50 minutes to complete this subsection, but learners are taking 3 hours. Something may be wrong that is causing learners to take far longer than anticipated. After further analysis, it turns out learners are actually having to rewatch one of the videos repeatedly because it doesn’t do a good job of explaining the content, and it should be replaced.
Course staff are generally more responsible for the success of individual learners on the course, making learner progress tracking more relevant to instructor-paced courses. They also have a focus on ensuring the ongoing success of a course by identifying pain points and areas that need improvement, and progress tracking can be vital for this, allowing course staff to iterate on content that is causing progression issues. Signs of this include content that is taking learners longer to progress through than anticipated, content that causes gaps in learner activity (such as groups of learners not studying a section for a few days after studying every day consistently), and content that causes learners to stop progressing altogether.
Staff often use a lot of the metrics that learners care about in order to identify potential issues that specific learners are having, and where those issues affect many learners, it presents an opportunity to improve that content for all learners.
Learners
As a learner in a course whose sole interest is in my own progress, I have the following needs related to learner progress tracking:
I would like to know how far through the course I have gotten, and how much more there is to do
Scenario: This part of the content is hard and boring, but if I know there are only two more units to work through, I can push through to the end.
I would like to know if I have completed as much as is expected of me or not.
Scenario: I can’t get help if I don’t know I need it. I want to know how much I need to do today/this week in order to stay on pace.
I would like to know how much time that it’s going to take me to get through the rest of this content
Scenario: I have 45 minutes left to study before I have to get back to work. Based on what I’ve completed so far and what’s left, can I complete this section, or should I stop at the end of this subsection?
I need to know how close I am to achieving a passing grade
Scenario: I’ve completed 4 assessments and it’s not gone well. Based on the work I’ve put in, am I even going to pass if I continue the rest of the course?
Most learners only truly care about their progress in order to justify putting more effort into the course, which makes progress tracking a double-edged sword, as without an instructor or tool to help convert their progress into motivation, learners are very capable of demotivating themselves, for example by getting discouraged by their progress not matching the projected norm provided by the course (“this is week 2 content, but I’ve still not finished week 1 yet…”) or by the workload appearing overwhelming (“All that and I’ve only completed 5% of the course?!”).
On the other hand, progress tracking can help to keep learners going when they would otherwise drop out (“I’m over half way, I can do this!”) and justify their effort by instilling pride in the learner (“I’m in the top 20%! I’m great at this!” and “I’m on track to pass the course if I keep this up!”).
This means that learner progress metrics have to be extremely intentional in nature, and where possible, should provide learners with actionable insights to prevent them from misinterpreting the data. For example, rather than simply telling a learner “You’ve completed 5% of the course”, you can tell them “You’ve completed 5% of the course, let’s try to get to 10% by the end of the week to stay on pace”, providing smaller milestones relevant to that learner’s progress to keep them from growing discouraged, while also ensuring those milestones are actually achievable - for example “let’s try to get to 95% by the end of the week” is not a useful nudge for the learner at 5%, particularly if it’s Saturday already.
Few platforms actually do this due to the inherent difficulty involved, and this typically results in one of two approaches.
The platform presents raw data, and just hopes for the best
The platform breaks down the data it presents without aggregation (you’ve completed this content, but not this content - no expectations, totals, or implicit judgement)
Which of these is more effective is a subject worthy of research, not just anecdotal speculation.