Targeted interventions: Event-triggered notifications and notifying user groups
This document contains use cases identified through discovery with course delivery team members, who want to be able to communicate with subsets of learners in their course and/or communicate with learners when they do something on the platform (event-triggered notifications).
Note: I imagine a lot of these use cases will not be introduced with a notifications MVP, but I want to convey the ways in which we envision User Groups, Notifications, and Events to collide in the future.
First, I could see building out basic Notifications functionality (a course author or administrator can author and schedule or send notifications; users can specify their notifications preferences; users can view and dismiss notifications) all being built out first.
From there, I’d expect we’d need to facilitate batching of notifications similar to how other apps batch notifications (you have N deadlines) to make them user friendly for learners.
Beyond that, we’d want to allow course authors to send notifications to a subset of learners in their course (user groups). And finally, we’d want to facilitate being able to have events trigger notifications. These are the two use cases I’ve really focused on in the document below.
Use Cases
As a course delivery team member, I want to..
Communicate with:
All users on the platform, in an organization, or in a course
A subset of users on the platform, in an organization, or in a course
Note: These subsets of users will be created through the new user grouping feature we’re hoping to introduce in the Ulmo release and continue to build upon in future releases. I envision users that are able to draft notifications would be able to select a user group they’ve created as the recipient of a notification. We’ll be focusing on creating user groups at the course level first and from there we’ll start to expand that to allow administrators to create user groups of users across the platform further down the line.
Via:
Manually-sent communications: A notification that is sent upon button push
Scheduled communications: A notification that the user drafts and schedules to be sent at a later time and date (these can include scheduled, recurring communications)
Event-triggered communications: A notification that is drafted and sent to all applicable learners or all applicable learners among a subset of learners when a specified event occurs
Again, these subsets of learners will be created through the new user grouping feature we’re hoping to introduce in the Ulmo release and continue to build upon in future releases. I envision users that would be able to draft notifications would be able to select a user group they’ve created as the recipient of a notification.
So that… I can keep users informed, engaged, and encouraged.
I think one BIG question that may be worth asking as we shape the requirements for platform notifications, user communication preferences, and communication receipt: is how do notifications, mobile push notifications, and email communications overlap (if at all)? Who decides / what determines if something is an email, a push, or a platform notification? Message length, user preference, message context should all go into determining what communication is delivered by what mechanism. I’m not 100% sure what industry best practice is with regards to this. (Note that being able to preview email messages before sending them did come up as an important requirement for email communications during discovery).
As a course delivery team member, I want to be able to specify that a scheduled, recurring, or event-triggered notification is only sent to a learner once, so that learners don’t receive the same notification multiple times. ← This functionality will need to be in place before/alongside event-triggered and scheduled notifications to user groups.*
Sending notifications to a user group
Example of scheduled/recurring communications to a user group
*(See requirement to be able to specify that a scheduled, recurring, or event-triggered notification is only to be sent to a learner once above)
When a learner has not engaged with the course in a while
Daily, send a notification to a user group made up of learners who have not yet passed the course and who have not engaged with the course in N days (would likely make the most sense as an email or even a mobile push notification) (this is an Aspects-only event)
When a learner drops off after the course component where I lose the most of my learners
Daily, send a notification to a user group made up of learners enrolled in the course that has engaged with X unit page, X problem, or X video, but has not engaged in any content beyond this component (the component in the course where I lose the most learners), I want to send them a nudge to get them excited about the content that follows.
When a learner has not yet attempted an assignment that is almost due
N days before a graded subsection is due, I want to send a communication to learners who have not yet attempted any problems in a graded subsection.
When a learner might be ready to attempt an assignment, but has not started it
Daily, send a communication to a user group made up of learners on X enrollment track that have engaged with the last unit page, problem, or video before a graded subsection, but have not engaged in any problems in the graded subsection, I want to send them a nudge to encourage them that they are ready and encourage them to attempt the assignment.
When a learner has ORA reviews to do
Daily, send a notification to a user group made up of learners that have one or more incomplete ORA reviews to do.
Sending event-triggered notifications to all applicable learners or to a user group
As a course delivery team member, I want to be able to communicate with learners when (see events listed below), so that I’m able to provide feedback and/or encourage learners, to keep them engaged in their coursework, and to offer them alternatives and/or resources to help them succeed.
Example use cases of event-triggered notifications from discovery
Event-triggered notification to all applicable learners:
When a learner enrolls in the course, I want to send them a nudge to get them excited about what they’ll learn in the course.
When a learner chooses enrollment mode X, I want to send them a tailored nudge to get them excited about what they’ll learn in the course that is specific to their enrollment track.
When a learner enrolled in my course completes an assignment and their grade on the assignment is N or higher, I want to congratulate them on a job well done and encourage them to keep going.
When a learner enrolled in my course completes an assignment and their grade on the assignment is N or lower, I want to let them know if they get another attempt at any of the problems in the assignment, what resources they should re-review before revisiting the assignment, or even suggest a beginner-level course on the assignment topic.
When a learner earns N grade or higher in the course, I want to congratulate them and suggest another course to them.
When a learner earns a certificate in the course, I want to reach out to them to congratulate them and suggest another course to them.
Event-triggered notification to a subset of learners/users:
When a learner creates a discussion post, I want selected staff in my course to get an email or a notification letting them know.
When a learner submits their final allotted attempt for X problem and the response is incorrect or partially correct, I want to provide even more detailed feedback to them as to what the correct answer was and why. I want to point them to resources to help them better understand the concept, so that they can feel confident moving forward.
When a learner submits a correct response for problem X, I want to congratulate them and encourage them to keep going.
Events that course delivery teams want to be able to use to trigger a notification:
Learner enrolls in a course
Learner changes enrollment mode to X (Learner becomes verified learner)
Learner posts a discussion post
Learner has earned N grade or higher in a course
Learner has earned a certificate in a course
Learner has submitted a response for all questions in a graded subsection
Learner has earned N grade or higher (or lower) in a graded subsection
Events that trigger notifications are pluggable
Other events that might come up in the future, but are not an immediate need:
Learner has engaged with a specific unit page in the course
Learner watches X video (an Aspects-only event)
Learner has submitted a response to X ORA problem
Learner has a review to submit for X ORA problem