2023-09-18 Educators WG: Hyper-Personalized Nudge Emails

Recording

https://drive.google.com/file/d/157AujWuLDBDLHp3fbncpRzWRXHM3l096/view?usp=sharing

Date and Time

Monday, September 18, 1pm EST | 5pm UTC

Meeting Minutes

Alexandria Hsun Yen from The PostDoc Academy will give short presentation about a "personalized emails" experiment and its effect on student engagement and completion for a course of 1,200 students.

• 03:30 - Research Project Introduction

• 07:56 - Experiment Setup

  • Hyper-Personalized Nudge Emails (HPNEs) were delivered to an experimental group of approx 700 student during a 6-week, instructor paced course. A control group received normal, generic nudge emails every Monday.

  • Emails were customized based on student pace and progress through the course. Encouragement was given based on where they were, and call-to-actions were added for where to go next.

  • Tone of voice was more personal and recipients were encourage to give any questions or feedback to the sender (in this case, Alex’s email).

• 16:22 - Research Findings

  • HPNE’s appeared to increase participation by approx. 20% during the experimental run

  • HPNE’s increased participation mainly by pulling new non-participators into the course more effectively than the generic nudge emails:

    • Open

      

       

  • HPNE’s did not significantly increase completion.

    • Open

      

• 22:00 - The Week 5 Story: A Lesson in messaging

  • Week 5 saw a significant decrease in the effectiveness of HPNEs, mainly due to the fact that it failed to pull in new non-participators as other weeks had.

  • The team hypothesizes this is due to messaging. Week 5 messaging focused on flexibility and having “extra time” to complete a dense module. This messaging was not effective in drawing non-participators into the course.

    •  

• 26:08 - Q&A

• 38:19 - Guided Discussion Begins

  • POLL: How important are completion rates to you.

    •  

  • Every student in your course (an enrollment) has a different learner profile

  • It can be helpful to think about these profiles in learner archetypes

    • Marco M: Optimizers, completers, completionists, explorers, etc…

    • Mary Ellen W: Useful research from OpenMIT around Designing for Learner Motivations: https://medium.com/open-learning/focusing-course-design-on-learners-219bc000006d

    • Mary Ellen W: A lot of OpenMIT’s content is used by teachers who, for example, care more about getting the resources than about completion.

  • What we care about is “Meaningful learning”

    • Shira F: Completion rates are easy to measure. “Meaningful Learning” is a more difficult metric.

    • Meaningful learning is also more difficult to communicate.

      • I.e. Open edX shifting the conversation away from completion rates has been challenging

      • I.e. Proof of ROI (i.e. for grant-funding) is difficult to prove without completion rates.

• 49:00 “Blender” Dashboard demonstration from Campus.il

  • Encouraging teacher interventions in a blended learning environment by providing signals and insights about their students

  • Feedback on the dashboard from educators and creators is encouraged. Survey and other details will be posted in Slack.