Overview

This feature is switched off by default for this initial release, but can be turned on. 

Scope

For this release (and likely the first few releases), we must to support the creation of learner groups and interventions for learners at the course-level, but in the future we will need to support making groups at the: 

• Instance-level

• Organization-level

• Section, Subsection, Unit, and Component-levels?

Learner Groups

Open Questions

Where does this live?

• In the LMS, on the Instructor tab, between Open Response and Reports tab, on a tab called Learner Groups

  • Question: Will we be able to easily move this (Learner Group) page to an Admin Space if and when that space exists?

  • Note: Is this Instructor Dashboard MFEed? (No)

Are these groups to be manually refreshed? Automatically refreshed on a certain cadence? (A question for the delivery team to figure out if there is a reasonable cadence for a chron to refresh groups).

Learner Group Functionality

A Superuser, Admin, or Course Delivery Team member (for the courses they are course staff on) can…

• Define 

  • A group of learners that meet shared criteria (characteristics and/or events)

  • A list of learner(s)

  • Exclude a list of learners

  • Exclude learners that share criteria (characteristics and/or events)

• Edit learner groups by adding/removing learner(s) and/or refresh manual groups

  • Examples: 

    • Add a list of learners

    • Add learners that share criteria (characteristics and/or events)

    • Exclude a list of learners

    • Exclude learners that share criteria (characteristics and/or events)

• Name a learner group

• Save a learner group

• Disable learner group (not delete at least for MVP)

  • Disabling a learner group:

    • Stops automatic refreshing

    • Stops any interventions associated with the group

• View list of learner groups that have been made for my course

  • List includes Learner Group Name and metadata (criteria used to create learner group, number of learners in group, when it was created and last modified (if applicable))

• View data for learner group

  • For those using Aspects, we can start by allowing users to view learner group data via the Learner Group Dashboard (Learner Group is a filter on all of the Coursewide Aspects Dashboards)

  • For those not using Aspects, user can pull learner groups from MySQL or from tracking events that are created or from Open edX events.

• Import a list of learners to create/edit a learner group

  • What is the data we’re importing here?

    • Username and/or email address

  • Context: 

    • Here are the rows in the enrolled learner data from data downloads tab:

      • ID, username, name, email, language, location, location, gender, level_of_education, mailing_address, goals, enrollment_mode, last_login, date_joined, external_user_key, city, country

    • The information course delivery teams use to bulk enroll learners:

      • Username and/or email address

• Export the list of learners in a group (for those with and without Aspects)

  • What is the data we’re exporting here? (same as data downloads, but shouldn’t we omit mailing address? If not, we can keep it the exact same as data downloads format)

    • ID, username, name, email, language, location, location, gender, level_of_education, mailing_address, goals, enrollment_mode, last_login, date_joined, external_user_key, city, country (same as data downloads list)

Out-of-the-box Leaner Group

One default learner group that exists out-of-the box for courses (manually created by a button push):

• Pre-defined at-risk learner group: Learner enrolled in this course, has not yet passed the course, has done something in the course, has not engaged with the course in N or more days

  • N = 7, but can we make it easy for users to update N?

  • Automatically refreshed group (See Open Questions?)

  • Note: there are ways to see if a learner has engaged with a course in N or more days without using Aspects; however, after speaking with Dave Ormsbee, this may not be the most performant query…

Defining a Learner Group

Course delivery teams can define their own learner groups by specifying learner criteria. User can combine up to two criteria to create a group (including anti-criteria):

• Default criteria for inclusion/exclusion available out-of-the-box (Manual/static groups eligible for refreshed?)

  • Enrollment Status

  • Current Enrollment Track

  • By Enrollment Date (first enrollment date)

  • Learner’s Current Grade

  • Learner’s Last Sign In Date (we want last engagement date, but this might be our best bet outside of analytics)

  • Learner Email Domain Name

• Aspects plugin for learner group creation 

  • Learner engaged with any course content (pages - aside from course homepage) in N or more days

  • Learner has engaged with any course content (pages - aside from course homepage) ever

  • Learner has engaged with…?:: - 

    • Multi-select problem in the course

    • Multi-select ORA problem in the course (?)

    • Multi-select video in the course

    • Multi-select course section

    • Multi-select course subsection

    • Multi-select course unit

  • Learner got… correct (select) (on first attempt, any attempt ← select):

    • Multi-select problem in the course

  • Learner has created a post in discussion forums

  • Learner earned X grade or greater/lower (select) on… 

    • Select assignment

• The criteria for defining your own learner group must be pluggable, so that users can add the criteria for creating learner groups that are important for their use case.

Considering learner groups alongside existing grouping mechanisms

We recognize that there are already a few other ways to group learners on the platform for different purposes, which I’ve outlined below:

Cohorts

• Purpose: To A/B test course content, present different course content (content groups) to different learners, customize grading based on visible content, including custom forums

• How to create:

  • Define group randomly or provide a list of users either hand entering (username or email address) OR by uploading a CSV file with username/email (assign a content group to a cohort to control what content the learners in that cohort receive)

• Rules/Group-specific idiosyncrasies:

  • Learner can only be in one cohort per course

  • Cohorts only exist at the course level

  • Cohorts are static unless manually updated by course staff

  • Defined by a course delivery team member and never by the learner

Teams

• Purpose: Allows course teams to facilitate group discussion and/or group assignments

• How to create: 

  • In Studio, enable teams application

  • Course delivery team can create “team-sets” (groups of teams based around a specific topic or assignment)

    • Instructor-managed team-sets allow the instructor to create and control membership of teams within a team set

      • To create a team-set, instructor can upload a CSV file that dictates which team in a team-set each learner is assigned to

    • For team sets that are not instructor-managed, learners can join a team in a team set

• Rules/Group-specific idiosyncrasies:

  • Learner can only be in one team per team-set, but there can be multiple team-sets per course

  • Team-sets are linked to a project or discussion forum in a course

  • Teams are static unless team membership is modified by the instructor (for instructor-managed teams) or the learner

  • Can be defined by a course delivery team member (for instructor-managed team-sets) or by the learner

  • Teams can have a language and/or country associated with them to help people self-group

Course Groups

• Purpose: To control what course content an enrollment track has access to by assigning a content group (created in Studio) to a course group (defined by enrollment track) 

• How to create:

  • Assign a content group (created in Studio) to a course group (defined by enrollment track) 

  • The only variable by which a course group is currently created is the enrollment track

• Rules/Group-specific idiosyncracies:

  • Each user can belong to one course group at a time

  • Course groups only exist at the course level

  • Teams are likely dynamic - I would guess if a learner updates their enrollment track, their course group is automatically updated

  • Defined by the learner’s enrollment track

Learner Groups

Learner groups have some overlapping properties to the existing grouping mechanisms on the platform; however, learner groups also allow course delivery teams to segment learners by the things that they do on the platform. Learner groups can be either static or dynamic depending on how the group is defined. This mechanism’s primary purpose will be to facilitate targeted intervention and communication with learners on the platform. 

• Purpose: 

  • Primary purpose: To allow course delivery teams to notify or email a subset of learners (in their course, in an org, on an instance)

  • To allow course delivery teams to look at data for a subset of learners (in their course and eventually across an org or instance)

  • Customizable: It must be possible for instances to be able to do other things with the learner groups they create. 

    • For example, a common request I heard during user interviews: I want to automatically reset the number of attempts for learners who did not get problem X correct.

• How to create:

  • Pre-set learner groups exist

  • Custom learner groups can be created by an authorized user (such as a course admin or other course delivery team member) by either:

    • Including/excluding a list of learners uploaded via CSV

    • Identifying inclusion or exclusion criteria for membership in a group:

      • Example: Learners enrolled in my course who have not engaged with the platform in 7 or more days.

      • Example: Learners enrolled in my course with a grade of N or lower. 

      • Example: Learners enrolled in my course who have not yet attempted a problem in X subsection.

• Rules/Group-specific idiosyncrasies:

  • Can be used to segment learners in a course, across an organization, or an instance (for MVP, the scope will be segmenting learners in a course)

  • A learner can belong to multiple learner groups

  • Can be static (if created as a list of specific learner IDs, for example) OR dynamic (a group that identifies learners who haven’t engaged with the course in 7 or more days)

  • Not learner-defined (a course delivery team member defines the learner group)

  • Can be used to view data for a subset of learners

  • Can be used to email/notify a subset of learners

  • Can be used to email/notify a subset of learners and/or course staff

Can we expand the way we group learners (and how we use those groups) without creating madness?

We must be thoughtful about introducing a new grouping mechanism (like Learner Groups) to the platform so as not to create confusion for those managing courses and the platform where it isn’t needed. We’ll want to consider a careful rollout of this feature that potentially even condenses user grouping to one central place. Learner groups that are created could then be used for different purposes. 

For example: Over time, learner groups could potentially serve other functions such as Cohort, Course Group, and Team assignment. 

• Cohorts and Course Groups: A cohort is a learner group made up of a list of learners, and a course group is a learner group made up of learners based on their enrollment track. This would necessitate making it so that Learner Groups can be linked to a content group.

  • We’d want to be thoughtful about this functionality though (since dynamic groups would likely not serve as the best cohort and currently, learners can belong to more than one learner group).

  • However, the benefit of being able to link a learner group to a content group would be that you could potentially provide additional content to a learner who did not perform well on a particular assignment and even re-test that learner later on in the course. (This is an idea that came up a few times when talking to course delivery teams about learner grouping).

• Teams are a little more complicated since they can be learner defined, they are organized around projects and forums, and involve team-sets. Folding teams within the Learner grouping umbrella would require some additional thought.