WIP: Distraction Free (focus mode) Planning Doc

Open edX LMS "Distraction Free" UX Approach

Open edX LMS Distraction Free Mode

AU-1530: Creating a "distraction free" toggle to let learners focus on just the current learning materialTO DO

Problem:

There is not enough focus on learning material. We currently have 4 elements above the learning platform, that on a regular sized laptop take up nearly half the viewport.

Open

User research shows that students are excited about the potential of a “distraction free' or “focus mode”.

Approach:

• Run experiment with MIT biology

• A/B test experience with and without distraction free option

• Compare results

• Investigate any qualitative feedback

• Iterate and decide if/how to launch more widely

Aspects:

• Data/metrics

  • any thoughts about the data meeting today?

  • no big blockers arose

  • good to get better definitions

  • Layered impact measurement

    • Do people use it?

    • Does it impact engagement?

    • Does it impact future sales?

• Design

  • review a couple notes from yesterday

• A/B setup

  • In good shape, sounds like we can enable user based groups instead of session

  • Daniel setup a meeting with Alan J. to validate experiment

• Partner coordination

• Development (eng work)

  • Jansen shared exciting early prototype in code

A/B Test Design

• Data Requirement: A/B test selection must be stable for user (not session)

• Open question: do we need to be able to change the % of users over the course of the experiment? If so, do we need the opted-in users to be consistent between percentage changes?

• 50% of users will have access to feature (off by default) and 50% won’t have access

• Total number of users in course = XXX

• Course to A/B test in = YYY

Metrics:

Questions from Alan:

• How many people are in course (sample size)

Hierarchy of Metric Questions

• Are people using it?

• When they use it, are they engaging longer per session?

• Pacing through content?

  • Are students working through course material at a faster rate?

  • Simple version:

    • Simple benchmarks (to inform “pacing” through course)

      • Days to submit assignment

      • Days to complete first Section

• Grades, Complete Rate, Second Purchases

  • Potentially easy to measure but how to attribute

  • Let’s do it! (Alan can help)

Events needed to capture data:

Open

Metrics Definitions for A/B

Original Metrics Brainstorm Table

Additional Data interests:

• Being able to tell which xblocks have focus mode activated on it

• Time on content

• Do people who use distraction free purchase more courses eventually

Considerations:

• Should this eventually be a paid feature?

• Thinking about this as part of a larger project of creating a “customizable learning experience”

• Potential revenue hit from hiding CTA?

  • Ben idea to mitigate:

  • fwiw, I feel like we could embed just the upgrade for *current* course in the "distraciton-free" experience

    • still cuts out a LOT of distractions

    • Audit should always have upgrade option visible

Design Questions

Potential Future Ideas