Test certificate upsell on course sidebar instead of primary area

Description

Test shifting the hero upsell message to function as a secondary CTA instead of as a primary one on the course home page, making room for course messaging and goal setting as primary areas of focus for the course landing experience.

Acceptance Criteria

  1. Determine requirements for significance.

  2. Implement test in Optimizely.

  3. Ensure the test has a custom GA dimension configured for tracking.

  4. Update partner portal post with description.

  5. Retain existing enhanced ecommerce ("promotion analytics") behaviors

Variations
1) Current Hero
2) Sidebar Upsell Hero A
3.) Sidebar Upsell Hero B

Audience

  1. All EN traffic, divided evenly between variants (33% each)

  2. Exclude Mobile traffic

Metrics

  1. Track Revenue (Revenue)

  2. Track Click Through for Upgrade

  3. Track Progression through to content

Expected Impact

  • bounce rate improves by at least 5%

  • enrollment rate has no statistically significant change

Notes:

  • check for new vs returning visitors

Implementation and Technical Details

  • power analysis

    • how much required to reach significance => roughly how long to get an answer

    • people per variant (look at RET experiments)

    • for us - should be ok to skip because we should have this from a RET experiment

  • develop the experiment code

    • Time: .5 day or less optimizely + time for js/css implementatino.

    • optimizely (can develop in prod and just not publish the experiment)

      • optimizely-experiments (takes longer first time)

    • See RET DOM manipulation example to avoid pop-in effect.

    • script id=user-metadata has info about the user and course to fill in the banner.

  • metrics

    • implement optimizely event for upgrade

    • In optimizely, click button to send to GA, and need to create a session scoped custom dimension in GA for this. GA has the number. Optimizely has the name. Configure the number in Optimizely. Per experiment, everything is tagged.

Steps to Reproduce

None

Current Behavior

None

Expected Behavior

None

Reason for Variance

None

Release Notes

None

User Impact Summary

None

Status

Assignee

Marco Morales

Reporter

Andy Armstrong

Labels

None

Reach

None

Impact

None

Customer

None

Partner Manager

None

URL

None

Contributor Name

None

Groups with Read-Only Access

None

Actual Points

None

Category of Work

None

Stakeholders

None

Story Points

3

Components

Priority

Unset
Configure