Analytics and Diagnostics

This page supplements the wealth of knowledge in /wiki/spaces/AN/overviewand Mobile Analytics.

Analytics Events in edx-platform

Designing new events

When designing new events, see the /wiki/spaces/AN/pages/13205895.  

Determine if your new event should make it into the publicly documented data package, or if it will only be used for internal purposes.  For internal-only events, you use the special prefix 'edx.bi.' for your event name.

Choose between server-side and client-side implementation of the event, and consider whether or not Mobile Analytics events need to be implemented separately for a native client-side feature.

Implementing in edx-platform

Implementing events in edx-platform is documented in Read the Docs and /wiki/spaces/AN/pages/30967007.  This section provides some additional clarity and examples.

Server-side events (includes celery)

• Rule of thumb: Use server-side events where possible.  More context is available to server-side events.  Server-side events are less likely to have false duplicate events.
• Call tracker.emit
  • Emits to Tracking Log and downstream tools (e.g. Vertica/Tableau, edX Research Package, etc.)
  • Event must be whitelisted to emit to Segment.
    • In turn, Segment emits to other tools: Google Analytics (GA), Google Tag Manager, Optimizely, Qualaroo, Crazy Egg, Marketing tools.
    • Here are sample commits for updating the whitelist for edge.edx.org and the whitelists for edx.org.
    • At this time, it doesn't look like our code supports sending special labels to GA from server-side events.  See /wiki/spaces/AN/pages/30967007 for possibly enhancing this.

Client-side events

• Rule of thumb: Where you have an option of client-side or server-side event, choose server-side (see above).
• Logger.log
  • Important: This should be renamed to Tracker.emit, but that requires a deprecation strategy including telling course teams.
  • Emits to Tracking Log and downstream tools (e.g. Vertica/Tableau, edX Research Package, etc.)
  • Event must be whitelisted to emit to Segment (see above).
• analytics.track
  • Use this if you want Segment and its downstream tools only.  If you want the Tracking Log or both, use the other call (with or without the whitelist).
  • Emits to Segment and its downstream tools.

Ecommerce Events

For any ecommerce related activity, including any messages encouraging a purchase, you can use Segment's Ecommerce Events.  

Any message encouraging a user to purchase could be considered a Segment Ecommerce "Promotion" Event. These, in turn, can be tracked in Google Analytices: Conversions > Ecommerce > Marketing > Internal Promotions.  Here is an example of implementing a "promotion" event in edx-platform for a verified upgrade reminder.

Analytics Tool Usage by Team

Tools

Analytics Tools:

• Google Analytics (GA)
• Google Tag Manager
• Firebase Analytics
• Fabric (Mobile)
• Segment (a.k.a. Segment.io)
• Tracking Log
  • Splunk4Tracking
  • Vertica (Data Warehouse)
  • Tableau (Visualization using Data Warehouse)
  • edX Insights
  • edX Research Packages
• Optimizely (A/B Testing)
• Qualaroo (survey tool)
• Crazy Egg (screen recording/heat mapping, may or may not be used)
• Marketing
  • Sailthru (email marketing)
  • Facebook ad reporting

Diagnostic Tools (includes Performance):

• CourseGraph (query course content usage)
• Splunk (logs)
• NewRelic (errors, usage, health)
• Datadog (deprecated)

Consider the following data storage questions:

• Limits (e.g. Splunk Logs)
• Privacy (who has access?)
• How long does data live?