Analytics and Diagnostics

This page supplements the wealth of knowledge in /wiki/spaces/AN/overviewand Mobile Analytics.

Analytics Events in edx-platform

Designing new events

When designing new events, see the /wiki/spaces/AN/pages/13205895.  

NAMING CONVENTION

Determine if your new event is a research event for partners that goes into the publicly documented data package, or whether it will only be used for internal BI purposes.  

• For internal-only events, use the special prefix 'edx.bi.' for your event name.
• For research events, make sure you tag someone on the Docs team, to make sure your new event is documented in the edX Research Guide.

Choose between server-side and client-side implementation of the event, and consider whether or not Mobile Analytics events need to be implemented separately for a native client-side feature.

Implementing in edx-platform

Implementing events in edx-platform is documented in the edX Developer's Guide and /wiki/spaces/AN/pages/30967007.  This section provides some additional clarity and examples.

Server-side events (includes celery)

• Rule of thumb: Use server-side events where possible.  More context is available to server-side events.  Server-side events are less likely to have false duplicate events.
• Call tracker.emit
  • Emits to Tracking Log and downstream tools (e.g. Vertica/Tableau, edX Research Package, etc.)
  • Event must be whitelisted to also be emitted to Segment.
    • In turn, Segment emits to other tools: Google Analytics (GA), Google Tag Manager, Optimizely, Qualaroo, Crazy Egg, Marketing tools.
    • Here are sample commits for updating the whitelist for edge.edx.org and the whitelists for edx.org.
    • At this time, it doesn't look like our code supports sending special labels to GA from server-side events.  See /wiki/spaces/AN/pages/30967007 for possibly enhancing this.
• To send events directly to Segment:
  • This is appropriate when server-side event does not need to appear in tracking logs.
  • Call segment.track(), after "from common.djangoapps.track import segment".
  • If appropriate, also call segment.identify().  Currently the LMS only does this at login/registration.   

Client-side events

• Rule of thumb: Where you have an option of client-side or server-side event, choose server-side (see above).
• Tracking Log (and possibly Segment)
  • MFEs: sendTrackingLogEvent
  • Legacy code: Logger.log
  • Important: This should be renamed to Tracker.emit, but that requires a deprecation strategy including telling course teams.
  • Emits to Tracking Log and downstream tools (e.g. Vertica/Tableau, edX Research Package, etc.)
  • Event must be whitelisted to emit to Segment (see above).
• Segment (and downstream tools)
  • MFEs: sendTrackEvent
  • Legacy code: analytics.track
  • Use this if you want Segment and its downstream tools only.  If you want the Tracking Log or both, use the other call (with or without the whitelist).
  • Emits to Segment and its downstream tools.

Ecommerce Events

For any ecommerce related activity, including any messages encouraging a purchase, you can use Segment's Ecommerce Events.  

Any message encouraging a user to purchase could be considered a Segment Ecommerce "Promotion" Event. These, in turn, can be tracked in Google Analytices: Conversions > Ecommerce > Marketing > Internal Promotions.  Here is an example of implementing a "promotion" event in edx-platform for a verified upgrade reminder.

Analytics Tool Usage by Team

Tools

Analytics Tools:

• Google Analytics (GA)
• Google Tag Manager
• Firebase Analytics
• Fabric (Mobile)
• Segment (a.k.a. Segment.io)
• Tracking Log
  • Splunk4Tracking
  • Vertica (Data Warehouse)
  • Tableau (Visualization using Data Warehouse)
  • /wiki/spaces/PT/pages/159531726
  • edX Research Packages
• Optimizely (A/B Testing)
• Qualaroo (survey tool)
• Crazy Egg (screen recording/heat mapping, may or may not be used)
• Marketing
  • Sailthru (email marketing)
  • Facebook ad reporting

Diagnostic Tools (includes Performance):

• /wiki/spaces/PT/pages/96698687 (query course content usage)
• Splunk (logs)
• NewRelic (errors, usage, health)
• Datadog (deprecated)

Consider the following data storage questions:

• Limits (e.g. Splunk Logs)
• Privacy (who has access?)
• How long does data live?