Application Performance Monitoring

Owned by Jeremy Bowman (Deactivated)
Last updated: Oct 13, 2022

edx.org has used New Relic for APM (Application Performance Monitoring) for many years, and it is still arguably the most capable product in this category of services. However, there are many competing tools in this space, some of which are much less expensive or even open source. We would like to avoid imposing New Relic related overhead on Open edX users who don’t actually use the service, and would additionally like to make it easy to install at least one of the open source APM services alongside Open edX development and production environments. This document assembles some initial notes and ideas on how to make this happen.

Open Source APM Options

SkyWalking

SkyWalking is an Apache project licensed under the Apache 2.0 license, making it feasible from a licensing perspective to distribute alongside Open edX.

  • Can store data in either MySQL, PostgreSQL, Elasticsearch, OpenSearch, or h2

  • Compatible with OpenTelemetry, Prometheus, and other metrics/logging systems

  • Has a browser agent for reporting errors and performance data from browser sessions

  • Has significantly more contributors and GitHub Stars than the alternatives

  • Primarily implemented in Java

SigNoz

SigNoz is an “open core” APM service, with the bulk of the code under the MIT license but a handful of “enterprise” features (SSO, audit logs, granular access control, etc.) only available with a commercial license. This could work for development environments, but would have to be compared against other commercial APM solutions for production use.

  • Stores data in ClickHouse (which is Apache 2.0 licensed, but another data storage engine that would need to be installed)

  • Uses OpenTelemetry for agents

  • No browser agent yet?

  • Primarily implemented in TypeScript

Pinpoint

Pinpoint is Apache 2.0 licensed, but has a narrower range of supported server side languages than the alternatives (primarily Java and PHP, with Python added 2 years ago).

  • Stores data in HBase

  • No OpenTelemetry support or brower agent?

  • Primarily implemented in Java

Elastic APM

Elastic APM is “open source” to the extent that Elasticsearch (one of its dependencies) is. It can be downloaded and installed for most development and production contexts, but has restrictions to prevent commercial competition for Elastic that render it incompatible with most open source licenses. So I don’t think Open edX would want to recommend it as a default, but it could be a viable option for some users.

  • Stores data in Elasticsearch

  • Uses OpenTelemetry for agents

  • Has a browser agent

  • Primarily implemented in Go