edx.org has used New Relic for APM (Application Performance Monitoring) for many years, and it is still arguably the most capable product in this category of services. However, there are many competing tools in this space, some of which are much less expensive or even open source. We would like to avoid imposing New Relic related overhead on Open edX users who don’t actually use the service, and would additionally like to make it easy to install at least one of the open source APM services alongside Open edX development and production environments. This document assembles some initial notes and ideas on how to make this happen.
SkyWalking is an Apache project licensed under the Apache 2.0 license, making it feasible from a licensing perspective to distribute alongside Open edX.
Can store data in either MySQL, PostgreSQL, Elasticsearch, OpenSearch, or h2
Compatible with OpenTelemetry, Prometheus, and other metrics/logging systems
Has a browser agent for reporting errors and performance data from browser sessions
Has significantly more contributors and GitHub Stars than the alternatives
Primarily implemented in Java
SigNoz is an “open core” APM service, with the bulk of the code under the MIT license but a handful of “enterprise” features (SSO, audit logs, granular access control, etc.) only available with a commercial license. This could work for development environments, but would have to be compared against other commercial APM solutions for production use.
Stores data in ClickHouse (which is Apache 2.0 licensed, but another data storage engine that would need to be installed)
Uses OpenTelemetry for agents
No browser agent yet?
Primarily implemented in TypeScript
Pinpoint is Apache 2.0 licensed, but has a narrower range of supported server side languages than the alternatives (primarily Java and PHP, with Python added 2 years ago).
Stores data in HBase
No OpenTelemetry support or brower agent?
Primarily implemented in Java
Elastic APM is “open source” to the extent that Elasticsearch (one of its dependencies) is. It can be downloaded and installed for most development and production contexts, but has restrictions to prevent commercial competition for Elastic that render it incompatible with most open source licenses. So I don’t think Open edX would want to recommend it as a default, but it could be a viable option for some users.
Stores data in Elasticsearch
Uses OpenTelemetry for agents
Has a browser agent
Primarily implemented in Go