CHAOSS
- Usman Khalid
CHAOSS is an open source project at the Linux Foundation focused on creating analytics and metrics to help define community health. Work in the CHAOSS Project community is largely organized around software and metrics. Additionally, user groups provide ways to consider how software and practices can support the deployment of CHAOSS metrics.
The project goals are to:
Establish standard implementation-agnostic metrics for measuring community activity, contributions, and health
Produce integrated open source software for analyzing software community development
Develop programs for the deployment of metrics not attainable through online trace data
Build reproducible project health reports
Metrics
CHAOSS metrics are identified and defined using a continuous contribution process. The metrics are officially released biannually following a 30 day comment period. The releases and their history are available at https://chaoss.community/release-history/ .
The metrics are defined by 5 working groups for key areas of interest:
Diversity and Inclusion
Evolution
Common Metrics
Value
Risk
This article covers the evolution of the CHAOSS project and some of the lessons learned on the way about how and which kinds of metrics to track (including thinking about goals, categorization of metrics and standardizing definitions).
CHAOSS project bringing order to open-source metrics
Software
3 software projects are officially part of the CHAOSS project.
GrimoireLab
This is the most comprehensive of the 3 software projects. It can “gather data from several platforms involved in software development, merge and organize it in a database, and produce visualizations, actionable dashboards, and analytics of all of it”.
This article covers the evolution of GrimoireLab from its beginning and includes details about some key architectural and product choices that were made over the years in response to various challenges.
Tough lessons learned from measuring community health with open source software
Some key features:
It supports a wide variety of data sources including repos, issue trackers (including Github Issues, JIRA, Gitlab issues), PR reviews, forums (including Discourse), chat services (including Slack).
The data is stored in an Elasticsearch database and the interface is powered by Kibana, which allows very easily creating custom reports and dashboard.
These reports and dashboards can be embedded in other places too. Because the data is stored in Elasticsearch it is also possible to create fully custom interfaces. An example is The Linux Foundation Insights.
It has a powerful tool for managing identities: “Sorting Hat maintains an SQL database of unique identities of community members across (potentially) many different sources. Identities corresponding to the same real person can be merged in the same unique identity with a unique uuid. For each unique identity, a profile can be defined, with the name and other data shown for the corresponding person by default. In addition, each unique identity can be related to one or more affiliations, for different time periods. This will usually correspond to different organizations in which the person was employed during those time periods.”
Public examples:
The Open edX community metrics can be viewed at https://openedx-metrics.herokuapp.com .
Augur
Augur is another “software suite for collecting and measuring structured data about free and open source software (FOSS) communities.
We gather trace data for a group of repositories, normalize it into our data model, and provide a variety of metrics about said data. The structure of our data model enables us to synthesize data across various platforms to provide meaningful context for meaningful questions about the way these communities evolve.”
It can only collect data from repositories.
Cregit is “a framework of tools that facilitates the analysis and visualization of the evolution of source code stored in git repositories.”
It also has limited features and does not seem to be under active development.