Platform-Level Analytics and Reporting Comparator Research

Summary

These three platforms show three very different forms of platform analytics and reporting.

Moodle’s provision is actually extremely weak, and it relies heavily on third-party plugins such as Intelliboard to address the different needs of their broad market audience. The native reports are clunky and require significant amounts of data handling to turn them into actionable insights, in a very similar way to Open edX. Their report builder is interesting, and Moodle’s custom reporting interface reminds me a lot of a lower-power version of Metabase. It’s basically a frontend for constructing basic SQL queries without writing SQL, which isn’t a terrible model, just it’s very limited in what it can do.

Canvas is going through a period of transition. Their old reporting and analytics are not good enough, so they’re essentially doing a similar thing to what we’re doing, just they’re further down the path. Their “interactions” are under-implemented, with many features currently not reporting interactions to their reporting interface. Their tools and reports are still very tightly controlled and opinionated, aimed expressly at synchronous classroom training (as they should be), so we should be extremely careful in copying their reporting, as it’s only suitable for a specific use-case.

Skilljar has a lot of flaws, but their analytics are the best in the marketplace, and that’s where they won a lot of sales deals. They track everything learners do, report on it clearly, and their strategic insights dashboard offers some very powerful benchmarks for training teams to measure themselves against. It’s not perfect, I think that the benchmark data could absolutely provide more than it does, but the concept is extremely compelling. I wonder if we could in some way enable that kind of anonymous cross-platform data sharing as an optional feature for Open edX sites and orgs. Opt in to share your organisation’s aggregate data, get access to the benchmark data?