2023-11-01 Meeting notes

 Date

Nov 1, 2023

Recording

Open edX Data Working Group (early) (2023-11-01 08:09 GMT-4)

 Participants

  • @Brian Mesick

  • @Chelsea Rathbun

  • @Edward Zarecor

  • @Emad Rad

  • @Vincent Laberge

  • Yagnesh Nayi

 Discussion topics

Time

Item

Presenter

Notes

Time

Item

Presenter

Notes

5 min

Funded Contribution Updates

@Brian Mesick

  • Aspects data pipelines & reports wrapping up 11/17

  • New proposals are being iterated on

5 min

Welcome Chelsea!

@Chelsea Rathbun

Introductions and update on the Aspects product-side work

Anyone interested in Aspects beta user testing, please reply to @Chelsea Rathbun on the #wg-data Slack thread.

20 min

Aspects updates

@Brian Mesick

  • Beta server is updated with changes based on feedback

  • @Cristhian Garcia is working on tracking log message delivery over the event bus

  • Translation pipeline is up and mostly working

  • Backend work continues while Chelsea gathers feedback on charts and dashboards

?

Event bus discussion

@Brian Mesick

The event bus use cases are growing quickly, I’d like to take a temperature check of the group and start putting out feelers around how comfortable people are with it and whether anyone would want to use it for Aspects event delivery. This may be an ongoing discussion point through several future meetings.

Why Aspects needs the event bus for performance & scaling:

  • Current approach (event-routing-backends + Ralph) creates many celery tasks per tracking log event.

  • Clickhouse prefers getting data in chunks rather than row-by-row.

 Action items

 Decisions