/
September 6, 2016

September 6, 2016

Sept 27, 2016

Meeting with edX lead architects: @Dave Ormsbee (Deactivated) and @Calen Pennington (Deactivated)

  • Clear requirements on messaging layer

    • async versus sync

    • in-process versus inter-process

    • routing requirements

      • e.g., named queues with or without filtering by wildcards

  • Scalable to both edX-size (broadcast to all edX users) and large course enrollments

  • Course-size explosion

    • quarter-million students in the course

    • possibly support "echo service" versus requiring a synchronous API request response (even though paginated)

    • enqueue the request to the user-target resolver

  • Presentation step

    • To scale, minimize the number of responsibilities of the template renderer

      • considering details of localization and personalization

    • Would like to take advantage of channels that support bulk-delivery, such as email and push

  • User Target Resolver

    • consider having multiple implementations instead of just multiple instances of the same generic implementation

    • each implementation would be scoped to the group that it's responsible for (e.g., thread followers, course enrollments)

    • each implementation would use the notification library component with the UserTargerResolver abstract class that automatically provides support for listening to the queue and enqueuing to the next step in the pipeline.

    • can be possible by having a named-queue per UserTargetResolver type.

  • Devops, in the past, has mentioned preference for AWS SQS/SNS instead of Rabbit.

  • TODO: how does this system handle priorities?  different queues for different priorities?