September 7, 2016

Owned by Nimisha Asthagiri (Deactivated)
Last updated: Sept 27, 2016

Meeting with devOps: Kevin Falcone (Deactivated) and MaxR (Deactivated)

  • Notifications Pipeline Configuration
    • Having a centralized notifications service for configuration settings adds much more complexity than needed.
      • version management, caching settings in each IDA, etc..
    • Conclusion: keep it simple.
      • If needed in the future, we can have the configurable setting be on the message object itself.
        • This also allows different pipeline steps for different message types.
      • For now, the pipeline steps (order and named queues) can be configured in each IDA.
        • As we don't expect this to change for a single open edX deployment.
  • Number of Recipients per message
    • Is there a use-case for having multiple recipient targets per message?
      • De-duplication of messages - in case a user is in more than on target type
        • However, this can be achieved using exclusion selectors / list instead.
    • Is there a need to support bulk sending?
      • Push notifications have some support of "channels" for bulk sending
        • However, these channels have to be pre-configured.
        • Issues when users have multiple devices and channels are not configured on all devices.
        • Doesn't support localization well.
      • Experience from the Bulk Email feature
        • Upon failures, messages are sent multiple times if failure happens at a later offset.
    • Conclusion: single "virtual" user at every step in the pipeline.
      • simplifies the interface
      • messages are smaller and easier to handle
      • After a certain step (e.g., User-Targeting), just create a new message per user.
  • Messaging technology
    • Starting with Rabbit
    • Amazon SQS
      • may be able to support this if we implement within constraints

  • Time-to-live

    • Use timestamp in UTC as expiration time

    • Verified by each step in the pipeline

    • Message dropped (though logged) once time exceeded
  • UUID
    • When messages branch off, keep track of parent UUID
  • Queues per durability/priority, also per version?
    • Will need to dig into this further.
    • Complexity possibly localized to the messaging layer.
  • Versioning of the libraries
    • - e.g., NotificationPipelineStep.Presentation.v2
    • treat queues as data models, similar to django migrations?
    • Conclusion: To avoid coordinated rollout of IDAs, important for the messaging format to be thought of as a public API