• In progress

    • Web Notification Architecture

    Owned by Asad Azam
    Last updated: Jun 06, 2023
    7 min read

    Web Notifications Architecture

    1. System Overview

    2. Notification Preferences

    3. Notifications

    4. Monitoring and Tracking events

    5. Rollout

     

    System Overview

    Create a system that allows users to receive notifications from different areas of the system (discussions, grading).

    The user should be able to enable/disable all types of notifications. We intend to achieve this by setting notification preferences for users at a course level. 

    We have added a new notifications django app in openedx which will contain all the code related to notifications. 

     

    Introduction:
    The Notifications App plays a critical role in our Learning Platform by keeping users informed about relevant activities in real-time. The App operates based on several key concepts including "Notification Type," "Notification Content," and "Notification Preference". Each Notification Type corresponds to a unique kind of notification, such as "New response on your post", and has a unique ID. The Notification Content defines how a specific type of notification is structured, while Notification Preference holds user-specific preferences for each type of notification.

    Workflow:
    Our platform apps, like "Discussions", interact with the Notifications App by emitting signals (using open edX events. These signals carry crucial information for generating notifications:

    1. The intended Notification Type.

    2. The list of users who should receive the notification.

    3. The context for the notification (e.g., "username", "post_title", "course_id").

    The Notifications App is listening for these signals from the platform. Upon receiving a signal, the app follows a sequence of steps to process it and generate the relevant notifications:

    1. It identifies the Notification Type associated with the received signal.

    2. It fetches the preferences for that Notification Type for all users specified in the signal.

    3. It shortlists users who have opted in to receive this type of notification, according to their preferences.

    4. It uses the corresponding Notification Content and the context provided in the signal to create the notification message.

    5. It then dispatches the generated notification to all the shortlisted users.

    As for the flow chart, I can provide a textual description of what it might look like:

    1. Start -> Discussions App emits a signal.

    2. Discussions App signal contains (Notification Type, Users, Context) -> Received by Notifications App.

    3. Notifications App identifies Notification Type.

    4. Fetches Notification Preferences for all listed users.

    5. Shortlists users who have opted in for this Notification Type.

    6. Creates the notification message using the Notification Content and provided Context.

    7. Dispatches the notification to the shortlisted users.

    8. End

     

    Diagram

     

    Notification Preferences

    We want the users to be able to control the notifications they want to receive. The notification preferences will be created on a per course basis. The notification preferences database architecture is as follows. The notification preferences should be aware of all the notifications that are intended to be sent. So all new notifications must be added to the notification preferences.

     

    Database schema

    from django.contrib.auth.models import User from django.db import models from jsonfield import JSONField NOTIFICATION_CONFIG_VERSION = 1 NOTIFICATION_CHANNEL_CONFIG = {       "web": False,       "push": False,       "email": False, } COURSE_NOTIFICATION_CONFIG = {   "discussion": {     "new_post": NOTIFICATION_CHANNEL_CONFIG   }, } class CourseNotificationPreferences(models.Model):     """     A model to store user notification preferences.     """     user = models.ForeignKey(  # example: User1         User,         related_name="notification_preferences",         on_delete=models.CASCADE,         db_index=True,         help_text="User whose notification preferences are being stored.",     )     course = models.CharField(    # example: "course-v1:edX+DemoX+Demo_Course"         max_length=255,         blank=True,         default=None,         null=False,         help_text="Course whose notification preferences are being stored.",     )     notification_config = JSONField(default=COURSE_NOTIFICATION_CONFIG) preferences_version = models.BooleanField(default=1)

     

    The model has a unique together constraint on user and course. The notification preferences for a course will be created when a user enrolls in a course. For all existing active enrollments we plan to write a management command that will create notification preferences. 

     

    Adding a new notification preference

    To add a new notification you will need to update the COURSE_NOTIFICATION_CONFIG and increment the NOTIFICATION_CONFIG_VERSION

    This will not automatically update all the existing configurations. When that happens is explained in the next step

     

    Updating notification preferences version

    For all existing notification preferences if the notification version is updated the change will not be immediately reflected. There are two cases for updating the notification preferences to the latest version.

    1. When the user accesses the preferences page. In this case if the notification preferences version is out of date it will automatically be synchronised with the latest version.

    2. If the user is intended to receive a notification for which the  user notification preferences do not exist yet the notification preferences will be updated to the latest version. For example if the user is intended to receive a notification for a New Forum post but this notification type was added later on and the user preferences version does not contain this type of notification. The notification preferences version will be updated with the default values and the notification will be sent accordingly. 

    3. A user un enrols from a course the notification preferences for that course will be marked as in-active.

     

     

    Notifications

     

    The database model schema for a notification (In progress)

    class Notification(TimeStampedModel):     app_name = models.CharField(max_length=64)     notification_type = models.CharField(max_length=64)     # e.g {"response_text": "This is a response"}     content_context = models.JSONField(default={})     # This is the url that the user will be redirected to when clicking on the notification [Optional]     content_url = models.CharField(max_length=1024, null=True, blank=True)     # the user foreign key will be indexed     user = models.ForeignKey(User, related_name="notifications", on_delete=models.CASCADE)     read_at = models.DateTimeField(null=True, blank=True) seen_at = models.DateTimeField(null=True, blank=True)

     

    app_name (Char Field, char_limit: 64): This is the application/service that generated the notification e.g “DISCUSSION”. Keeping this in a separate field will make it possible to quickly filter out app specific notifications. This is a potential field to be indexed as well but I have kept this as non indexed for now as the user field is already indexed and the most likely query is on a per user basis.

    notification_type (Char Field, char_limit: 64): This is the notification type e.g “NEW_POST”. The content of the notification will be assigned based on the notification type. This is a choice field and each new notification type needs to be added as a choice.

    content (Not a model field): This is the notification content which is calculated according to the notification_type This can also contain variables which will be filled using the provided context. The content will also be translated. e.g “You have received a new response on your post {response_text}”

    content_context (JSON Field): This supplies the context variables for the content field. e.g for the content above the context would be {"response_text": "This is a response"}. This is an optional and can be null if the content does not have any variables

    content_url (Char Field, char_limit: 1024): This contains the url of the content the notification is related to. e.g in case the notification is about a new post in the discussions forum this url will contain a link to the newly created post and the user will be redirected to it on clicking the notification. This is an optional field

    user (Foreign key to User): The user this notification was created for. This is also indexed.

    read_at (DateTime): A DateTime to record if the notification was read or not.

    seen_at (DateTime): A DateTime to record if the notification was seen or not. Seen is different from reading. The notification is marked as seen if you click the notification bell icon and view the notifications list. You have to click the notification to mark it as read.

    Note: The notification content here can also be created in a separate model and we can put a FK to that model here. This new model could contain the notification_type and content. This way we will not have to save the content for each notification but this also has a downside that if the notification content is updated at a later stage it will affect all existing notifications as well. This is the reason we have not opted for this design.

    Translations: All the notification content will be translated using transfix in a similar way all strings are translated in the edx-platform.

    Indexing: As the number of notifications could potentially scale into the millions, indexing is really important here. The most common use case is to filter the notifications per user and per app name. We have added indexing to both these fields. There is another solution to this problem which we will discuss in the next section.

    Notifications Expiry: Since notifications are only useful for the user for a short period of time it makes sense to delete notifications older than a particular date. The industry standard here is 2-3 months e.g Facebook only keeps notifications that are 2 months old. This will also solve the problem of the notifications table becoming too big in size and making queries slow. We are recording the created date for a notification and we can create a script that deletes notifications older than a particular date. The final call on the expiry date will be made by Product.

     

     

     

    Monitoring and tracking events

    This is a section which is largely unexplored right now. 

    Do we need events to track the activity of updates happening when notifications preferences and notifications are interacted with?

    we should consider the following use cases where event tracking could be useful:

    1. Identifying which notification types are disabled the most: By tracking events, we can determine which notification types are being disabled the most by users. This information can help us refine our notification strategy and improve our overall user experience.

    2. Measuring how many notifications are being generated: Event tracking can help us understand how many notifications are being generated and sent to users. This information can help us optimize our notification frequency and avoid overwhelming users with too many notifications.

    3. Determining the most engaging notification types: By tracking events, we can identify which notification types are the most engaging to users. This can help us prioritize certain types of notifications over others and improve the overall effectiveness of our notification system.

    We can always edit/remove/add event tracking at any stage of the project progress.

     

    Rollout

    All of the above implementation is behind a course waffle flag.