System Overview

Create a system that allows users to receive notifications from different areas of the system (discussions, grading).

The user should be able to enable/disable all types of notifications. We intend to achieve this by setting notification preferences for users at a course level.

We have added a new notifications django app in openedx which will contain all the code related to notifications.

Notification workflow

In the diagram below the green color represents actions that are going to occur outside the notifications system. The blue color indicates the actions that will occur inside the notifications system

Notification Preferences

We want the users to be able to control the notifications they want to receive. The notification preferences will be created on a per course basis. The notification preferences database architecture is as follows. The notification preferences should be aware of all the notifications that are intended to be sent. So all new notifications must be added to the notification preferences.

Database schema

from django.contrib.auth.models import User
from django.db import models
from jsonfield import JSONField

NOTIFICATION_CONFIG_VERSION = 1

DEFAULT_NOTIFICATION_CHANNEL_CONFIG = {
      "web": False,
      "push": False,
      "email": False,
}

COURSE_NOTIFICATION_CONFIG = {
  "discussion": {
    "new_post": DEFAULT_NOTIFICATION_CHANNEL_CONFIG,
    "new_comment": DEFAULT_NOTIFICATION_CHANNEL_CONFIG,
  },

  "coursework": {
    "new_assignment": DEFAULT_NOTIFICATION_CHANNEL_CONFIG,
    "new_grade": DEFAULT_NOTIFICATION_CHANNEL_CONFIG,
  }
}

# Non Course Notification Types (Assumption: Only email)
DEFAULT_NON_COURSE_NOTIFICATION_CONFIG = {
    "marketing": {
        "recommendation": True,
        "course_updates": True,
    },
}

class NotificationPreferences(models.Model):
    """
    A model to store user notification preferences.
    """
    user = models.ForeignKey(  # example: User1
        User,
        related_name="notification_preferences",
        on_delete=models.CASCADE,
        db_index=True,
        help_text="User whose notification preferences are being stored.",
    )
    course = models.CharField(    # example: "course-v1:edX+DemoX+Demo_Course"
        max_length=255,
        blank=True,
        default=None,
        null=True,
        help_text="Course whose notification preferences are being stored.",
    )
    notification_config = JSONField(default=COURSE_NOTIFICATION_CONFIG)
    preferences_version = models.BooleanField(default=1)

The notification preferences for a course will be created when a user enrolls in a course. For all existing active enrollments we plan to write a management command that will create notification preferences.

Adding a new notification preference

To add a new notification you will need to update the COURSE_NOTIFICATION_CONFIG and increment the NOTIFICATION_CONFIG_VERSION

This will not automatically update all the existing configurations. When that happens is explained in the next step

Updating notification preferences version

For all existing notification preferences if the notification version is updated the change will not be immediately reflected. There are two cases for updating the notification preferences to the latest version.

When the user accesses the preferences page. In this case if the notification preferences version is out of date it will automatically be synchronized with the latest version.
If the user is intended to receive a notification for which the user notification preferences do not exist yet the notification preferences will be updated to the latest version. For example if the user is intended to receive a notification for a New Forum post but this notification type was added later on and the user preferences version does not contain this type of notification. The notification preferences version will be updated with the default values and the notification will be sent accordingly.

Notifications

The database model schema for a notification

class NotificationApplication(models.TextChoices):
    DISCUSSION = 'DISCUSSION'

class NotificationTypes(models.TextChoices):
    NEW_POST = 'NEW_POST',
    NEW_RESPONSE = 'NEW_RESPONSE'

class NotificationTypeContent(models.TextChoices):
    NEW_POST: 'You have received a new response on your post {response_text}'

class Notification(TimeStampedModel):
    app_name = models.CharField(max_length=64, choices=NotificationApplication.choices)
    notification_type = models.CharField(max_length=64, choices=NotificationTypes.choices)
    # e.g "You have received a new response on your post {response_text}"
    # this content will be rendered with the given context and also translated
    content = models.CharField(max_length=1024, choices=NotificationTypeContent.choices)
    # e.g {"response_text": "This is a response"}
    content_context = models.JSONField(default={})
    # This is the url that the user will be redirected to when clicking on the notification [Optional]
    content_url = models.CharField(max_length=1024, null=True, blank=True)
    # the user foreign key will be indexed
    user = models.ForeignKey(User, related_name="notifications", on_delete=models.CASCADE)
    read = models.BooleanField(default=False)
    seen = models.BooleanField(default=False)

app_name (Char Field, char_limit: 64): This is the application/service that generated the notification e.g “DISCUSSION”. Keeping this in a separate field will make it possible to quickly filter out app specific notifications. This is a potential field to be indexed as well but I have kept this as non indexed for now as the user field is already indexed and the most likely query is on a per user basis.

notification_type (Char Field, char_limit: 64): This is the notification type e.g “NEW_POST”. The content of the notification will be assigned based on the notification type. This is a choice field and each new notification type needs to be added as a choice.

content (Char Field, char_limit: 1024): This is the notification content which is calculated according to the notification_type and is also a choice field. This can also contain variables. The content will also be translated. e.g “You have received a new response on your post {response_text}”

content_context (JSON Field): This supplies the context variables for the content field. e.g for the content above the context would be {"response_text": "This is a response"}. This is an optional and can be null if the content does not have any variables

content_url (Char Field, char_limit: 1024): This contains the url of the content the notification is related to. e.g in case the notification is about a new post in the discussions forum this url will contain a link to the newly created post and the user will be redirected to it on clicking the notification. This is an optional field

user (Foreign key to User): The user this notification was created for. This is also indexed.

read (Bool): A bool to record if the notification was read or not.

seen (Bool): A bool to record if the notification was seen or not. Seen is different from reading. The notification is marked as seen if you click the notification bell icon and view the notifications list. You have to click the notification to mark it as read.

Note: The notification content here can also be created in a separate model and we can put a FK to that model here. This new model could contain the notification_type and content. This way we will not have to save the content for each notification but this also has a downside that if the notification content is updated at a later stage it will affect all existing notifications as well. This is the reason we have not opted for this design.

Translations: All the notification content will be translated using transfix in a similar way all strings are translated in the edx-platform.

Indexing: As the number of notifications could potentially scale into the millions, indexing is really important here. The most common use case is to filter the notifications per user and per app name. We have added indexing to both these fields. There is another solution to this problem which we will discuss in the next section.

Notifications Expiry: Since notifications are only useful for the user for a short period of time it makes sense to delete notifications older than a particular date. The industry standard here is 2-3 months e.g Facebook only keeps notifications that are 2 months old. This will also solve the problem of the notifications table becoming too big in size and making queries slow. We are recording the created date for a notification and we can create a script that deletes notifications older than a particular date. The final call on the expiry date will be made by Product.

We intend to create a generic method that will be used by all other apps to create notifications.

Monitoring and tracking events

This is a section which is largely unexplored right now.

Do we need events to track the activity of updates happening when notifications preferences and notifications are interacted with?

we should consider the following use cases where event tracking could be useful:

Identifying which notification types are disabled the most: By tracking events, we can determine which notification types are being disabled the most by users. This information can help us refine our notification strategy and improve our overall user experience.
Measuring how many notifications are being generated: Event tracking can help us understand how many notifications are being generated and sent to users. This information can help us optimize our notification frequency and avoid overwhelming users with too many notifications.
Determining the most engaging notification types: By tracking events, we can identify which notification types are the most engaging to users. This can help us prioritize certain types of notifications over others and improve the overall effectiveness of our notification system.

We can always edit/remove/add event tracking at any stage of the project progress.

Rollout

All of the above implementation is behind a course waffle flag.

Web Notification Architecture