/
Spec: Replace email cron jobs with celery tasks

Spec: Replace email cron jobs with celery tasks

As of now, in order to trigger daily and weekly notification emails (digests), instance operators need to configure 2 separate cron jobs. See the operator docs for more details: https://docs.openedx.org/en/latest/site_ops/how-tos/enable_notifications.html#configure-daily-and-weekly-email-digests.

Problems

  1. Silent failure: Digest emails rely on a scheduler outside the app (cron). If it’s missing, misconfigured, or changes over time, the platform can still show email preferences even though no digests will be sent.

  2. Retry is clumsy: With cron, retrying a failure means rerunning a big batch job, and we need extra tracking to avoid duplicate emails. With per-user tasks, retries are easier and more targeted.

    1. Note: Retry for email digests does NOT exist in current implementation.

  3. Inaccurate time info: Because the timing is controlled outside the app, the preference center can’t reliably show when a user will actually receive a digest. Any fixed time shown in the UI can easily be wrong (as is the case at present).

  4. Bad fit for global users: Cron runs on server time. That doesn’t match users across timezones, and it makes it hard to later support “send in my timezone” or “send at my preferred time” without hacks.

Proposed solution

Replace cron-triggered digests with celery tasks. When a new notification is generated for a user, we will create or update that user’s next digest task based on their cadence (daily/weekly). For performance reasons, we need to keep, at most, one pending digest task per user per cadence

Pros

  • No manual configuration: Operators won’t have to configure cron jobs manually.

  • Reliability: Digest scheduling is owned by the application so cron jobs won’t fail silently.

  • Optimized delivery: In future, we can improve these tasks to respect user’s timezone or even their specified times/days.

  • Info correctness: Preference center can show the actual next scheduled send time (and later linked with user’s timezone).

  • Retries: In future, we can add retries for failed emails comparatively more easily.

Cons

  • A lot of celery workers will be engaged if a large course creates a course wide notification like course update and if a lot of users have set email cadence to daily/weekly.

Rollout consideration

We need to make sure that instances that are currently using cron jobs don’t start sending duplicate emails once they upgrade and this feature enabled.