Long-Term Requirements for the Unified Model

This section lists system-level requirements that describe how the new user group system should behave. These requirements can help guide design decisions around performance, scalability, integration, and flexibility.

⚠️These requirements are not fixed and may change as the project progresses. As we learn more, we might add or update new requirements to better suit the system's needs.

Many of these requirements came from the discussions in this document: Architectural Design Questions . Please take a look for more context!

Functional Requirements

1. Group Model Design

Explains how user groups are structured. Supports all legacy group types, keeps change history, and clearly separates criteria-based and manual groups.

• Any group must support all the capabilities from legacy groups, such as cohorts (used for content segmentation), teams (used for collaboration), and course or track groups (used for visibility and access control). This list of capabilities is non-exhaustive.

• Each user group must store metadata to support traceability and clear UI representation:

  • Definition changes:

    • Maintain audit logs for modifications to the group’s definition (e.g., criteria updates, behavior changes).

    • Record the timestamp of each definition change.

  • For UI display:

    • Display the group name.

    • Show the configured grouping criteria (if any).

    • Show the current number of users in the group.

    • Display the date of the last refresh.

    • Whether is enabled

    • Stale warning (if possible): For some groups, something could be stale if its a day old (learner hasn’t engaged in course in N or more days OR learner hasn’t engaged with X assignment), whereas if its something like course enrollment track and preferred language… one week might be considered stale.

    • Whether a group is dynamic (criterion-based) and will be updated at a frequency or manually (user-selected).

• Actions taken using a group (like sending messages) should be auditable, but scoped to the type of action and tracked separately.

• Group criteria must be editable after creation. For example, changing the inactivity window from 7 to 14 days should not require a new group.

• Any change to group criteria must trigger re-evaluation of membership.

• Manual groups must be created explicitly. The model should not assume manual mode by default. Instead, the user must choose it.

2. Criteria Management

Covers how rules for grouping users are defined, what values they accept, and how the system handles errors or unavailable data.

• Each such criterion must define:

  • What data it uses (e.g., activity events)

  • What filters it applies (e.g., "active within 7 days")

  • What API or queries it depends on

  • Recommended refresh strategy

• Each type of criterion must define what values and operators are allowed:

  • For example, "Last login" supports < or >= and accepts dates or days.

  • "Country" supports exact matches or checking from a list.

  • Review more examples here: MVP Requirements: Static User Group Creation, Management, and Usage | User Group Creation

• A group can combine up to three criteria, but the system should be flexible enough to support more down the line. For example, a group could be defined by users who are in a certain enrollment track, have watched a video, and have submitted a problem.

• If a criterion becomes invalid (e.g., due to a plugin issue), the group stays active using its last known members.

  • A clear error message should be shown in the UI.

  • The reason for the error must be visible (e.g., "criterion missing").

  • Refresh attempts must be paused until fixed.

• Only course staff or admins can disable a group. This must never happen automatically.

• A group that is no longer valid (e.g., past a due date condition) may be automatically disabled without requiring admin intervention.

• The model must allow criteria that work beyond the course level, at the org or platform level. These can reference data across multiple courses, organizations, or instance levels.

• The scope of each criterion (course, org, platform) must be clear in the UI.

• Dynamic criterion can use data from Aspects if installed (learner behavior analytics). See more examples here MVP Requirements: Static User Group Creation, Management, and Usage | User Group Creation

• If required data is unavailable (e.g., not tracked or not exposed via API), the criterion should not be selectable or must fail gracefully.

• Multiple criteria assigned to a group are joined using a logical &.

3. User Assignment to Groups

Describes how users are assigned to groups based on criteria and how group definitions behave over time.

• Each group may be assigned up to three (non-mutually exclusive) criteria to determine membership.

• A user can belong to more than one group unless explicitly restricted by the group's membership criteria. (TBD)

• If a group requires exclusivity (e.g., "one group per user" policy), assigning a user to a new group must automatically remove them from any conflicting group to maintain consistency. The UI should show a warning letting the user know this will happen.

• Criteria-based groups should be reusable across course reruns.

• Manual groups (e.g., uploaded via CSV) should not be reused by default.

• When a new CSV is uploaded, all existing users in the group must be removed. Only users listed in the CSV who are currently enrolled should be added to the group.

• This behavior should align with how cohorts are treated during course reruns: On course re-run, user groups created manually via CSV upload must not persist. Only user groups based on grouping criteria should be retained across course runs.

4. Refresh Strategies & Staleness

Covers how group data is updated, how outdated information is detected, and how refresh operations are tracked.

• The system must support multiple refresh strategies for updating group membership:

  • Manual: triggered by the user clicking a "refresh" button with a cache to avoid consecutive unnecessary refreshes.

  • Scheduled: automatic updates at a defined frequency (e.g., daily).

  • Event-based: triggered by specific user actions or platform events.

• Data that becomes outdated should be marked as stale (future, if possible).

  • Time-sensitive data (e.g., recent login activity) may go stale quickly.

  • Static data (e.g., preferred language or enrollment track) updates less frequently.

  • The threshold for what counts as stale must be configurable per criterion type or group.

• All refresh actions must be auditable (via logs), including:

  • When and how the group was updated.

  • When users were added or removed, and the reason why.

  • Automated refreshes must log the same data as manual updates.

• The UI must show when group data is out of date (e.g., "refreshed 2 days ago") to help operators track freshness.

• The manual refreshment would only be available for staff / superuser users.

5. Backend and Data Source Integration

Explains how the system gets data from internal and external sources and what happens when those sources fail.

• When groups depend on data from external systems like Aspects or SIS:

  • The criterion must define where data comes from: LMS or external source

  • LMS should be the primary source when it has the data

• Each criterion must explicitly document the backends used and depends on (MySQL, Aspects, SIS), it can depend on multiple.

• New data backends can and must be registered before use.

• A criterion can support fallback evaluation – if the preferred source fails, use another.

• Cached data must be reused where possible to reduce load.

• If refreshing fails, the group uses its last known valid data, and a UI warning is shown.

• Groups with external dependencies must not be considered active until their first refresh is successful.

• The system must ensure Aspects data are accessible via the Superset API.

• Aspects should register what type of data is available to support criteria.

• The most reliable and available data source must be prioritized for each criterion:

  • If the LMS (e.g., MySQL) contains the required data and can be queried efficiently, it must be treated as the source of truth.

  • If the LMS does not hold the necessary data, an external system (e.g., Aspects) may be used as the source of truth, with the understanding that it might not be fully up to date.

  • Strict real-time synchronization between systems is not required, especially when using cached results with scheduled refresh.

6. Analytics with Aspects

• Dashboards in Aspects must support filtering by group.

• Aspects should support visualizing group-level data such as engagement and demographics.

Non-Functional Requirements

1. Performance & Scalability

Ensures the system works well even with lots of users and groups, with limits and fallbacks to keep things running smoothly.

• Group membership queries must be fast, even at large scale.

• The model must support thousands of learners per course or org.

• Query speed depends on the data backend (e.g., MySQL vs. Aspects).

• Each criterion must define limits for:

  • Query timeout (e.g., 5–10 seconds)

• Groups should have a constant max result size (e.g., max number of users) which can be overridden by operators.

• If limits are exceeded, use cached data and show a warning.

• Platform operators should be able to customize these limits.

• MySQL-based queries must be designed to avoid heavy joins or other performance-intensive operations.

• User groups should target manageable audiences; mass-scale communication is discouraged.

2. Reliability & Failover

Describes how the system should behave when something goes wrong, like a service or plugin failing.

• If an external service fails, the system must:

  • Retry, fall back to cached data, and flag the issue

  • Missing or broken criteria (e.g., plugin removed) should not break the platform.

  • Group data must stay consistent even if parts of the system fail.

• If the evaluation of a group fails due to a timeout, error, or external dependency issue, the system must:

  • Use the last known valid evaluation result if available.

  • If it is the group's first evaluation, do not mark the group as enabled until a successful evaluation is achieved.

  • Retry the evaluation up to X times. If no successful evaluation occurs, flag the group for review and show a clear alert in the UI.

  • Log evaluation failures, retries, and flagging actions for traceability.

• If a plugin defining criteria is uninstalled or missing, the system must disable the entire group and flag the failure until it’s manually fixed.

3. Extensibility

Covers how new data sources or criteria can be added without breaking the platform.

• The design must support adding new data sources or evaluation engines.

• The platform must remain usable even if a plugin fails to load.

• Each criterion must define how it is evaluated and by which backend.

4. Data Integration & Interoperability

Ensures the system works well with tools like Aspects and MySQL, using clear rules for how they connect.

• The grouping system must integrate well with existing tools like Aspects and MySQL.

• The LMS-Aspects communication must be clearly defined and documented.

5. Traceability & History

Makes sure it's easy to see why a user is in a group and what changes happened, both manually and automatically.

• It must be possible to understand how a user group was created and why a user belongs to it. We recommend a simple JSON string of the criteria values for the users.

• Manual and automated updates must be visible and auditable.

6. Maintainability & Evolution

Supports growth over time by allowing legacy features to be replaced gradually and rules to evolve.

• The model should eventually unify cohorts, teams, and course groups.

• Legacy groups must continue to work until fully replaced.

• The model should support long-term growth without breaking existing features.

• Criteria and group behaviors must be flexible to change over time.

7. Usability & Operator Control

Focuses on making the system easy to use for instructors and staff, with simple controls and clear information.

• Instructors and course operators must be able to create and manage groups without needing engineering help.

• The model must provide clear options to define, update, and refresh groups.

• Operators must be able to easily see when groups are outdated and how to fix them.