Overview

In our user grouping system, each group defines one or more criteria to decide which users belong to it. This design combines database models and runtime logic to support flexibility and future extensions.

We introduce two ways of modeling this approach:

1. Criterion as an instance of a criterion type: In this approach, each Criterion represents a single rule used by the group. It stores the necessary configuration for evaluation, such as the type (e.g., "last_login"), the operator (e.g., >), and the config value (e.g., 10 days). The logic is instead associated with the type string and dynamically resolved. This information is used at runtime to execute the correct criteria type class via a registry.

2. Criteria as a list of criteria: In this approach, Criteria acts as a container for a list of CriterionType entries. Each CriterionType defines all the logic and configuration required for evaluation: it includes fields like property (e.g., "last_login"), operator, config, and a reference to the criteria type class. The criterion definition is centralized within the CriterionType, and could be shared across multiple groups. The Criteria serves primarily to organize and compose multiple criteria for a single group.

The main difference between these approaches is how fields are distributed across models and how relationships are structured. Regardless of the setup, the key idea stays the same: criterion types classes define how each criterion behaves, and these are only available at runtime. They're looked up using a string identifier, similar to how Django's ContentType system works, which allows the logic to stay more decoupled from the database schema.

Goals

Instead of creating a separate model for every criteria type, this approach uses a model (Criterion) and links it to a criteria definition (CriterionType) persisted or not. The evaluation logic for each criteria is handled by Python classes that are registered at runtime.

This setup offers several benefits:

• Keeps the database schema simple while allowing flexible criteria configuration

• Makes it easy to add new criteria types without needing migrations

• Loads evaluation logic dynamically, keeping the system lightweight

• Support lightweight group definitions with minimal setup

• Allows logic or inputs to evolve without changing the database schema

This design aims to balance flexibility, extensibility, and simplicity, thus making it a strong option when you want runtime behavior with some structure in place.

Model Structure Overview

This section shows the structure of the registry-based approach for user group criteria. It includes a high-level diagram and explains how the main parts connect to support extension, evaluation, and configuration.

Option 1: Criterion as an Instance of a Criterion Type

Open

Core Components

• UserGroup
  Represents a named user group (e.g., "Inactive Learners"). Holds basic metadata and connects to multiple Criterion entries that define who belongs in the group.

• Criterion
  Stores the full configuration of a single rule (e.g., operator and value like “> 10 days”) and includes a type field (e.g., "last_login"). This type is used at runtime to find the associated evaluator logic. Each Criterion is a self-contained rule specific to the group using it.

• CriterionType
  Holds the property (e.g., "course_progress"), and other metadata that helps define how the rule works.

• Runtime Evaluator Registry
  Maps each type (e.g., "last_login") to a registered class that handles logic, expected input formats, and validation. The system uses this during evaluation to match users.

Option 2: Criteria as a List of Criterion (Criteria 1:N CriterionType)

Open

Core Components

• UserGrouping
  Defines the overall group, such as “High Risk Learners” and links to one Criteria, which holds the full criteria set.

• Criteria
  It serves as a container for multiple (possibly reusable) criteria. It references one or more CriterionType entries that describe each condition in the group, allowing multiple criteria to be bundled.

• CriterionType
  Holds the property (e.g., "course_progress"), operator, and any required configuration (like {"days": 30}). It defines how the rule works and could be reused by multiple groups.

• Runtime Evaluator Registry
  Loads classes for each CriterionType based on its property. These methods define the filtering logic for each rule type.

Extension Workflow

Here's what the extension process should look like, based on how the system is expected to work:

1. Create a criteria class
   Define the criterion class in your plugin. This class will hold any (typed) field needed to configure the criteria (e.g., days since last login, enrollment track, etc.) and the criteria evaluator.

2. Define the evaluator logic
   Write the logic that will return the list of matching users based on the models data. This method should be part of the class criterion type class.

3. Register the new type
   Use the registry to register the new criterion so it becomes available to the system. This lets it show up in the UI, etc.

4. Install the plugin
   Install the plugin or app where your new criterion is defined.

Evaluation Workflow

Here's what the evaluation process should look like, based on how the system is expected to work:

1. Start Evaluation
   The system begins evaluating a specific user group, usually triggered by a refresh request or schedule.

2. Retrieve User Group and Criteria
   It fetches the UserGrouping, gets the corresponding type (this depends on the chosen approach).

3. Resolve Each Criterion Type
   Each Criterion/Criteria instance has a type associated with a concrete criterion type, after getting it loads the corresponding class registered. This should associated somehow to the model in runtime so it's easy to access.

4. Evaluate Each Criterion
   The evaluator for each criterion type returns a list of user IDs that match the criteria defined by that criterion (e.g., last login > 10 days).

5. Combine Results
   The system combines all lists using logical criterias (e.g., AND across criteria) to produce the final set of user IDs.

6. Update Group Membership
   The resulting user list is used to update the user group membership in the system.

The main difference with other approaches is how each criterion type is accessed, and how the Criterion is used to reference it. In this model, evaluators which are merely classes and JSON values (e.g., “10 days”) are stored in a single model, making the evaluation process dependent on runtime definitions and registries. The Django model doesn't work by itself, it depends on runtime definitions like classes where the evaluation happens. As mentioned before, this works similarly to Django ContentTypes.

Pros & Cons

This section outlines the pros and cons of this runtime-heavy approach. It's lightweight and easy to extend, but it relies on dynamic logic and lacks built-in structure or validation, which can become harder to manage over time.

Pros

Cons