/
[Proposal] Discussion Feature Additions: Soft Deletion, Mute/Ban, Spam Detection

[Proposal] Discussion Feature Additions: Soft Deletion, Mute/Ban, Spam Detection

Open edX Proposal (OEP) โ€” Discussions: AI-Assisted Spam Moderation and Learner Mute/Block Controls

OEP Metadata

OEP: Assigned by maintainers on acceptance

Title: AI-Assisted Spam Moderation and Learner Mute/Block Controls for Open edX Discussions

Author(s): Spencer Tiberi (2U/edX)

Stakeholders: Product (2U/edX), Partner Institutions, Open edX Community Maintainers, Security/Privacy Reviewers

Status: Proposal

Type: Feature / Architecture (cross-repository)

Created: Apr 15, 2026

Target Release: Next named release after acceptance (to be proposed during review)

Related OEPs: OEP-17 Feature Toggles; OEP-30 PII Markup and Auditing; OEP-4 Application Authorization (Scopes), as applicable

Github: [Proposal] Add Discussion Moderation Features: Soft Delete, Mute/Ban, AI Spam Flagging ยท Issue #511 ยท openedx/platform-roadmap

1. Abstract

This OEP proposes two enhancements to Open edX Discussions (Forums v2):

  • Soft delete: allow messages to be hidden from view without permanently deleting them, if the need to recover becomes necessary. Course team members can permanently delete messages if desired upon review.

  • AI-assisted spam/scam moderation (AI Monitoring): Automatically flags and temporarily hides high-risk posts from general learner view pending human review (i.e. โ€œsoft deletesโ€); opt-in via feature toggles.

  • Learner mute/report controls: Enables learners to hide another userโ€™s discussion content for themselves, reducing exposure to spam or harassment without requiring staff intervention. Learners can report others for potential bans, with course teams making the final decision.

  • Course ban controls: Similarly, staff can review reported messages and ban learners.

The features are opt-in, privacy-conscious, human reviewable (โ€œhuman-in-the-loopโ€), and compatible with existing moderation workflows.

2. Motivation and Problem Statement

2.1 AI-Assisted Moderation

At-scale Open edX deployments can experience persistent, adaptive spam and scams in discussion forums. Purely reactive strategies (manual moderation, keyword blacklists, CAPTCHA) create operational burden and lag behind adversaries. AI-assisted monitoring provides earlier detection, reduces learner exposure, and allows staff to focus on nuanced cases.

2.2 Learner Mute/Block

Even with staff moderation, learners may face harassment, repeated unwanted contact, or spam that slips through. A per-learner mute/block feature enables immediate self-protection and improves learning focus while preserving staff moderation for policy enforcement.

3. Goals and Non-Goals

3.1 Goals

  • AI Monitoring

    • Minimize learner exposure to likely spam/scams via near real-time temporary hides.

    • Provide a human review queue with restore/soft-delete actions.

    • Offer org/site-level opt-in toggles and a safe โ€œshadow mode.โ€

  • Learner Mute/Block

    • Allow learners to mute/block another learner so their posts are hidden or collapsed for the muter.

    • Provide clear affordances to unmute/unblock or show content temporarily.

3.2 Non-Goals

  • Automatic hard deletes by AI.

  • Configurable policy-builder UI for institutions (beyond enable/disable and thresholds).

  • Broad โ€œtoxicityโ€ classification; focus is spam/scam.

4. Proposed Solution Overview

4.1 AI Monitoring Flow

  1. Learner submits a thread/reply/comment.

  2. Cheap, local triggers evaluate risk (links, contact patterns, velocity, etc.).

  3. If triggered, system sends a minimal payload to a pluggable classifier.

  4. Classifier returns a verdict. On โ€œhide,โ€ post is hidden from other learners while visible to the author and moderators.

  5. Moderators review and either restore or soft-delete; all actions are logged.

4.2 Learner Mute/Block Flow

  1. Learner chooses โ€œMute/Block userโ€ from a post or profile in Discussions.

  2. Platform records a per-learner preference (course-scoped by default).

  3. Rendering logic collapses/omits muted user content for that learner; muted user is not notified.

  4. Learner can unmute/unblock at any time.

5. Detailed Specification

Implementation details map to the modern Discussions stack (MFE + backend services) and should be adapted as needed for legacy forums if required.

5.1 Feature Toggles and Configuration

  • DISCUSSIONS_AI_MONITORING_ENABLED (org/site): enables classification pipeline.

  • DISCUSSIONS_AI_MONITORING_SHADOW_MODE (org/site): log-only classification; do not hide.

  • DISCUSSIONS_AI_MONITORING_HIDE_ENABLED (org/site): enforce hides from learners.

  • DISCUSSIONS_LEARNER_MUTE_ENABLED (site): enable learner mute/block UI and APIs.

All toggles should follow OEP-17 guidance (clearly documented owner, default off, explicit cleanup plan).

5.2 Trigger Heuristics

  • High author velocity (e.g., more than 5 posts in 24 hours).

  • Presence of external links or URL shorteners.

  • Contact info patterns (phone, WhatsApp, Telegram, email obfuscations).

  • Impersonation-like usernames (e.g., โ€œedX staff,โ€ โ€œcourse_adminโ€).

  • Configurable suspicious keyword list.

If no trigger matches, skip classification to minimize cost and latency.

5.3 Classifier Interface (Pluggable)

Introduce a provider abstraction with a standard API:

POST /api/discussions/moderation/classify { "post_id": "uuid", "course_id": "course-v1:Org+Num+Run", "org": "Org", "user_id_hash": "sha256(...)", "content_text": "Post content (truncated as needed)", "links": ["example.com", "t.me/..."], "metadata": {"client_version":"...", "lang":"..."} } Response: { "verdict": "allow" | "hide", "confidence": 0.0-1.0, "reasons": ["link_farm","contact_info"], "classifier_version": "vX.Y" }

Backends: external LLM API, in-house model, or rule-only fallback. Institutions can disable or select providers via settings.

5.4 Post Hiding Semantics

  • Hidden posts are not shown to learners other than the author.

  • Authors see their own post with a neutral banner (e.g., โ€œUnder reviewโ€).

  • Moderators/staff see full context and can restore or soft-delete.

  • Hidden state is reversible; no hard deletes by AI.

5.5 Moderator Review Queue

  • Filtering by course, date, confidence, trigger reason, and author.

  • Batch actions for restore and soft-delete.

  • Context view (thread lineage, author history, prior flags).

  • Action logging with actor, decision, timestamp, reason codes.

5.6 Learner Mute/Block Data Model

  • Table schema: (muter_user_id, muted_user_id, course_id, created_at).

  • Scope: default course-scoped. Consider optional site-wide scope later through settings.

  • Constraints: unique key on (muter, muted, course_id) to prevent duplicates; soft-delete for reversibility.

5.7 UI Behavior and Accessibility

  • Mute/Block action accessible from post overflow menu and profile cards.

  • Muted content is collapsed with โ€œShow postโ€ and โ€œUnmuteโ€ options.

  • Respect in-thread and list views; ensure consistent behavior in notifications.

  • Keyboard navigable controls; clear labels and status messages for assistive tech.

5.8 API Surfaces and Permissions

  • POST /api/discussions/v1/mutes/: body {muted_user_id, course_id}; authenticated learner; idempotent.

  • DELETE /api/discussions/v1/mutes/{muted_user_id}?course_id=...: unmute.

  • GET /api/discussions/v1/mutes?course_id=...: list muted users for current learner.

  • GET /api/discussions/v1/moderation/queue: staff-only; paginated queue of AI-hidden posts.

  • POST /api/discussions/v1/moderation/{post_id}/restore and /soft_delete: staff-only.

If scopes are used, align with OEP-4; otherwise, use role checks consistent with Discussions APIs.

6. Privacy, Legal, and Security

6.1 Data Minimization and Redaction

  • Send only essential fields to classifiers; hash user identifiers.

  • Redact or mask emails/phones before external transmission when feasible.

  • Truncate long content and exclude attachments.

6.2 PII Handling and Auditing

  • Document data flows per OEP-30; mark fields containing PII.

  • Maintain audit logs for classification calls and moderator actions.

6.3 External Provider Use

  • Require contractual guarantees: no training on customer data, retention limits, SOC2/ISO controls.

  • Provide admin controls to disable or run shadow mode.

6.4 Abuse and Adversarial Risks

  • Do not expose granular detection reasons in learner UI.

  • Rate-limit classification calls; circuit-break on provider errors.

  • Monitor false-positive rates and adjust thresholds.

7. Backwards Compatibility

  • When toggles are off, behavior is unchanged.

  • New hidden state is ignored by legacy UIs unless enforcement is enabled.

  • Mute/block is additive and limited to personal visibility; no impact on roles/permissions.

8. Observability and Metrics

  • AI Monitoring: classification rate, hide rate, moderator restore vs. delete, latency percentiles, provider error rate, cost per 1k posts.

  • Mute/Block: adoption rate, unmute rate, effect on report/spam flags, support contacts related to harassment.

9. Rollout Plan

  1. Develop feature flags, APIs, and minimal UI; add unit/integration tests.

  2. Enable shadow mode on a small set of courses/orgs; collect metrics and tune triggers.

  3. Enable hide enforcement for pilot orgs; closely monitor false positives and learner feedback.

  4. Gradually expand opt-in availability; publish documentation and admin guidance.

  5. Reassess defaults and consider deprecating shadow mode after maturity.

10. Alternatives Considered

10.1 Rules-Only Filtering

Pros: simple, low cost; Cons: high evasion risk, ongoing maintenance, poor generalization.

10.2 Always-On Classification

Pros: uniform coverage; Cons: cost, latency, privacy exposure. Decision: trigger-gated classification.

10.3 Learner โ€œBanโ€ vs. โ€œMute/Blockโ€

Platform-wide โ€œbanโ€ by learners invites abuse and retaliation. Decision: per-learner mute/block for visibility control; staff retain ban/suspension tools.

11. Risks and Mitigations

  • False positives hide legitimate posts: conservative triggers, shadow mode, easy restore, ongoing threshold tuning.

  • Posting latency increases: async classification, optimistic UI, fast-path allows, short timeouts.

  • Privacy concerns: minimize/redact content, provider contracts, clear opt-out.

  • Moderator workload spikes: batch actions, filters, triage heuristics, alerting only on confidence thresholds.

12. Open Questions

  1. Exact Discussions surface(s) targeted: legacy forums, modern MFE, or both?

  2. Terminology preference: โ€œmute,โ€ โ€œblock,โ€ or โ€œignoreโ€ per community norms?

  3. Initial classifier backend and configuration strategy?

13. Implementation Notes

  • Repositories likely touched: Discussions MFE, Discussions backend service or edx-platform APIs, IDA settings/config.

  • Database migrations: new table for mute relationships; new post-state field for โ€œhidden_by_ai.โ€

  • i18n: new strings for banners, actions, and accessibility labels.

  • Docs: admin guide for toggles; moderator guide for the review queue; learner guide for mute/block.

Comments