Proposal: PII linter for edx-lint

Overview

This proposal introduces a default PII (Personally Identifiable Information) linter within the existing edx-lint Pylint plugin.

The goal is to help developers identify potential PII risks during development by:

• Detecting incorrect or stale PII annotations.

• Detecting unsafe PII exposure patterns in application code.

The linter will be enabled by default and will rely solely on static AST analysis, requiring no Django runtime configuration.

PII Definition Source

The linter will use the guidance defined in OEP-30 as the source of truth for PII-related concepts and annotations:

https://docs.openedx.org/projects/openedx-proposals/en/latest/architectural-decisions/oep-0030-arch-pii-markup-and-auditing.html

The rule implementations proposed in this document are intended to complement and enforce the PII annotation model established by OEP-30.

Problem Statement

Current PII annotation tooling helps document where PII is stored, particularly in models and annotated classes. However, annotations can become stale over time as new fields are added. A class that was originally marked with .. no_pii: may later introduce fields that contain PII without the annotation being updated.

Additionally, developers can unintentionally expose PII through regular application code.

Common accidental exposure points include:

• Logs

• Print/stdout statements

• Exception messages

• API error responses

• Tracking and analytics events

• External service requests

• URLs or query parameters

• Cache and session keys

These issues are often discovered during code review or after deployment. Detecting them during linting provides earlier feedback and reduces the risk of accidental PII exposure.

Rules

The PII linter is intended to support the following rule categories:

Note: The coloured blocks are already taken care in CI quality checks as pii_check in edx-platform. These are included in this as we plan to add them as part of edx-lint.

Current Scope

The initial implementation focuses only on Django models that use the existing OEP-30 PII annotations.

• pii-invalid-no-pii-annotation is evaluated only for models annotated with .. no_pii: to ensure they do not introduce fields that likely contain PII.

• pii-missing-squelch [pii-in-log, pii-in-print, and pii-in-exception ] are evaluated only for models annotated with .. pii: to ensure PII is not exposed through logs, print statements, or exception messages without following the required SQUELCH_PII_IN_LOGS pattern.

The goal of this first phase is to validate both PII annotation enforcement and PII exposure detection before expanding to additional rule categories.

All other rules listed in this document are considered future scope and are not part of the initial implementation.

Proposed Solution

A new Pylint linter will be added to edx-lint:

The linter will be registered through the existing plugin framework and enabled by default through the bundled Pylint configuration.

The initial implementation will focus on two areas:

1. Validation of .. no_pii: Annotations

Classes annotated with .. no_pii: will be inspected for likely PII fields using configurable PII term matching.

If a class marked as containing no PII introduces fields such as email addresses, usernames, phone numbers, IP addresses, or similar sensitive attributes, the linter will emit:

This helps ensure that .. no_pii: annotations remain accurate as code evolves.

2. Detection of PII Exposure in Application Output

The linter will inspect:

• Logging statements

• Print/stdout/stderr output

• Exception messages

When likely PII is detected, the linter will verify that the output follows the existing SQUELCH_PII_IN_LOGS pattern and provides a corresponding non-PII alternative.

The linter will rely entirely on static AST analysis and will not require Django runtime configuration.

Technical Implementation

New Linter

Create a new linter:

The linter will implement a PiiLinter class extending Pylint's BaseLinter.

AST Visitors

The initial implementation will primarily inspect:

• visit_classdef

• visit_call

• visit_raise

Class Annotation Validation

For classes annotated with .. no_pii:, the linter will analyze class members and field definitions for likely PII fields using configurable PII term matching.

The analysis may include:

• Django model field definitions

• Class attributes

• Assignment targets

• Annotated assignments

• Instance attributes assigned within class methods

Examples of likely PII members include:

• email

• username

• phone_number

• ip_address

• retired_username

Reference: https://2u-internal.atlassian.net/wiki/spaces/AT/pages/3851157511/Proposal+Default+PII+linter+for+edx-lint#PII-Definition-Source

If a likely PII field is detected in a class annotated with .. no_pii:, the linter will emit:

PII Exposure Detection

The linter will analyze common output sinks including:

Logging

• logger.info(...)

• logger.warning(...)

• logger.error(...)

• logger.exception(...)

• logging.info(...)

• logging.warning(...)

• logging.error(...)

Print / Stdout

• print(...)

• self.stdout.write(...)

• self.stderr.write(...)

• stdout.write(...)

• stderr.write(...)

Exceptions

• raise ValueError(...)

• raise Exception(...)

• Other exception constructors containing formatted messages

Recursive PII Detection

Arguments passed into these sinks will be recursively inspected, including:

• Variables

• Attributes

• Dictionaries

• Nested structures

• Formatted strings

• Function arguments

• Collection types

The linter will use configurable PII term matching to identify likely PII references.

SQUELCH_PII_IN_LOGS Enforcement

When likely PII is detected in a log message, print statement, or exception message, the linter will verify that the code follows the existing SQUELCH_PII_IN_LOGS pattern.

The linter should recognize common flag usages such as:

• settings.FEATURES['SQUELCH_PII_IN_LOGS']

• settings.FEATURES.get('SQUELCH_PII_IN_LOGS')

• settings.SQUELCH_PII_IN_LOGS

• getattr(settings, 'SQUELCH_PII_IN_LOGS', False)

The linter will validate that:

• The PII-containing output is controlled by SQUELCH_PII_IN_LOGS.

• A corresponding non-PII alternative exists.

• The non-PII alternative does not contain likely PII.

• Safe identifiers such as user_id are used where possible.

Violations will result in pii-missing-squelch .

Reviewed Exceptions

In some cases, names such as email or username may not represent user PII and may trigger false positives.

For reviewed exceptions, developers may:

• Adjust repository-specific PII configuration.

• Add approved safe key patterns.

• Use standard Pylint inline suppressions after review.

Example:

These suppression mechanisms should be used sparingly and only when the detected field does not represent user PII in the given context.

Local Development

Developers can run the linter locally using the standard edx-lint workflow.

Examples:

or

depending on how the repository invokes edx-lint.

This allows developers to identify and address violations before opening a pull request.

CI Integration

The linter will be enabled by default as part of the existing edx-lint Pylint plugin.

When a pull request is opened, the repository's existing quality checks will execute the linter as part of the standard linting workflow.

Any violations will fail the linting step and prevent the pull request from merging until:

• The issue is fixed.

• The code is updated to follow the required pattern.

• An approved suppression is added where appropriate.

This provides early feedback during code review and helps prevent accidental introduction of PII-related issues.

Future Improvements

A future phase may complement the static linter with runtime validation as part of the existing GitHub Actions CI workflow.

During the existing test execution, a lightweight runtime scanner could capture emitted log messages and exception messages and inspect their rendered output for likely PII. This would be implemented by integrating with the Python logging framework and the test runner, allowing the scanner to analyze the actual runtime output without requiring changes to application code.

This approach helps detect PII that cannot be determined through static AST analysis alone, such as PII exposed through object __repr__() implementations or dynamically constructed messages.

The runtime validation is intended to complement, rather than replace, the static AST linter by providing additional confidence in detecting PII exposures during CI.

Repository-wide Audit

The reusable PII detection logic can also be executed as a scheduled repository-wide audit. Unlike PR validation, the audit scans the entire repository, identifies existing PII-related issues, and generates a report for maintainers without blocking development.

The audit report may include:

• Rule violated

• Location (file path, line number, and corresponding source code)

• Detected PII term

• Recommended remediation

Example:

The audit can be scheduled (for example, weekly through GitHub Actions) to provide continuous visibility into repository-wide PII compliance while reusing the same detection engine as the Pylint linter.