/
Process for Integrating Django Security Patches in an Open edX Named Release

Process for Integrating Django Security Patches in an Open edX Named Release

1. Introduction

  • Purpose: This guide details a systematic approach for integrating Django security fixes into the release branches, aiming to enhance the platform's security and performance.

  • Scope: This process covers steps from the initial security alert to the final release, focusing on clear and efficient execution.

  • Target Audience: Designed for maintainers and members of the Build Test Release (BTR) working group.

2. Prerequisites

  • Skills and Knowledge: A foundation in Django, Python, and the Open edX codebase.

  • Tools and Resources: Ensure access to the Open edX codebase, tools to build the environment locally, and community communication channels.

3. Security Alert and Notification

  • Channels: Security alerts can be sourced from Django’s official channels, mailing lists, or weblog notifications. The best and most reliable source is the public Google group, as it announces with a week's anticipation when the security patch will be released and its severity.

  • Identifying Which Repositories Should Be Patched: All tagged repositories that use Django as a dependency should be patched. Usually, when a new release is made, a document is created listing the tagged repositories. From there, you can identify which repositories should be patched (e.g., Redwood release-tagged repos).

  • Notify BTR Working Group by Creating an Issue: An issue should be manually created in the BTR GitHub project to initiate the patch application process. Make sure to list all the repositories that need to be patched in the task list in the issue description.

4. Evaluation and Prioritization

  • Severity Assessment: Evaluate the advisory description and categorize the severity level to understand its potential impact on the Open edX project.

  • Prioritization: Refer to the Django release process. Given that Django security fixes are crucial, they should be applied when the patch is out. All compatible fixes should be implemented, with priority given to the latest patch release.

5. Implementation Process

  • For Each of the listed repositories to be patched:

    • Fork and Branch: Initiate a branch from the named release to isolate the security-fix application.

    • Code Modification: Update the Django version in all requirements files and, if necessary, examine and update Django dependencies to align with the security patch.

    • Open PRs: Launch PRs in each repository, linking them to the BTR project issue for tracking.

  • Alternative: Use Script to Open PRs: An easier and much quicker alternative is to use a script to streamline the PR opening process across multiple repositories. The following script can be used as a guide.

    # Run a command for every repo found somewhere beneath the current directory. # # $ gittree git fetch --all --prune # # To only run commands in repos with a particular branch, use gittreeif: # # $ gittreeif branch_name git fetch --all --prune # # If the command has subcommands that need to run in each directory, quote the # entire command: # # $ gittreeif origin/foo 'git log --format="%s" origin/foo ^$(git merge-base origin/master origin/foo)' # # The directory name is printed before each command. Use -q to suppress this, # or -r to show the origin remote url instead of the directory name. # # $ gittreeif origin/foo -q git status # gittreeif() { local test_branch="$1" shift local show_dir=true show_repo=false if [[ $1 == -r ]]; then # -r means, show the remote url instead of the directory. shift local show_dir=false show_repo=true fi if [[ $1 == -q ]]; then # -q means, don't echo the separator line with the directory. shift local show_dir=false show_repo=false fi find . -name .git -type d -prune | while read d; do local d=$(dirname "$d") git -C "$d" rev-parse --verify -q "$test_branch" >& /dev/null || continue if [[ $show_dir == true ]]; then echo "---- $d ----" fi if [[ $show_repo == true ]]; then echo "----" $(git -C "$d" config --get remote.origin.url) "----" fi if [[ $# == 1 && $1 == *' '* ]]; then (cd "$d" && eval "$1") else (cd "$d" && "$@") fi done } gittree() { # @ is in every repo, so this runs on all repos gittreeif @ "$@" } # Define the branch to work on export BRANCH=<open-release/ztree.master> # What package are we upgrading? From what maj.min to what new version? export PKG=Django OLDMAJ=<old-major-version> OLDMIN=<old-minor-version> NEWVER=<4.2.alpha> gittree git fetch --all gittreeif origin/$BRANCH "git switch $BRANCH; git pull; git status -s -b" gittreeif origin/$BRANCH git branch --set-upstream-to=origin/$BRANCH $BRANCH # Eyeball the current state gittreeif origin/$BRANCH -q pwd | while read d; do rg -i "^${PKG}==" -g '*.in' -g '*.txt' $d | sed 's/#.*//'; done # Create a shell script to make the change gittreeif origin/$BRANCH -q pwd | while read d; do rg -n -i "^${PKG}==${OLDMAJ}\.${OLDMIN}" -g '*.in' -g '*.txt' $d | awk -F: "{print \"sed -E -i '\" \$2 \"s/==${OLDMAJ}\\\\.${OLDMIN}(\\\\.[0-9]+)?/==$NEWVER/' \" \$1 \" # \" \$3}" ; done > /tmp/doit.sh # Run the shell script source /tmp/doit.sh # Eyeball what the shell script did gittreeif origin/$BRANCH git status -s -b GIT_PAGER=cat gittreeif origin/$BRANCH git diff # Checkout a custom branch gittreeif origin/$BRANCH git checkout -b <patch-django-ztree-4.2.alpha> # Commit the changes gittreeif origin/$BRANCH git commit -am "chore: upgrade ${PKG} to ${NEWVER}" # Authenticate the Github CLI tool gh auth login # Create personal forks of all repos gittreeif origin/$BRANCH gh repo fork --remote=true --remote-name=<personal-remote-name> # Push to the fork gittreeif origin/$BRANCH git push --set-upstream m <personal-remote-name> # Create pull requests gittreeif origin/$BRANCH /bin/sh -c 'gh pr create --repo $(git remote get-url origin) --base $BRANCH --title "chore: update Django to <4.2.alpha> for <ztree>" --body "See: https://github.com/openedx/wg-build-test-release/issues/<issue-number>"'

    .Credit for this script goes to the conversation from this link: Django 2.2.19 security update

  • Alternative: Use workflow in edx-platform to update a single Python requirement: You can also use the GitHub workflow to update a single Python requirement in the edx-platform edx-platform/.github/workflows/upgrade-one-python-dependency.yml at master · openedx/edx-platform.

  • Testing: Ensure all PR tests and checks pass before proceeding to request reviews or merge the PRs.

6. Review and Verification

  • Code Review: Each PR should undergo a thorough peer review to validate the quality and security of the code modifications.

7. Release and Reporting

  • Release: Coordinate with the BTR Release Manager to ensure a new release of the latest named version is made to integrate the security fixes effectively.

  • Reporting: Document or communicate any anomalies or issues by opening Issues in the BTR project for fixes and future reference or starting a conversation in the public Slack channel of the BTR.

Notes, Considerations, and Open Questions: