...
All of the Django-using Open edX IDAs are now on Django 2.2, which will reach end of life in April 2022. edX wants to upgrade Django to version 3.2, the next LTS (long term support) release. We want this project complete by October 15, 2021, in time for the next Open edX release, Maple. Key contacts are Jeremy Bowman (Deactivated) and Natalia Berdnikov (Deactivated).
What is this about?
Briefly describe service and its role and impact on our systems.
Who owns it within edX?
What do we plan to do with it in a nutshell?
Is this a small, medium or large project (ballpark)?
Any other relevant context to help understand importance, size or impact.
Most of the services in Open edX use the Django framework to handle all of the low-level “this is what you need to even have a functional, maintainable web application” bits. Each repository in Open edX has a designated owning team ultimately responsible for keeping it working with a still-supported version of Django at all times, but the Arch-BOM and Arbi-BOM teams will be utilizing prior experience with major Django upgrades to do much of the planning, automation, and coordination for the process. We will be upgrading essentially all Django-using code in Open edX to work not just with the prior long-term support (LTS) release (Django 2.2), but also the recently released one (Django 3.2). This is expected to require less effort than previous Django upgrades that made more backwards-incompatible changes, but still involve more code changes than upgrading most other components of our technology stack due to the fact that Django influences how we structure our code in many ways.
Why are we doing this?
...
Why does this need to be done? What is the impact of not doing this?
...
we
...
doing
...
Why did we pick this version to upgrade to?
...
Why does this need to be done now?
...
this
...
?
Because web application security depends on effectively mitigating all currently known attack vectors, it’s pretty important to stay on a framework version that’s still getting regular updates to fix newly discovered attacks. New Django versions also often address pain points in previous releases, improve performance, and/or add useful new functionality that can accelerate future development. Django is one of the best web application frameworks with respect to facilitating upgrades and managing backwards-incompatible changes; this would be even more challenging in most available alternatives (in any programming language). In fact, many Java web applications end up forking and maintaining a specific version of their framework indefinitely because the effort to do an upgrade exceeds available development resources.
The Django 2.2.x release series will cease getting security fixes in April 2022, before the end of support for the next Open edX release (Maple). The latest, 3.2.x release we’re upgrading to will continue getting security fixes until April 2024.
...
Have we done this before
...
Did the last time go well?
...
?
...
We have already successfully upgraded Open edX from one Django LTS release to the next 3 times; the Django release schedule is such that we have to do this roughly every 2.5 years. Each time, both edX and Django get better at streamlining the process. The Open edX codebase is growing over time, but the effort needed to perform each upgrade is nevertheless reducing slightly over time due to these improvements. We get a lot of assistance from the Arbi-BOM team at Arbisoft and other members of the Open edX community, but previous attempts to enlist further aid from consulting firms haven’t proven successful because the Open edX codebase is too large for them to be able to scope and estimate the work involved in a timely manner.
...
Analyze and track Django compatibility issues with a master list of all Django-using dependencies of Open edX, rather than on a per-IDA basis (since many packages are used in multiple IDAs, and some IDAs start the upgrade in earnest much later than others). The current state of this analysis can be found at Dependency Supported Python/Django Versions.
Figure out early on which upgrade tasks can be effectively taken on by the broader Open edX community, and add them to a GitHub Project board so they can be efficiently distributed even to community members with little Jira experience.
Explicitly provide a menu of options for each squad owning an IDA to decide how much of the development work to delegate to the Arbi-BOM team. We expect this to reduce confusion around who is responsible for what, and allow squad managers to better account for the upgrade in their planning process.
What do we plan to do?
...
What is the scope of this project?
...
On high level, what is our plan? Identify big milestones.
...
do
...
?
We need to perform the steps in Django 3.2 Upgrade: Steps to Upgrade a Repository for each of the IDAs in Django 3.2 Upgrade List of IDAs, as well as each of their dependencies which is maintained by edX or another member of the Open edX community. The key stakeholders are identified in https://openedx.atlassian.net/wiki/spaces/AC/pages/2960392389/Django+3.2+Upgrade+Roles+and+Communication#Team-and-Roles.
Are there risks and dependencies?
...
Are there any known risks?
...
How can we mitigate them?
...
Have we reviewed system requirements (e.g. AWS, Python, etc.) for this upgrade?
...
dependencies
...
?
Each time we perform one of these upgrades, we inevitably find a few dependencies which don’t add support for the new Django versions in a timely manner (or cease to be maintained altogether).
...