Rationale: Beyond DEPR

Today, we have the Deprecation and Removal (DEPR) process, and in many ways it serves us well. What could be better? In decreasing order of importance:

• Lack of a defined grace period, i.e., an explicit window of time between the end of the comment period (“DEPR Accepted”) and the beginning of removal (“DEPR Removing”). The DEPR OEP assumes that as soon as a ticket is accepted, it is safe to be removed. This usually isn’t the case--typically, the decision to make a breaking changes happens ahead of of the change itself.

• “Proposed” and “Communicated”: Devs often creat tickets in “DEPR Proposed” and leave them there, assuming that they have been proposed. But they haven’t, they’re just a draft! They’re not truly proposed until they’re moved to “DEPR Communicated”, which is actually the comment period.

• Not all breaking changes are deprecations or removals We now use DEPR for all breaking changes, including environment upgrades and changes to API behavior, neither of which are “deprecations” in the commonplace usage of the term. It’s weird to say something like “Open edX deprecated Python 3.12”; it’s much more normal to say “Open edX dropped Python 3.12”. It’s also weird that the end state of every DEPR ticket is “Removed”, when the end state is often something like “we added a table, and your app will crash if you do not backfill it”-- quite the opposite of a removal!

• “Deprecated”: Many devs understand “deprecated” to mean that something is no longer supported. The DEPR OEP defines it, somewhat tautologically, to mean something that raises a deprecation warning. However, many devs and most nontechnical contributors take “deprecated” to be a synonym to “removed”. Statements like “we are deprecating $old_feature” are confusing, when the $old_feature repo is still there and it still works. To be clear, we have to say unambiguous things like “we are proposing that $old_feature will not be supported in $release, please comment by $date”.

Introducing: DROP-IT

DROP-IT is a unified process to propose, consensus-build, and disseminate any backwards-incompatible or otherwise disruptive change to Open edX code. It is an evolution and rebranding of the DEPR (deprecation & removal) process. Changes covered by DROP-IT include:

• Deletions: Removal of features, behaviors, APIs, and/or code

• Replacements: Switching one implementation out for another

• Operational Impacts: Changes requiring site operators (or course operators) to update their environment, migrate their content, etc.

Open

The DROP-BEAR, the unofficial mascot of DROP-IT. Photo credit CNN.

To propose a DROP-IT, developers create GitHub issue on the DROP-IT board with the following queryable fields:

• Title: One-line synopsis of the drop, e.g. [DROP-IT] End of support for Python 3.12

• State: Status of the DROP-IT, explained in more detail below

• Assignee: GitHub user responsible for moving the DROP-IT along

• Last date for comments: Last date for giving feedback on the proposed DROP-IT

• Grace period start date: Earliest date when site operators using master can begin prepping for the DROP’s impact. (In some cases, this will be the same as Last date for comments)

• Grace release: First named release during which operators can prepare for the DROP-IT before backwards-incompatible impacts happen. In some cases, this will be “None”.

• Grace period end date: Earliest date when site operators using master should expect the DROP-IT to impact them.

• Impacted release: First named release after the grace period. The backwards-incompatible impacts of the DROP-IT will occur in this release.

and the following details:

• Details: What are the deletions, replacements, and operator impacts included in this ticket? If there are deletions, who are they likely to impact? If there is a replacement, what is the feature delta?

• Migration: How should developers, site operators, and end users prepare for the impacts?

• Rationale: Why will the benefit of these impacts outweigh the cost?

• Timeline Considerations: What is the rationale for the proposed grace period (or lack thereof)? If the Grace period start date is delayed, then will the Grace period end date extend further into the future, or is it a fixed date (e.g., an external end-of-life date)?

DROP-IT States

Draft

DROP-ITs start here and stay here until they are ready for the formal comment period. It is fine to use this state for ideation and informal discussion. It is also fine to use this state to rework an existing ticket.

What’s next? When ready, the ticket moves to Comment Period. If it stall for multiple release cycles with no hope of becoming ready, then it may be moved to Cancelled.

Comment Period

The DROP-IT has an assignee who has fully fleshed it out with details and a timeline. They have formally communicated the ticket out so that interested community members can express their support, concerns, questions, and/or suggestions. In addition to commenting on the DROP-IT's impact, impacted parties are invited to give feedback on the ticket’s timeline.

How long? By default, leave two weeks for comments. Controversial DROP-ITs may deserve an extended period. Trivial DROP-ITs may need less. Generally, just try to leave a reasonable amount of time so that an impacted stakeholder could see the ticket and respond.

What’s next? Considering all comments, the assignee should consider whether or not the DROP-IT will be net beneficial to the Open edX community. If so, they can move it to Approved. Otherwise, they can make minor amendments to the ticket and extend the Comment Period. If the ticket needs major rework it can be sent back to Draft; if it is not viable at all, it is Cancelled.

Approved

The DROP-IT comment period is closed. The assignee now needs to coordinate some upstream work, which they will strive to complete this work within the timeline that they specified:

• For deletions: As is appropriate and feasible, deprecation messages are logged by code and APIs which are slated for removal. Deprecation notices are added to documentation.

• For replacements: The replacement UIs / APIs / etc. are all production-ready. If there are feature deltas, they are documented.

• For operational impacts: The migration path is ready-to-use, tested, and documented.

What’s next? Once the above is complete, the Grace Period can begin.

What if there are issues? Replacements can take longer than expected, and migrations can get blocked. Within reason, the assignee should post updates on the ticket and push the timeline out in response to delays. However, if there is major timeline slippage, or if the assignee actually wants to accelerate the timeline, then the ticket should move back to Draft or the Comment Period for feedback.

Grace Period

For those following master, impacted site operators, devs, and users can now prepare for the impacts of the DROP-IT.

• For deletions: They can see deprecation warnings and notices, and react accordingly.

• For replacements: They can switch to the replacement UI/API/etc. in their production environment.

• For operational impacts: They can update their production environment and perform any necessary migration steps.

How long? Back in the comment period, you will have proposed how long the grace period is and whether it includes a named release cut (a “grace release”).

• When in doubt, default to a 3-month period, including a grace release.

• You might propose a longer grace period when some or all of the following are true: the impacts are in the Core Product; the deleted features/code/APIs have known users; the replacement has a feature delta; the migration steps are non-trivial; the impacted stakeholders express a need for a longer timeline; supporting the grace period carries little maintenance burden.

• You might propose a shorter grace period and/or no grace release when some of the following are true: the impacts are on the periphery of the platform; the deleted features/code/APIs are not being used; replacement is fully at feature parity; the migration is trivial; the impacted stakeholders express support for an accelerated timeline; supporting the grace period incurs significant maintenance burden; the change timeline was already communicated via documentation.

  • If most or all of the above are true, you might propose to skip the grace period and grace release entirely. In this case, the Last date for comments, the Grace period start date, and the Grace period end date are all identical.

Grace Ended

Now that the grace period has wrapped up, upstream Open edX contributors are free to make the changes specified in the DROP-IT. The ticket should not stay in this state for more than one release cycle. If it does, it may be moved back into Draft.

Executing

Slash and burn! The breaking changes promised in the DROP-IT are being applied. This may include: deleting code, removing toggles, removing support for old versions, etc.

Execution Blocked

There is code-level consideration stopping the DROP-IT from being carried out. For example, the DROP-IT promised to remove feature X, but it turns out that feature X’s code is tangled up with feature Y’s code, where Y is something that needs to remain supported.

Executed

The DROP-IT changes have been applied, work is done!

Cancelled

The ticket is infeasible, no longer relevant, rejected based on the comments, or simply stalled due to a lack of interest or capacity. It may be re-opened again in the future, but it must start in Draft and move through Comment Period as if it were a new DROP-IT.

Could we just make all these changes to the DEPR process instead of inventing a new thing?

We sure could.I still think it’s confusing to “DEPR” every breaking change, but the most important thing is changing the states to have a clear grace period.