DEPR Meeting Notes (2023)

Owned by Diana Huang
Last updated: Jan 11, 2024
20 min read

DEPR Project board

14 December 2023

  • [Jeremy] I think we’re clear to move ahead with deprecation/archiving of edx-analytics-pipeline

    • Actually, no, sometime in 2024 to get rid of the last usage…sigh

  • [Diana] Deprecation of insights

  • [Jeremy] MySQL 8! Is there anything 5.7-specific we can ditch?

    • We can drop the migration tests in GitHub because we don’t need them yet

    • No new LTS for MySQL yet

    • Hooray! We are just happy that we are now current and up to date!

    • Jeremy will ask Arbi-BOM to update the Support Windows doc

  • [Jeremy] APM integration deprecation/generalization

    • We should create a DEPR ticket for New Relic

      • be clear that we aren’t going to be adding a direct replacement

    • Trying to target OpenTelemetry as a starting point for this refactoring work

    • A chance to try to ensure that there are the proper integration points for orgs/operators to build out their own observability stacks

    • @Diana Huang to create the DEPR ticket for this work

  • [Diana] EdxRestApiClient code removal from edx-rest-api-client library will be

  • [Jeremy] edx-platform legacy UI removal status

    • Removal of the old UI code from edx-platform depends on some of this work

    • Keep finding corner cases where things stick around

16 November 2023

02 November 2023

 

Past Agendas

19 October 2023

  • Account and profile UI removal is still causing issues

  • How do we reduce duplication of effort?

    • GitHub issue has moved around

  • Quince release notes

05 October 2023

  • [Dave] Hide the Matlab key as part of the matlab deprecation work

  • XQueue deprecation

  • [Feanil] Bok Choy tests and references are going away in edx-platform

    • Badges also has a PR to remove it from edx-platform (yay)

    • xblock-utils got merged into xblock repo, so the separate repo will be archived and moved to unsupported

      • should be a simple migration, but does involve some work

  •  

21 September 2023

07 September 2023

  • We have a profile MFE. Should there be a DEPR ticket for removing the legacy page from edx-platform?

  • Askbot references still lingering: https://github.com/search?q=%28org%3Aopenedx%29+askbot&type=code

    • We can make a depr ticket for removing the comments and put it on the depr board as supporting work

  • We have a lot of staff grading XBlocks. Should we try consolidating them?

    • https://github.com/openedx/staff_graded-xblock

      • This was an XBlock living in the edx-platform codebase until 2019, was split out to be a separate repo then.

      • Installed by default in http://bundled.in

      • Can possibly be replaced by gradebook.

      • @Diana Huang MITx folks are not aware of anyone using this xBlock in our courses. We’d be fine with a deprecation announcement, but we would still want a coursegraph search on http://edx.org just in case.

    • https://github.com/openedx/edx-ora2

    • https://github.com/mitodl/edx-sga

      • One feature in this one that doesn’t exist in ora2 - instructors can submit a file in response to a student file.

      • eduNext was talking about adding more features, not sure if that happened

      • Installed by default in http://bundled.in

    • Can we get rid of at least one of these?

      • Talk to product about this.

      • staff_graded seems like the most likely to be removabled/replaced

        • Need to understand how many people are actually staff_graded this on http://edx.org .

  • The comment period for xQueue xQueue-watcher has passed with some requests for delaying the deprecation. What comes next?

  • Last Old Mongo removal PR of FC-0004 has merged! (barring regressions)

    • This just went to production

    • Old static assets sticking around for a while to not block credentials

    • … so I guess this is still sorta in-progress?

      • @Dave Ormsbee (Axim) : Mark as Done, but put details about the bits that are left and why, for future reference.

      • @Dave Ormsbee (Axim) : Make follow-on DEPR tickets for specific pieces.

  • Upcoming Axim-funded projects that will likely kick off new DEPRs eventually, but not yet:

    • Prototype for safer XBlock rendering in the LMS

    • Very tentative and not committed to: Migrate forums to Django + MySQL stack

      • @Dave Ormsbee (Axim) : Draft DEPR for Ruby service

24 August 2023

  • Follow up on xqueue

    • This is still being looked at within 2U to see how much is still remaining.

    • There’s a request for the removal to be delayed until Dec 2024 because it would involve rewriting problems

  • Discussion of toggles-related DEPR work

  • Discuss https://github.com/openedx/xblock-adventure/issues/56

    • Who from 2U can take on the work to investigate how this will impact http://edx.org ?

      • Someone on 2U support has already been contacted to do some investigation

    • Came up during some maintenance work. Another case of ‘we don’t want to have to do this if we don’t have to.'

  • Tutor is moving to MySQL 8.1

    • MySQL 8.1 is not an LTS release.

      • This did fix a bug in 8.0.x

    • Bumping versions on Tutor as soon as possible

      • http://edx.org is lagging behind, because of licensing issues, technical difficulties with the upgrades/migrations.

        • AGPL Licensing is not compatible with Mongo and Elasticsearch.

    • Concerns with falling out of sync between devstack/edx.org and Tutor and ensuring compatibility.

  • https://github.com/openedx/edx-platform/issues/32343

    • Communicate with the 2U’s website team about this

  • Discussion of how to handle non-depr-ticket tickets on the DEPR board

    • Still need to track as they are deprecation work, just not specifically needing to be announced.

    • Use a label/field to mark as ‘depr-supporting' work.

10 August 2023

  • [Diana] xqueue + xqueue-watcher ticket

    • https://github.com/openedx/public-engineering/issues/214

    • We need someone at 2U to do a coursegraph search to find the existing usages of xqueue on http://edx.org .

    • Aiming for Quince, but MIT will need to do work to migrate existing problems onto the alternatives.

    • We will need to talk with partners who are also using xqueue to find out what technologies they are using. Are they using xqueue-watcher?

    • Can we do NewRelic searches to find usages?

      • we did do this and found a good list seen above

      • and Splunk

      • and the Mongo DB

  • Assigning tickets via comments? can we find docs around that?

  • Follow up on deadcode discussion:

    • Maybe we should do the 100% confidence ones and see if we find anything that is a false positive in the resulting PR.

  • Follow up on moving stuff from openedx org to the edx org

    • Still have to talk to lawyers about what it looks like for both 2U and AXIM.

27 July 2023

  • [Diana] Follow up on deprecating devstack/edx-configuration

    • Set the dates further out, about 6 months

    • Seems like there is still some use here that we need to be aware of and careful about

    • Next steps? Draft up a proposal about what this transfer of ownership would look like? Figure out how to pull more people into the discussion?

      • We do need to figure out what the lawyers have to say about the transfer as well.

        • Next step: get feedback from AXIM’s lawyers on the implications of this- to be done by @Diana Huang

      • Initial proposal is to just move from one org to another, keeping the repositories public and available for PRs.

  • [Diana] Take a look at deadcode and vulture results on edx-platform:

  • [Diana] We’ve been able to clean up some more small DEPR tickets thanks to Yagnesh!

  • [Robert] [inform] I’ve been implementing 2U-side of https://github.com/openedx/public-engineering/issues/190, and ran into a minor issue where I may take on edx-configuration.

  • [Peter] XQueue, XQueue Watcher

    • Can we get a ticket created?

      • Do we need to get 2U legal involved?

    • We can get a draft written up for the ticket.

    • LTI is the best alternative for this and should be the replacement for this code.

13 July 2023

  • [Robert] [inform/question] I noticed that I had referred to devstack as deprecated, and have discussed edx-configuration as deprecated, but neither of those things are true because there are only draft/proposed DEPRs.

    • Can we do better at moving DEPRs past Proposed so they are either Accepted or Rejected?

      • This means that folks need to go through the process of announcing and gathering feedback.

      • What does this mean for cognitive load?

    • Can we do better at not referring to things as deprecated when still in “Proposed”?

      • Can we say that Proposed is just a proposal and that we don’t consider something deprecated until it reaches ‘Accepted’? This is especially useful for making sure people feel comfortable giving feedback in the ‘Announced’ stage, because it means their feedback can sway things away from being deprecated.

    • Does the term “Proposed” sound too official, and would “Draft” change anything here? To me, “Proposed” sounds more official than what the actual status is, especially because the ticket could be no more than a placeholder title at that point.

    • [inform] I may miss this meeting. Feel free to discuss this topic without me, and maybe see which parts have lots of agreement and which do not.

    • 2U is the only part of the community that uses these even. Should we move them into 2U’s org and then just let the community move on?

      • This needs a set of processes and announcements to ensure that this happens.

  • [Diana] Review results of deadcode:

    • https://github.com/openedx/edx-platform/issues/32603

    • initial impression: this seems a little too noisy to be useful

    • Could we find django-specific filters/plugins for this library to help filter out some of the noise?

    • We need to do a better job of filtering out tests, because that adds to the noise.

    • Maybe also exclude settings since we’re hopefully tackling that with toggles work.

    • [Diana] [Action item] I’m going to leave a comment on the ticket with the feedback.

  • [Diana] Still need to schedule meeting around frontends

  • How do we want to ‘deprecate’ things in Open edX that only 2U uses?

    • Maybe we just announce that these things get moved to the edx organization.

    • What are the gaps that still remain for non-2U developers and Tutor?

      • MFE development

      • [Action Item][Diana] Announce the deprecation of devstack.

        • Make it clear that this is going to be moved only.

    • Can we just move configuration?

      • Just make the announcement for it with the caveat that we will be moving it into the edx organization vs. into unsupported.

      • This shouldn’t have as many blockers from the community.

    • Is this the best way to go about doing it?

    • [Action Item][Diana] Announce configuration deprecation.

      • update the ticket to make it clearer that this is 2U-owned.

  •  

29 June 2023

15 June 2023

1 June 2023

18 May 2023

  • xqueue & xqueue-watcher - time to start deprecating?

    • MIT says they no longer use it

    • Owned by Aurora, Mat says the number of users is small (but he doesn’t have a list handy)

    • May need some sort of fix for legacy certificates

    • Let’s make a new DEPR ticket for xqueue and xqueue-watcher.

      • New event bus Redis/Kafka are meant to be the replacement.

      • Ask Aurora to write up DEPR ticket.

        • This may also involve some edx-platform changes to remove xqueue code within capa or possibly at a higher level.

  • Cleaned out a bunch of unused Jenkins code

  • Checklist for Deprecations: what should be on it?

    • Useful as a related example? Upgrade Project Runbook | Appendix: Roadmap Issue Checklist

    • Can add this to the DEPR template

    • Some things to add to the checklist:

      • Announcing to Discourse.

      • Specify Open edX release (if known) on the ticket.

      • Name a team (of people) to coordinate the deprecation.

      • Verify that all the code has been removed.

        • Examples for how to check (create new documentation for this under DEPR HOW-TOs ):

          • Create a repo health check to verify that the thing/code has been removed.

          • Search for remaining instances of code on GitHub.

          • Linting checks

        • Do checks for contagion/new usages of the thing we’re trying to get rid of.

    • Diana to take on this work.

    • Look at RCAs of deprecation issues.

  • Made a DEPR for Matlab code in edx-platform

    • Just waiting for the acceptance date to arrive

    • Removes another usage of xqueue!

    • Check in with 2U about the usage internally (this is pretty broken, so we shouldn’t be).

  • Timeline for removing support/handling for MySQL 5.7?

  • Tackling frontend deprecation

    • Still need to ask FED WG to write up an OEP about their future vision of frontend.

      • MFE domains, Paragon

      • Product vision - don’t want to do a straight port of instructor dashboard

      • Making sure that our OEPs are updated to clarify how the frontend code

      • What should we do in situations when folks are stuck working in legacy frontend code?

    • How do we want to start some of these projects?

      • Schedule a meeting for both FED and DEPR to talk about how to make progress on this.

    • Set an agenda for it around scoping a couple different projects: support page (how to get help) - small, and instructor dashboard - large.

    • Diana to work with Adolfo on scheduling this.

04 May 2023

  • Legacy Frontend Next Steps

  • safe_lxml

    • The question is whether or not we wanted to keep maintaining our own solution with all the security risk and maintenance complexity that implies.

    • It seems like there’s still no real good solution that we could use instead.

  • Velocity of deprecations

    • We could try to graph how much we’ve managed to deprecate and remove over time and see if we could find blockers.

    • Could we use GitHub issue reports to track this stuff? - investigate new reports and features.

      • There are historical charts that we could take a look at, see if there’s something.

  • Elasticsearch Project: https://github.com/openedx/public-engineering/issues/16

    • 2U is the major blocker on this.

      • Arbi-BOM did do some discovery on this, but they aren’t equipped to do the migration for one team.

    • How urgent is this? What is the community doing?

    • Can we ask BTR about it?

    • Algolia usage

      • Can we make sure Algolia integration code is separated into plugins?

  • Deprecation Checklist:

    • What could we do to prevent issues like the one we saw with the EdxRestApiClient and remaining code still using it?

    • Maybe new API client work can be worked into auth rethinking work.

20 April 2023

  • Conference happened!

    • Some notes on the discussion: LMS Endpoints Rendered By Django | Process/Next Steps Suggestions

    • Different modes of open source conference presentation:

      • Open edX platform does work in different modes across the platform

      • Different repos could work under different modes

      • Does a single DEPR process make sense for all our different repos?

        • Maybe we could do different timelines for different rings around the core product

        • Waiting on product working group’s work on defining the core product

  • Legacy frontends: what’s next?

    • Lots of energy around this.

    • Follow up with the FED WG about next steps.

    • Keep checking in with Frontend WG about what is going on.

    • Not mechanical enough.

      • How to make it easier to turn our existing Django frontends into APIs that can be used?

    • Auth cleanup is a blocker here. Need to make it easier to authenticate so we can make it easier to access REST APIs.

      • AXIM is going to review auth cleanup and whether or not we can make a funded contribution to do this work.

  • OpenCraft’s BD-13 Modulestore cleanup work is culminating with:

    • https://github.com/openedx/edx-platform/pull/31472 (approved and scheduled to merge next week)

    • https://github.com/openedx/edx-platform/pull/31492 (to follow after the previous one lands)

    • Shim that logs deprecation warnings

      • 2U/edx.org should probably have monitoring for this.

      • Put an inform on the DEPR ticket to talk about the shimmed attributes on the system that folks should stop using (that they never should have used)

    • Makes it easier to implement new storage backends.

    • May help clean up memory leaks that have been plaguing Open edX for a while.

  • Major Old Mongo removal PR (this is the lobotomy PR)

    • https://github.com/openedx/edx-platform/pull/31134 (approved, scheduled to merge)

    • Returns an empty list when courses are asked for children.

    • Mainly being kept around to support CourseOverview(s) and how those are regenerated periodically and otherwise anything else that tries to use the Course object instead of CourseOverview.

  • [Robert] Attempted to disable symmetric JWTs for my hackathon, but backed it out. I have ticketed new follow-up work. See https://github.com/openedx/public-engineering/issues/83.

  • Palm branches cut, go through board to figure out release notes

  • [Dave] Going to make an SOW for work to move course data off of MongoDB and into (still under construction) Learning Core data models.

  • Discussion of maintainer work to remove temporary waffle flag things

    • Kyle is taking on edx-platform maintainership.

  • [Feanil] Sphinx theme deprecation moving along

    • Will hopefully be done in the next few weeks.

 

23 March 2023

  • Open edX Conference (3rd time around)

    • Teaming up with Frontend WG to make a plan for legacy frontends!

      • How are we doing this?

      • Which are the frontends that we want to deprecate?

        • “current” status: [Archive] MFE Rewrite Statusarchived

        • in edx-platform, which Django views still exist?

          • 2U can look at which views are used with which frequency.

          • Diana to try to prepare this for the conference.

        • Copy vs. Redesign on views

          • Bring up as an idea

        • For each view, come up with a list of questions around the view.

          • What do we want to make the process to be?

            • Come up with all the steps that need to happen for a replacement to happen.

            • Find the open questions around.

          • What are the acceptance criteria for each part?

      • Focus on specifying the deprecation process.

      • Agenda:

        • Intro to the session

        • Bring up list of existing Django views in edx-platform.

          • Backbone views should show up in this list as well.

        • Ask for opinions on which ones are the most important to remove.

        • Come up with a process as a group of how to deprecate these frontends.

  • Account settings pages PR update

    • PR was reverted due to unforeseen dependencies

  • State of the WG update

  • Software Engineering at Google Deprecation Chapter

    • Advisory vs. Compulsory Deprecation

      • Can we sequence deprecations to ensure a decent deprecation WIP?

      • Can we turn on certain warnings for everyone everywhere vs. turning on all the warnings for everyone everywhere?

      • Python Deprecation Best Practices

        • We should write up a good HowTo on this and/or add to the existing OEP.

        • Can we extend DeprecationWarning to allow for more fine-grained control over them.

          • Use something in edx-django-utils to expose them correctly.

    • Tooling? - alerting people when they’re writing code that they’re using deprecated components/libraries

      • Static Analysis

        • pyupgrade or django-upgrade? Can we make use of these tools?

    • Process Owners (ha ha ha)

      • can we dedicate resources towards getting things deprecated?

    • Intermediate Milestones

  • Shrinking the size of edx-platform requirements

    • removing 2U-specific requirements

    • this needs to be handled properly on http://edx.org with something like pip-compile

    • can we rid of other dependencies?

      • scipy is a big one, mostly used by codejail

      • which modules get exported

      • repo-health-dashboard work to keep track of dependency trees

  • Django 4.2 RC is out, so we should be preparing to have deprecation warnings and upgrade things.

    • django-storages seems like the biggest thing that requires care and concern in migration.

  • Removing boto from edx-platform - making progress!

    • video uploading

    • bulk email

  • Tension between putting things on the board that we are planning on working vs. putting things on the board that we want to work on and not lose track of.

    • Can we set something up in GitHub projects?

      • Add a sponsor field

      • Use assignees more seriously and ensure they are up to date and things without assignees aren’t in.

    • Update the OEP to account for these new processes.

  • How can we move things along the board?

    • Use board review time to prioritize the deprecation work.

    • Try to figure out when things end up on the maintenance board for 2U ownership review.

09 March 2023

  • Old Mongo

    • PR coming in to prevent the updates and usage of Old Mongo

    • Static assets are one of the last things

      • certificate signature are stored as static assets

  • Modulestore cleanup

    • renaming things and terminology usage

  • State of the WG update

23 February 2023

  • Open edX Conference (again)

    • Interesting things to do with our time

      • Can we collect feedback from community on other things they are interested in removing/excited about

        • Nominate-a-thon - gather a long list of interests and filter through them later

      • Get in on the Data WG’s ice cream order

    • What sort of a pinata we should do?

      • Raccoons

      • Leafy Vegetables

      • Giant X

  • Kyle’s continued work with pavelib

    • About half of it was already replaced/replicated with Makefile/shell scripts

    • Some cleanup already happening (yay!)

    • Toughest part of this is JS tests and Assets

      • Want to convert JS tests from Karma to Jest anyway, may be a good time to throw out the JS test pavelib code.

  • Also work to get Python typechecking working

    • Some push on Typescript as well

  • MFE tickets for progress page and learner dashboard

    • check in on the status of these MFEs to see if they are mature enough yet.

  • How do we get momentum on the stuff that is already on the board?

    • Can we specify different levels of ownership?

      • Who is pushing for this?

      • Who is responsible for the work?

      • Who owns the code that this work is in?

    • Would like to get this done before a bunch of upgrades.

    • We can do more FCs/Blended, but there is a concern about getting code merged.

      • Can we involve teams earlier?

    • Updates to the DEPR board:

      • can do a workflow that can track how long a DEPR ticket has remained in the same state?

      • custom fields to track levels of ownership to clarify what needs to happen on each thing

        • Code owner tracking

      • Put things onto the Maintenance Board

        • When we review items on the board, move things from Removing into

09 February 2023

  • Cross-posting should now be enabled from Discourse to Slack

  • Open edX Conference

    • Anything we want to share specifically?

      • Updates from the RG FC project

        • Getting rid of EdxRestApiClient

        • Removing old course views in favor of MFE

        • Getting rid of references to PDF certs

        • Persistent Course Grade flags and corresponding code

      • Upcoming work?

        • Getting rid of paver

        • Draft Modulestore removal

        • Doing some real work to remove legacy frontends in favor of MFEs

  • Ticketing up replacement of legacy front end code with MFEs

  • Update from Kyle: Mostly gotten rid of paver asset functionality in favor of bash scripts.

 

26 January 2023

  • FYI: We got rid of about 1,200 lines of paver code in edx-platform: https://github.com/openedx/edx-platform/pull/31180

    • We are aiming to get rid of all of our custom paver tooling!

    • Trying to remove all the testing stuff first as we target the frontend specific code as well.

    • Trying to rewrite the asset management code into sh, ADR incoming

  • Getting close to removing https://github.com/openedx/edx-platform/pull/30336

  • Will be removing/archiving/moving to unsupported edx-e2e-tests today

  • Looking for a review: https://github.com/openedx/edx-platform/pull/31633

    • toggles - can we put into the process that toggles should only last for a specific amount of time?

    • part of the work of maintainers going forward?

    • figure out a process for handling this

      • can we make toggles temporary?

  • edx-sphinx-theme DEPR Update

  • Moving things from Accepted to Removed

    • Adding a ticket to the Maintenance GitHub Project

    • If it impacts multiple repos, put it on the Maintenance Board and assign a coordinator to help put it on the owning teams.

    • How does this work compared to upgrades?

      • We can assign deadlines.

      • Need to force team to make a decision about these DEPR tickets. Can’t leave them on the board forever.

  • Accepted vs. Deprecated

    • Can we move more things into ‘deprecated,' even if we can’t get things into the removed column?

    • Try it out with some of the auth DEPR tickets that we aren’t.

  • Can we make use of the Discourse/Discuss-Slack integration to automatically announce things to the correct Slack channel?

    • Create a Github tCRIL request for enabling this work.

12 January 2023

  • edx-e2e-tests

    • using DEPR process for repos in edx vs openedx

    • still going to carry on as usual. make sure to announce things earlier rather than later.

  • Maintenance GitHub Project

    • Does it make sense to add at least some DEPR issues to this board?

  • Repo health dashboard

    • Do any DEPR-related checks make sense?

      • ex: # of DEPR tickets per repo

      • ex: bok-choy dependency - usage

        • maybe even going as far as making generic checks looking for specific packages

  • CFP for Open edX Conference

    • DEPR topics welcome

    • upgrade template discussion

      • conference driven development

      • what can we do to deprecate things before we have to upgrade it?

  • Review board

    • bok_choy: we still have some lingering tendrils in various places, but we don’t have any current work or resources yet.

      • do we have mechanisms into incentivizing cleaning out some old code or doing small maintenance work?

    • OpenSearch vs. ES work - who can lead this effort? can the Arbi-BOM team do so? Can the community take this on?

    • Courseware pages - can we monitor usage of the old views on http://edx.org ? does a method (like a custom attribute) already exist?

  • Proposal for future DEPR:

    • Reorganize edx-platform configuration variables

      • base things on OEP-45

    • Simplify the configuration overall and the various settings files.

    • Suggested changes:

      • get rid of the FEATURES dictionary

      • simplify Celery configuration

    • what is the minimal thing we can get to run a version of edx-platform?