2023-02-21 DevExp Meetup Notes

Feb 21, 2023


Please be advised: DevExp Working Group meetings are recorded.

Link: https://tcril-org.zoom.us/rec/share/Cpjc5H0-m5zCQvsmEciXcmgW4O4sW7GZxcfX-i1c1xrBRdVLvbfxMGfAHIZ03DAv.3OnQsJx_O1SXinLB


Meeting captain: @Kyle McCormick

Notetaker: @Rebecca Graber (Deactivated)

Discussion topics






@Kyle McCormick : removing github requirements and moving them to PyPI

  • Working example: Blockstore. Pushed to PyPI with a GH action

    • In repo: 181 files

    • In PyPI zip: 60 files

  • In Olive edx-platform, we have a list of libraries in github.in where we install the entire repo (as of Olive, there are 9 of these)

    • As of Olive, this includes the Blockstore

    • Developers don’t always know the benefits of PyPI, and in the middle of an upgrade it’s probably a waste. github.in is meant for temporary things

    • pip install requirements/edx/base.in on a container that already has everything installed (ie a noop): 55s, the VAST majority of the time being spent on Github cloning and running setup.py

  • In the latest edx-platform (ie master)

    • only 3 GH libraries, two of which we expect to be removed shortly

    • pip install requirements/edx/base.in on a container that already has everything installed (ie a noop): 36s!

    • We may be able to get down to 25s when we remove everything


[Becca] Q: Was the work mostly just copying over the GH action?

A: In some cases, yes. For Codejail, setup.py was missing a few modules so had to track those down

[Jeremy] Q: 1. Yay! Have we done investigation on wheels vs source distribution?

A: Not yet, but there’s a ticket for it.

[Jeremy] Note: At some point it may make a difference which PyPI mirror you’re using

[Kyle] Also, pruning edx-platform requirements will be another major time save

Cross Pollination

  • Tools/features/opportunities

  • Tips and tricks

  • Roadmap news


  • Each challenge should have a follow up action.

  • If you have an idea for follow up actions, add it.

  • Or if you aren’t sure, leave it blank.

  • We’ll discuss all actions either way.

  • Your Name

    • Your Item of Challenge to Share

    • Idea for Follow Up Action

  • Kyle

    • Lot of unknowns with devx work, which makes it hard to break our work into small issues that are good for newcomers

      • devstack work?

      • looking for docker optimizations

      • image layering

        • good for people who know docker but don’t necessarily have open edx knowledge

        • Mafer note: Maybe it's a good idea to rethink the order of the layers in docker or perhaps have names in the docker file to be able to build a specific part.

      • issue reportingDifference between core developers and majority of small-service users

        • small-instance users would strongly prefer not to have to build as often since that takes so much time. Needs to be easier to customize without a full build for every change

  • Jeremy

    • Formulating a concise mental model of our development environment

      • What does our env look like? What is standard between repos? What is tied to the orchestration layer?

    • First stab at this is at https://docs.google.com/spreadsheets/d/1XAipznjIcTaE2H454M74No2Ro_9VftbolfV4YVv-B9I/edit#gid=0 (mentioned earlier), suspect we’ll need some diagrams before we’re done

    • On Arch BOM we’re going in and out of devstack mindset, hard to dedicate time to thinking through this

    • Should there be a hard line between what the individual repo provides vs what the environment provides?

      • this is changing with Tutor

        • One of the goals is to smooth out the differences between repos

      • some Makefile targets are common across repos, some common-ish. Would be nice to have a common interface across all repos

  • Jeremy

    • Smoothing learning curves

    • Things like Kubernetes, docker-compose, Terraform, Ansible, etc. have a wide range of possible knowledge depth; find appropriate resources for each level

      • It’s hard to identify a minimum set of required knowledge

      • @Feanil Patel is working on a kind of minimum development guide, eg

      • Becca: Maybe 2u should be part of these quickstart documents? We could include in them “basic things you need to know”

      • Higher level personas:

Meeting captain: Transpose action items into the “Action Item” section below.


  • DevExp wins!


  • Your Name

    • Your Item of Success to Share

  • Kyle

    • Bash rewrite of edx-platform assets build matches the output of paver implementation

  • Ali and Saad of 2u have a working Ansible-less edx-platform on sandboxes!

Suggested Action Items

0-2 minutes:

Last Time:

@Rebecca Graber (Deactivated) share dev feature doc more widely
@Rebecca Graber (Deactivated) and @Jeremy Bowman (Deactivated) share devstack pain points doc/process, wherever it ends up living
Note: ^ are now to be created as issues in the devstack repo
@Kyle McCormick Will write a TEP for local package workflow
(been focused on static assets, haven’t had time to do this yet)

This Time:

Everyone: Think about a demo you might want to do!
@Maria Fernanda Magallanes Z will demo Patch Listing CLI next time


Meeting captain: Move any remaining action item’s to next week’s minutes.

Look at the board