2023-02-21 DevExp Meetup Notes

Demos

@Kyle McCormick : removing github requirements and moving them to PyPI

• Working example: Blockstore. Pushed to PyPI with a GH action

  • In repo: 181 files

  • In PyPI zip: 60 files

• In Olive edx-platform, we have a list of libraries in github.in where we install the entire repo (as of Olive, there are 9 of these)

  • As of Olive, this includes the Blockstore

  • Developers don’t always know the benefits of PyPI, and in the middle of an upgrade it’s probably a waste. github.in is meant for temporary things

  • pip install requirements/edx/base.in on a container that already has everything installed (ie a noop): 55s, the VAST majority of the time being spent on Github cloning and running setup.py

• In the latest edx-platform (ie master)

  • only 3 GH libraries, two of which we expect to be removed shortly

  • pip install requirements/edx/base.in on a container that already has everything installed (ie a noop): 36s!

  • We may be able to get down to 25s when we remove everything

Q&A

[Becca] Q: Was the work mostly just copying over the GH action?

A: In some cases, yes. For Codejail, setup.py was missing a few modules so had to track those down

[Jeremy] Q: 1. Yay! Have we done investigation on wheels vs source distribution?

A: Not yet, but there’s a ticket for it.

[Jeremy] Note: At some point it may make a difference which PyPI mirror you’re using

[Kyle] Also, pruning edx-platform requirements will be another major time save

Cross Pollination

• Tools/features/opportunities

• Tips and tricks

• Roadmap news

• Your Name

  • Your Item to Share

  • Additional notes

• Jeremy

  • Devstack minor fixes

  • Renamed dev.down make target (Diana), fixed some documentation (Diana), fixing SQL init script (Becca)

    • Note from Becca: by “fixing” we mean “not including the demo course.” With the split module store, it becomes too tricky to keep everything in the initial sql without having references to mongo that won’t exist on a first provision

• Becca

  • Bring us your devstack issues! https://github.com/openedx/devstack/issues

    • Note from Mafer: I will communicate this to the eduNEXT team that still uses devstack

• Kyle

  • docs: add ADR: "Reimplement edx-platform static asset processing" by kdmccormick · Pull Request #31790 · openedx/edx-platform ADR up for review

Challenges

• Each challenge should have a follow up action.

• If you have an idea for follow up actions, add it.

• Or if you aren’t sure, leave it blank.

• We’ll discuss all actions either way.

• Your Name

  • Your Item of Challenge to Share

  • Idea for Follow Up Action

• Kyle

  • Lot of unknowns with devx work, which makes it hard to break our work into small issues that are good for newcomers

    • devstack work?

    • looking for docker optimizations

    • image layering

      • good for people who know docker but don’t necessarily have open edx knowledge

      • Mafer note: Maybe it's a good idea to rethink the order of the layers in docker or perhaps have names in the docker file to be able to build a specific part.

    • issue reportingDifference between core developers and majority of small-service users

      • small-instance users would strongly prefer not to have to build as often since that takes so much time. Needs to be easier to customize without a full build for every change

• Jeremy

  • Formulating a concise mental model of our development environment

    • What does our env look like? What is standard between repos? What is tied to the orchestration layer?

  • First stab at this is at https://docs.google.com/spreadsheets/d/1XAipznjIcTaE2H454M74No2Ro_9VftbolfV4YVv-B9I/edit#gid=0 (mentioned earlier), suspect we’ll need some diagrams before we’re done

  • On Arch BOM we’re going in and out of devstack mindset, hard to dedicate time to thinking through this

  • Should there be a hard line between what the individual repo provides vs what the environment provides?

    • this is changing with Tutor

      • One of the goals is to smooth out the differences between repos

    • some Makefile targets are common across repos, some common-ish. Would be nice to have a common interface across all repos

• Jeremy

  • Smoothing learning curves

  • Things like Kubernetes, docker-compose, Terraform, Ansible, etc. have a wide range of possible knowledge depth; find appropriate resources for each level

    • It’s hard to identify a minimum set of required knowledge

    • @Feanil Patel is working on a kind of minimum development guide, eg https://docs.openedx.org/en/latest/documentors/quickstarts/first_documentation_pr.html

    • Becca: Maybe 2u should be part of these quickstart documents? We could include in them “basic things you need to know”

    • Higher level personas: https://docs.openedx.org/en/latest/index.html

Successes

• DevExp wins!

• Your Name

  • Your Item of Success to Share

• Kyle

  • Bash rewrite of edx-platform assets build matches the output of paver implementation

• Ali and Saad of 2u have a working Ansible-less edx-platform on sandboxes!

Suggested Action Items

0-2 minutes:

Last Time:

This Time:

Look at the board

Developer Experience Working Group • openedx