2023-02-07 DevExp Meetup Notes

Demos

@Rebecca Graber (Deactivated) Kafka in Tutor

• archbom at 2U has spent much of the past year working on the event bus – more robust async way of passing messages between services

• 2u has chosen kafka as the implementation of the bus

• Demo: Plugin to add Kafka to Tutor

  • Example event: When something is changed in Studio, send an event to the Courese Discovery service

    • Event can be seen in Kafka dashboard

  • Plugin is pretty simple. Only for local dev now.

  • Currently on a branch: https://github.com/edx/devenv-wg/tree/rsgraber/kafka-plugin/plugins/contrib_kafka/tutor-contrib-kafka

  • Four services are patched in voa local dev docker compose: https://github.com/edx/devenv-wg/tree/rsgraber/kafka-plugin/plugins/contrib_kafka/tutor-contrib-kafka

• Ghassan: How does Discovery know about Kafka hostname?

  • Currently hard-coded into devstack.py for both services

  • In the future: could be templated into the plugin’s patches, via discovery-production-settings and the equivalent patch for edx-platform

Cross Pollination

• Tools/features/opportunities

• Tips and tricks

• Roadmap news

• Your Name

  • Your Item to Share

  • Additional notes

• Jeremy Bowman

  • https://docs.google.com/spreadsheets/d/1XAipznjIcTaE2H454M74No2Ro_9VftbolfV4YVv-B9I/edit#gid=0

  • Trying to figure out which aspects of “the development environment” are conventions for individual repos, which are tied to container orchestration, which are tied to a deployment solution, etc.

• Connor

  • No notable gains on tutor auto-bindmount plugin

• Rebecca (2u - ArchBOM)

  • Mostly been taken over by event bus cleanup, some discussion on hosted solutions (see Jeremy’s update)

• Mafer

  • Today, I will open a PR about Add CLI tool for listing patches available from installed plugins (issue 50)

  • This week I want to take a look at index PR here

• Kyle

  • Generally, I am trying to remove the local-code workflow pain points from Tutor:

    • This week, specifically, I think I’ve found a way to make the local package development experience less painful. Would love feedback:

• Ghassan

  • uses Tutor, tutor local not tutor dev, cloud-hosted using another computer. Heavy to run edx-platform locally.

  • if you want to make a new xblock, you can use the xblock-sdk. This was a good developer experience, rather than having to run the whole platform.

Challenges

• Each challenge should have a follow up action.

• If you have an idea for follow up actions, add it.

• Or if you aren’t sure, leave it blank.

• We’ll discuss all actions either way.

• Your Name

  • Your Item of Challenge to Share

  • Idea for Follow Up Action

• Jeremy Bowman

  • Anecdotally, a lot of people are dissatisfied with devstack. But we don’t have a good list of the problems for remediation or comparison with alternatives.

  • Define some location for capturing these with low barrier to participation. Do we just want to use devstack repo issues and curate from there, or would a Confluence page or such work better?

    • [added after] Collecting as repo issues:  devstack github repo 

  • Lots of discussion (see recording). Some notes:

    • general agreement with the sentiment

    • tutor is quicker out of the box

    • devstack takes longer to get working, but seems fine when you know what you’re doing

    • issues are different depending on what you’re working on

      • ex: becca needed libraries installed & certain microservices running. that was easier in devstack. But tutor might work better when you’re not working on those things

    • Ghassan: unlike tutor, devstack is less production-like. How can you be sure that your changes work once you’ve made them in devstack?

    • Jeremy: Operations and development are completely different when you’re using k8s at scale for prod and devstack/tutor for development.

• Rebecca

  • Time

    • +1

• Kyle

  • We do “sprints” in this working group. Does anyone find those helpful or nah?

  • I want to keep github-based requirements out of edx-platform except for specific circumstances (security & upgrades – both temporary). Curious if folks have any ideas. CODEOWNERS on github.in maybe?

• Jeremy Bowman

  • Insufficient teams to fully parallelize work on DevExp tasks that could be done in parallel.

  • Is there an opportunity for a BC/FC project here?

    • BC/FC = blended contribution / funded contribution = 2U / tcril contract work

    • Jeremy: Developer data is a potential blended project, or maybe BTR?

    • Kyle will be writing up a statement of work for upgrading frontend frameworks in edx-platform

Successes

• DevExp wins!

• Your Name

  • Your Item of Success to Share

• Kyle

  • Working with arbi-bom to destroy edx-platform’s github.in requirements file. Over halfway done. By the time we’re doing we’ll have removed 30+ seconds from the edx-platform build time, 60+ if you include the development variant of the image.

• Mafer

  • The proposal Multiple Tutor environments with TVM, has been accepted for the Conference.

    • TVM:

Suggested Action Items

0-2 minutes:

Last Time:

This Time:

Look at the board