...

2021-11-03

• ++[analysis] (Dave O): Extracting a low-level learning core out of edx-platform and into a new repo.

  • (Original post/thread).

  • Motivation:

    • Create a smaller/simpler dependency to build extensions on top of (instead of edx-platform). This means smaller, more stable APIs that can add incremental value, instead of the backloaded benefits of removing stuff from edx-platform.

    • Promote innovation by making it easier to create different experiences on top of Open edX (like LabXchange).

    • Advance the Studio/LMS split (these would be the core of the LMS).

  • Some potential apps: publishing, navigation, policy, composition (what’s in a Unit for this user?), scheduling, partitioning (what users are in what groups for various tests).

  • Strategy for dealing with tricky extractions: Core data models and frameworks exist in new repo. Plugins are implemented in edx-platform. Examples:

    • For navigation, an outline processor framework exist in new repo, but an EnrollmentOutlineProcessor exists in edx-platform, keeping knowledge of enrollments out of the new core.

    • For partitioning, the data models to store user/partition mappings live in the new core, but actual partition bucketing logic remains implemented in edx-platform.

  • Strategy for data migration: Start with content data, that can be rebuilt/backfilled from Modulestore. We do this kind of thing all the time already.

  • Feedback:

    • Robert: In general, like the idea. Where to start: serving particular needs, like course overview-like data, verify that it serves the needs. Interfaces vs. implementations: do we need both? E.g. for course overviews, does it just need to move or will work need to be done–define an interface for a mocked version?

    • Jeremy: Any idea of things currently installed in edx-platform that directly call edx-platform–what are they doing?

      • Course overviews

      • Things that reach into modulestore for lack of better APIs that we should probably create

      • Scheduling

    • Jeremy: might it be worth identifying APIs in edx-platform that are the main code called by many other parts of the platform, so extracting it could allow all those to also be extracted?

      • There may be some cases where we move enough of the core of such an app (models, etc.) to a separate repo, but leave all the tangled implementation details in place to minimize the up-front work needed.

• + [quest] (Jeremy) Docker Desktop changes - how does this influence build vs. buy decisions?

  • Docker is starting to charge for Docker Desktop usage for orgs like ours

  • There are possible alternatives like Minikube, but we haven’t really evaluated how well they would work for us

  • Quite possibly worth edX paying for this, not as clear for Open edX at large

  • Jeremy discussed this with Régis; we agreed it needs to be discussed/resolved, but didn’t come up with any immediate answers

  • BTR working group at large hasn’t discussed this yet

  • Does moving to Codespaces or something similar change this

  • Costs ~$21/user/month for 50+ user orgs

  • Open edX will probably support multiple options in the future, but edX is likely to pick a default for its own developers

  • Probably just paying the license fee for now, to free up resources for acquisition-related stuff

•  Has anyone tried Tutor or Codespaces?

  • Tutor: https://docs.tutor.overhang.io/run.html , written by Régis Behmo

  • Codespaces, etc.: sounds cool, nobody’s had time to look at it very closely

    • Quicker to get started (and to restart!)

    • Internet speed: are codespaces better or worse than downloading docker images for poor internet speed?

      • Codespaces are latency, docker images are bandwidth

    • Doesn’t work at all offline, but that may not really be an issue for most people these days

2021-10-06

• [discuss] [Jeremy] mypy: where are we, where are we going

  • We’re running this for edx-platform in an optional GitHub Action check

  • We don’t have many annotations yet

  • There are tools that would let us add a lot pretty quickly by analyzing our test suite, etc.

    • But we probably won’t do this just yet, maybe in a couple of months

  • Feanil is somewhat excited about this as a way to catch potential problems early

  • Available for people who are excited about it, but any broader push for adoption will wait for at least a couple of months for other projects to settle down

  • It’s a tool to enforce best practices on interfaces, but we haven’t necessarily declared such best practices yet

  • If you find a good starting guide, consider adding to Learning Resources  

• [quest](Feanil) What are people’s expectations of arch-hour?

  • Awkward silence

  • Updates on what other people are concerned about or coming down the pipeline

  • Figure out how architecture is handled at edX

  • Cross-team interaction

  • Insights into engineering task prioritization

  • Feedback on what architectural challenges people are facing

• [quick question (hopefully)]: Is anyone familiar with the QTI spec? As in worked with it enough for me to ask some questions regarding structure/capabilities?

  • Not really, it seems

• [inform] (Dave O) https://github.com/fanout/django-grip looks like a potentially useful alternative to channels

• [question] (Jeremy B) Does anybody feel it’s worth investigating Django alternatives like FastAPI yet?

  • (Dave O) Feels like the data layer is the bigger performance problem

• [discuss] (Dave O) Possible next steps to resolve database performance issues

  • It’s hard to optimize this locally without awareness of the overall context of a given request

  • We may need to more often create custom APIs rather than extending existing ones with new (possibly performance impacting) data

  • Monitoring is key, especially to catch regressions

  • People don’t have a good sense on thresholds for action required to improve performance

...