2015.09.23 - IDA discussion

Action items

Notes

in-room: Mark, Ned, Nimisha, Peter Pinch, Joel, Steve, Clinton, Matt, Miki, Ed

hangout: Dave O, Felipe, Jim, Cale

• What are things that are IDAs now?

  • edx-platform

    • LMS

    • Studio

  • comments service (aka forums)

  • insights

  • analytics api

  • ecommerce

  • student notes

  • notifier

  • xqueue

  • Drupal

• What is not?

  • ORA

  • submissions

  • edX search

  • milestones

  • proctoring

  • teams

• What should be an IDA but is not yet?

  • LMS

  • Studio

  • Search

  • Certificates

  • Student state storage

  • Users and Auth

  • XBlocks

  • Modulestore

  • Veda

• Criteria for IDAs

  • Scale

  • Security

  • Group ownership, decoupling

  • Replaceability

  • Barrier to entry

  • Deployment

  • Isolation of failure

  • Velocity

• Non-criteria

  • Don't be too granular

  • Doesn't have it's own data store?

    • why isn't that a library?

• Be careful:

  • No need to introduce network hops just to get decoupling

• What is "too many" about services?

  • operational complexity

  • need infrastructure and tooling

  • performance implications of network hops

• Independently Deployable Libraries?

  • make stuff pip-installable, then release versions of your library.

  • we can also host our own pypi

  • people need to review all their requirements on a regular basis

• Open edX

  • How do we help adopters with the complexity of micro-services?

  • Create a shim so you inline IDAs.

    • This is a common idea

    • but doesn't seem to exist in the Python world

  • Having both local and remote is extra complexity,

    • we don't want to have to test both/all configurations all the time

• Many of our IDAs now are optional

  • Once we have a user service IDA, then everyone needs to use it.

  • can we have default implementations as alternatives to what edx.org uses?

• Programs in particular

  • there's no point in programs without courses.

  • but there's no point in insights without courses, either.

• Seems like there are more IDAs than we thought (more than 10)

  • Each new IDA should be discussed with the Arch Council

• We have a spectrum of IDAs

  • think xqueue vs insights

• Do we have incentives in place to split apart LMS?

  • new stuff is definitely separate

  • splitting lms and cms is in plan for this year

    • it's more than just moving code

    • it also means different data stores

    • it also means one-to-many studio/lms

  • putting interfaces into edx-platform could start cleaning up edx-platform (see Extending edX Platform)

• There are simple things we can do:

  • Move XBlock and other pluggable requirements out of edx-platform, so we can update XBlocks without pushing edx-platform

  • Splitting common functionality into libraries

  • Pluggable interfaces give good isolation

• Extracting existing code into a new library is a big job.

  • do we need separate projects to do clean up?

  • or do we budget it as part of the projects that need the separation?

  • we need patterns established

  • do we feel too much time pressure to do the clean up?

  • it's hard to estimate the clean up work

    • can we time-box it?

  • we aren't in a position to quantify the impact of debt on velocity

  • product has been reasonable about giving time where eng needed to clean up

  • engineers need to be able to market their changes

• Follow-ons:

  • Operations technology that can help with this?

    • troubleshooting IDA constellations

    • will need coordination technology that can run locally and remotely

    • Clinton is working on templates and patterns for IDAs

  • Can deploys be just a pip install?

    • no one is working on this