2015.09.23 - IDA discussion

Owned by Joel Barciauskas (Deactivated)
Last updated: Jan 04, 2016Version comment

Action items

Notes

in-room: Mark, Ned, Nimisha, Peter Pinch, Joel, Steve, Clinton, Matt, Miki, Ed
hangout: Dave O, Felipe, Jim, Cale
  • What are things that are IDAs now?
    • edx-platform
      • LMS
      • Studio
    • comments service (aka forums)
    • insights
    • analytics api
    • ecommerce
    • student notes
    • notifier
    • xqueue
    • Drupal
  • What is not?
    • ORA
    • submissions
    • edX search
    • milestones
    • proctoring
    • teams
  • What should be an IDA but is not yet?
    • LMS
    • Studio
    • Search
    • Certificates
    • Student state storage
    • Users and Auth
    • XBlocks
    • Modulestore
    • Veda
  • Criteria for IDAs
    • Scale
    • Security
    • Group ownership, decoupling
    • Replaceability
    • Barrier to entry
    • Deployment
    • Isolation of failure
    • Velocity
  • Non-criteria
    • Don't be too granular
    • Doesn't have it's own data store?
      • why isn't that a library?
  • Be careful:
    • No need to introduce network hops just to get decoupling
  • What is "too many" about services?
    • operational complexity
    • need infrastructure and tooling
    • performance implications of network hops
  • Independently Deployable Libraries?
    • make stuff pip-installable, then release versions of your library.
    • we can also host our own pypi
    • people need to review all their requirements on a regular basis
  • Open edX
    • How do we help adopters with the complexity of micro-services?
    • Create a shim so you inline IDAs.
      • This is a common idea
      • but doesn't seem to exist in the Python world
    • Having both local and remote is extra complexity,
      • we don't want to have to test both/all configurations all the time
  • Many of our IDAs now are optional
    • Once we have a user service IDA, then everyone needs to use it.
    • can we have default implementations as alternatives to what edx.org uses?
  • Programs in particular
    • there's no point in programs without courses.
    • but there's no point in insights without courses, either.
  • Seems like there are more IDAs than we thought (more than 10)
    • Each new IDA should be discussed with the Arch Council
  • We have a spectrum of IDAs
    • think xqueue vs insights
  • Do we have incentives in place to split apart LMS?
    • new stuff is definitely separate
    • splitting lms and cms is in plan for this year
      • it's more than just moving code
      • it also means different data stores
      • it also means one-to-many studio/lms
    • putting interfaces into edx-platform could start cleaning up edx-platform (see Extending edX Platform)
  • There are simple things we can do:
    • Move XBlock and other pluggable requirements out of edx-platform, so we can update XBlocks without pushing edx-platform
    • Splitting common functionality into libraries
    • Pluggable interfaces give good isolation
  • Extracting existing code into a new library is a big job.
    • do we need separate projects to do clean up?
    • or do we budget it as part of the projects that need the separation?
    • we need patterns established
    • do we feel too much time pressure to do the clean up?
    • it's hard to estimate the clean up work
      • can we time-box it?
    • we aren't in a position to quantify the impact of debt on velocity
    • product has been reasonable about giving time where eng needed to clean up
    • engineers need to be able to market their changes
  • Follow-ons:
    • Operations technology that can help with this?
      • troubleshooting IDA constellations
      • will need coordination technology that can run locally and remotely
      • Clinton is working on templates and patterns for IDAs
    • Can deploys be just a pip install?
      • no one is working on this