2015.01.13 E-Commerce Architecture Review

Presenter: Will Daly

Document: https://docs.google.com/a/edx.org/document/d/1qBeZG8Z-qqeHdTzQlS5e2-s6iuui7OEnzGZLzFp45W4

Attendees: Ned, Will, Steve Sanchez, Miki, Beth, Piotr, Dave O., Jason Bau, Steve Magoun, Nimisha, Cale, Ed

Will walks us through slides.

Goals next quarter:

  • Enable e-commerce features that we could build ourselves but would be one-off work, so we want to be able to get them elsewhere
    • flexible product catalog
    • product bundles
  • Independent deployment of e-commerce
    • a/b tests done quickly, optimize the pipeline
    • (miki) could also reduce the time to run tests

An open-source shopping cart implementation, more features than we have now

  • backend APIs for products, orders, discounts, etc.
  • more payment processors
  • provides an admin UI
  • NOT using the off-the-shelf user UI
    • whitelabel will need it
    • assuming the worst, and will need to a/b test the cart experience itself.
  • looking at existing open-source cart implementations
  • want to be good open-source citizens, will contribute back

Services

  • E-commerce front-end
    • UI for payment and verification
    • eventually, UI for course discovery and student dashboard
    • communicates with cart and LMS through well-defined APIs
    • need to support whitelabel, and open-source theming.
    • (cale) wasn't part of the point to avoid the complexity of whitelabel and theming?
      • (will) maybe we'll start with separation, and then add back in theming, but where we can, build in the hooks as we go.
      • for Q3, our focus will be on the velocity of the destedx team.
      • (steve m) can't compromise the cart in white label, it needs to work all the time
      • (will) end of q3, edx.org can switch over, then we can talk about whitelabel.
      • (steve m) could be tricky to support two carts, with untested assumptions between them.
      • (steve s) need good test automation for microsites to ensure no breakage.

(Beth) why are we not looking at whole e-commerce suites?

(Chris D joins)

(Miki) likes a homogenous tech stack, and wants to be able to support the open source community.

[Q3 Architecture drawing]

  • Payment processors: external now, but not necessarily
  • Report and Audit: Chris D has started it.
  • Looking to move course modes to be variants of products in the shopping cart.
  • "Product" vs "Course": carts will call it "product", and we need to be able to sell things other than products
  • in the diagram, e-commerce front-end and shopping cart are shown as two services, but we could combine them in one-process
  • (ed) will credit-card numbers ever be on our network?
    • for pci-compliance, we want them not to be.

[Post-Q3 Vision]

  • better encapsulation
  • drupal has a reduced role
  • more, smaller services
  • should we make it a requirement that services can run in-process for ease of debugging, etc?
    • defining a flexible in/out process layer will be a good strategy for pulling code out of the LMS
  • (steve m) resiliency in the face of service failure?
    • (cale) we have patterns for this, but we haven't implemented them.
    • (miki) is this different than the db being down?
    • (ed) yes, because db is essential, nothing can happen if it's down.  These services, we could make more optional.
    • (miki) IDA addresses a lot of these sort of patterns. we need to do that.
    • (nimisha) feature-flag thinking already gets us thinking about optional services.
  • (piotr) latency? many network hops will mean a slow UI
    • (miki) we'll need to solve the response time problem.

Providers in other languages:

  • The bar is really high to choose a provider in a language other than Python.
  • If we choose one, we have to have really good reasons.

Comments needing response

  • What patterns will be used to make the system resilient to service failures?
  • All APIs should come through the Architecture Council
  • What approach is being used to be certain of interoperability with external services?
    • Payment processors can be re-chosen by other operators, but not other services.