Work with the Open edX community on Needed Architecture Investments

Upgrades: Foundation for sustainable upgrades; Django2, Python3

• Outcomes from doing upgrades: (1) Enable quality engineering with up-to-date technology standards, (2) avoid potential security risks by using only supported versions of the technology.

• Outcomes from having a foundation for sustainable upgrades: (1) Proactively keep the platform up-to-date with intentional resourcing and outsourcing, (2) Engineers who are focused on other critical functions are not interrupted for emergency upgrade situations.

• Plans:

  • Foundation for future upgrades

    • Empower multiple upgrade teams

    • Automated reports on upgrade issues and deprecation warnings

    • Standardize repos: OEP-2 (repo metadata) and OEP-18 (dependency tools)

  • Django: Django 2.2 Upgrade Plan

  • Python-3: partner support of Code Jail upgrade

  • Elastic-Search: cross-team discovery and alignment

Technical Ownership & Observability: Empower teams to be proactive business-aligned Tech Owners

• Outcomes: (1) Proactively and efficiently respond to user-facing issues that impact business metrics, (2) empower teams to make strategic technical decisions for the business features that they own, (3) empower teams to maintain and upkeep their own technologies according to agreed-upon SLOs.

• Plans:

  • Owners assigned to teams for all repos.

  • Owners assigned to teams for all apps in edx-platform.

  • Develop and communicate ownership rollout plan

  • Empower shared owners of edx-platform

    • Define boundaries; define core versus extensions; define content versus features.

    • With Alerting, Observability, etc.

Deprecation & Removal: Speed up development in inter-service authentication

• Outcomes: (1) Speed up future development by reducing the complexity of having multiple implementations in the platform and (2) expedite future upgrade efforts by eliminating unsupported implementations and their dependencies.

• Plans:

  • DOP-recation (DEPR-47): eliminate an old (now-forked by edX) OAuth implementation.

Backend Efficiency & Clarity: Initial plans to speed up & clarify backend development

• Outcomes: (1) Speed up backend development, (2) reduce and stabilize a defined core, (3) clarify architectural principles and vision.

• Plans:

  • Feature Toggle annotations and reporting

  • Remove unneeded, fragile, and slow integration (Bokchoy) tests

  • Define edx-platform vision in ADRs

  • Rollout Architecture Principles and Architecture Manifesto

• Design Sprint exercises to Tame the Monolith (docs, linting, tests, APIs, real-time eventing, decoupling, ???)

Upgrades

• Outcomes: (1) Enable quality engineering with up-to-date technology standards, (2) avoid potential security risks by using only supported versions of technology.

• Plans:

  • Complete Python-3 upgrade of edx-platform.

    • The Python community's support of Python 2 ends in December 2019. We are upgrading our platform to Python 3.5, which is supported until September 2020.

  • Complete Node upgrade of edx-platform.

    • Node is a framework we use to download and run frontend libraries. We currently use Node 8, support ends in December 2019. We are upgrading our platform to Node 12, support ends April 2022.

  • Foundation for Django upgrade and other future upgrades.

    • Automated Dashboards (for now: edx-platform and its dependencies)

      • Reduces effort to coordinate and delegate upgrade work.

      • Deprecation Warning Dashboard 

      • Upgrade Status Dashboard

    • Explore to experiment with outsourcing options for upgrade efforts.

Observability & Ownership

• Outcomes: (1) Proactively and efficiently respond to user-facing issues that impact business metrics, (2) empower teams to make strategic technical decisions for the business features that they own, (3) empower teams to maintain and upkeep their own technologies according to agreed upon SLOs.

• Ownership Plans:

  • Clarify the granularity of Ownership Assignment (OEP-2 update)

  • Comprehensively assign ownership to all edX technologies (code repositories), using business-aligned terminologies.

Deprecation & Removal of Old Superceded Implementations

• Outcomes: (1) Speed up future development by reducing the complexity of having multiple implementations in the platform and (2) expedite future upgrade efforts by eliminating unsupported implementations and their dependencies.

• Plans:

  • DOP-recation (DEPR-47)

    • Eliminate an old (now-forked by edX) OAuth implementation.

  • Login and Registration (DEPR-52)

    • Consolidate and co-locate the 3 different implementations spread across multiple locations.

Micro-frontend (MFE) Runways

• Outcomes: MFE framework is readily consumable by teams, with minimal maintenance burden.

• Plans:

  • Anonymous access in MFEs: allow access to MFEs by users that are not logged in.

  • Reduce maintenance of frontend library dependency upgrades

    • Automate Renovate to merge updates that are marked as non-breaking.

    • Refactor MFE runway libraries to create a “frontend platform” that reduces the complexity of dependency management via dependency inversion design patterns.

• Frontend development time reduced

  • Micro-frontend (MFE) Page rewrites

    • Payment Page: MFE reimplementation shipped and handed off to the Revenue squad.

  • Supporting projects (runways)

    • Publish a revamped Paragon library, with:

      • updated components to improve performance & reusability

      • updated UX-reviewed visual styles

      • a documentation site

    • Extract MFE commonalities into libraries to reduce maintenance effort. (DRYness)

    • Enable MFE differentiation based on context: B2B, B2C, or Open edX. (Config)

• Reach: Extensible, small and stable core

  • Strawman design of MFE plugin architecture.

• Platform upgrades

  • Python 3 upgrade of edx-platform and its dependencies

    • Goal: Tests run successfully in Python 3

    • Reach goal: run Python 3 in production