The following architectural investments will benefit edX, Open edX, and the ecosystem. The developer estimates listed are for delivering initial iterations that we can continue to build upon.
Note: This is not a roadmap or backlog that is sorted in priority order. This is a categorized list of investments.
Ask: Please provide your thoughts via comments, etc. What is missing? What is unclear? What do you disagree with?
In order to substantially reduce modifications to the core of our platform and sustainably accelerate platform development, invest in an API-based extensible-core with business-prioritized integration points.
Frontend Plugin Framework. A framework that allows dynamic configuration and injection of custom widgets onto web frontend experiences. These frontend customizations are configured by users (e.g., course teams) without the need for operational changes to the platform instance (i.e., no backend installations or updates are required). Possibly implemented in relation to LTI. (2 squad-months) [Platform-TNL Roadmap, FY21-Q1]
Platform Frontend APIs. An API abstraction layer externally exposed from the monolith platform that enables more rapid frontend development of platform features. Possibly implemented using GraphQL. (framework: 2 squad-months, 1 squad-week per API) [Platform-Arch Roadmap, FY21-Q3]
Platform Backend APIs. API abstraction layers (in Python) that enable sustainable maintenance of Backend Plugins of the monolith. (framework: 2 squad-months, 1 squad-week per API) [Platform-Arch Community Project, FY21-Q1]
(Real-time) Eventing APIs. Standardized event-driven APIs that provide learning events to permitted external services as described in OEP-26. (3 squad-months, includes base TNL events listed in the OEP) [Platform-Arch Community Project, FY21-Q1]
Data Reporting APIs. API abstraction layers externally exposed from data-marts (downstream of a data warehouse) that provide domain-based data. For example, data about users, courses, and enrollments. These may be in the form of batch asynchronous reports. (framework: 2 squad-months, 1 squad-week per API)
xBlock v2 Framework. Redesigning and updating the xBlock framework so it is further advanced to the latest technological trends in frontend development and supports offline mobile use cases. Possibly implemented based on learnings from the Frontend Plugin Framework implementation. (6 squad-months)
Frontend Replatforming Completion. The edX architecture team developed the infrastructure and runways to separate the frontend from the complex core in modern frontend technology (our React-based micro-frontend re-platforming effort). While we rewrote a few existing pages to this new model, to feel the full impact of this effort, we need to complete the re-platforming effort and remove the old frontend technologies from the core. (3 squad-years?) [Platform-TNL Blended Projects, FY20-21]
Specific API enhancements:
Catalog Integration API. Support the pluggability of different marketing catalog implementations so the platform’s discovery service can be readily used by other Open edX instances. (1 squad-month)
Credentials Integration API. Support the pluggability of different digital credentials so the platform’s credential service can be readily extended to integrate with other services/standards. (1 squad-month)
Course Blocks API. Make this API further extensible so updates (supporting additional block transformers and additional fields) do not require modifications to the core platform. (2 squad-weeks)
In order to speed up development in the platform, invest in addressing high-impact technical standardizations that are table stakes for any web platform. Further delay in addressing these only results in further debt.
Toggles. Improve feature toggle sustainability and discoverability by annotating our toggles and implementing automated reporting as described in OEP-17. (1 squad-month) [Platform-Arch Roadmap, FY21-Q1]
Inter-service Communications. Provide guidelines for communicating between micro-services and synchronizing data across services. Possibly introduce an eventing architecture, such as Kafka. (2 squad-months) [Platform-Arch Roadmap, FY21-Q1]
Authentication. Improve speed of API creation and usage by providing a consistent JWT+OAuth authentication framework across all of our microservices, with updates to all Django middlewares and Django APIs. (1 squad-month) [Platform-Arch Roadmap, FY21-Q2]
Authorization. Improve performance of permission checking and developer speed of permission management by implementing the Permissions ADR and updating has_access calls in this sheet. (2 squad-months) [Platform-Arch Roadmap, FY21-Q3]
API Practices. Improve the speed of API creation and usage by aligning on API standards and technologies. (1 squad-month)
Platform Direction. Direct the future state of the platform by documenting (in the form of ADRs) and aligning on technical decisions of how the platform should be (its structure, tests, APIs, etc). (1 squad-month)
In order to improve developer onboarding experience and engagement, invest in platform enhancements that improve satisfaction for new (and returning) developers.
Tests. Improve the testing experience so testing doesn’t take ¾ of developer time. (? dev-month) [Platform-Arch Blended Project, FY21-Q1]
Devstack. Improve the speed of local development by decomposing the current implementation into distributed subdomains using Docker orchestration technology. (1 squad-month) [Platform-SRE Roadmap, FY21]
Documentation. Improve onboarding documentation, including architecture onboarding, feature how-tos, etc. (3 squad-weeks + ongoing)
Codebase Organization. Improve structure of repositories (particularly, edx-platform) to enable quicker discovery and better comprehension of code. (1 squad-month + ongoing)