...
| Note |
|---|
We have decided not to go forward with decentralized devstack after doing further evaluation. This documentation is now out of date. See OEP-5 Decisions for more details. |
Note: This is a copy of Decentralized Devstack Spike Team Report. This is meant as a historical doc and is no longer completely up to date of current design.
...
Decentralized Devstack Spike Team Report
Team: Adam Blackwell, Bill DeRusha, Diana Huang, Kyle McCormick
Summary
Outcomes
85% Development Environment Setup Speed Improvement
Improved Troubleshooting Time
Things We Tried
Things We Didn't Try
Action Items h1.
| Anchor | ||||
|---|---|---|---|---|
|
Over the week of July 20th 2020, the Decentralized Devstack Spike Team tested the viability of the technical approaches proposed in the Devstack 2.0 One-pager Memo. Decentralizing devstack aligns with our ownership strategy by allowing code and repo owners to have more control over their own development environments and development workflows. Allowing developers to develop in their own apps without having to maintain an edx-platform development environment means that the pain of dealing with edx-platform development is largely mitigated for much of the engineering organization. The steps required to make that possible also benefit those who are working directly in edx-platform.
As a proof of concept the team created a development environment for the enterprise catalog IDA that has no dependencies on our existing devstack and can be managed entirely within the enterprise-catalog repository. By the end of the week the team had an environment that could be spun up from scratch in less than 5 minutes (an 85 - 90 percent improvement)! The methods described in later sections prove out the approaches and provide performance wins today, but are just first passes that will benefit from further iterations to improve performance and maintainability.
The next steps for this work are to validate the setup with someone from the enterprise-catalog dev team and provide more detailed documentation/training for how to convert a repo to this new development method. Beyond that there are additional investments that can be made depending on the willingness of the business to commit staff to this endeavor.
...
The enterprise-catalog repo is able to be fully provisioned and started from scratch in less than 5 minutes. [Video]] This was accomplished by improving data provisioning, removing our dependency on Ansible and standardizing Docker containers for dependencies.
...
We have gathered all of the recommendations from our notes and placed them into three categories based around relative effort.
Immediate Next Steps:
Clean and Test
Clean up & merge edx-platform PR to improve Dockerfile.
Clean up & merge course-discovery PR to improve Dockerfile.
Collect developer feedback from Enterprise
Clean up & merge enterprise catalog spike PR as an alternate dev environment.
Test proof of concept decentralized devstack pattern with an enterprise developer
...
Improve image performance
Smarter caching for requirements in edx-platform Dockerfile [PR ready]
Ensure Github Actions are building and publishing production and devstack docker images for LMS, Discovery & Enterprise Catalog
Create automatic MySQL dumps for all apps
Discovery and enterprise catalog run migrations and would benefit from consistent data loading from dumps
LMS data dumps would benefit from automation to prevent staleness
Dumps can be better managed in s3 rather than committed to repo
Improve UX of decentralized devstack
Make getting a devstack running in a simple, no-nonsense manner
Deploy Enterprise Catalog as container to production to avoid drift between container and ansible deployment patterns
Modify/enhance this decentralized pattern to support MFE / plugin development
...