Architecture Principles
| Answers the Question(s) | Brief Description | edX Example(s) | References (Reading List) |
|---|---|---|---|---|
Domain-Driven Design (DDD) | Where should this code live? Which service/app should be the source of the truth of this data? What should be the boundary (or single responsibility) of this service/app? What should we name "it"? How should we structure our code? How should we keep this service/app isolated from others? What are edX's main business domains and bounded contexts? | DDD provides principles for modeling our system and our code. Eric Evan's original book is hefty and contains many finer points, but the biggest takeaways for edX are:
|
Non-Examples:
| |
Reactive Manifesto (RM) & Self-Contained Systems (SCS) | How can we achieve low latency when responding to end-user requests? How can we scale the communications between services in our architecture? What should be the communication relationship between backend services? What should be the separation of concerns between backend and frontend? | The Reactive Manifesto recommends asynchronous (non-blocking) communication channels between services wherever possible in order to create a scalable loosely coupled architecture. This then results in a system that is responsive, resilient, and elastic. The principles of Self-Contained Systems take the Reactive Manifesto one step further and advocate for highly decoupled verticals in the system, with each vertical owning its own frontends and data. Each SCS is highly cohesive and completely independent of other SCS's. This may lead to data duplication across SCS's, but results in a highly loosely coupled architecture. This is consistent with DDD's recommendation for data ownership within each context. |
Non-Examples:
| |
SOLID | Where should this code live? How do we scale development so volatile changes don't require modifications to a stable core? What should be the design of this API? Should we break up this component/API into multiple parts? How do we create an API suitable for a plugin architecture? | Following SOLID design principles lead to creating a scalable and maintainable codebase where:
|
|
Comments
The manifesto might benefit from concrete examples.
I very much agree.
For instance, "Intentional beats Emergent" might be interpreted to be at odds with "Autonomy beats Governance" when taken at the overall system level.
True. And that apparent contradiction speaks at heart to the matter of the tension between team autonomy and still creating an intentionally architected platform.
I'm unclear on what exactly "Coordination beats Collaboration" means.
We can probably remove this line as it may not add as much value in this context. First of all, I got it reverse. It should be "Collaboration beats Coordination". It's intended to inspire an organizational structure where we are highly aligned but loosely coupled. That is, we collaborate towards a common goal, but don't get bottle-necked through centralized coordination.
"Simple beats Complex" is one of those mantras I've always had difficulty wrapping my head around because nobody goes out with the intention of chasing complexity for complexity's sake. One of the reasons I did my whole "Simple is Better" lightning talk is to try to explore what people mean when they say simple (e.g. simplicity of interface vs. simplicity of implementation) and how it trades off against things like completeness and correctness. Again, maybe just a case where concrete examples could help?
What I've found is that Simplicity takes extra effort and iterations. When one implements or designs and submits a first draft/iteration as the completed state, it will typically be far from its ideal elegant potential. For example, when creating a diagram with all its dependencies, the first pass may be complete in its correctness but may result in something that is very complicated to understand. Further iterations to simplify that diagram (fewer crossovers, improved naming, better visualizations, etc) may require additional sweat but worthwhile in creating an artistic representation that is elegant and welcoming to the eyes, and as importantly, better understood. In my experience, the same holds for any piece of code and any piece of interface. For example: reducing complexity by eliminating circular dependencies through explicit layers or by extracting explicit concepts, breaking up large complicated functions into explicit and properly named helper functions, creating APIs with screaming business concepts. In the end, the extra effort results in something much simpler than where you started. I'd love to see the Open edx Platform become a piece of art. We can achieve that by striving for simplicity.
. Is "Loosely Coupled > DRY" the same as "Duplication > Shared Business Logic"?
There are some examples that apply to one and not the other.
As an example of the first case:
As developers, our initial inclination is to share utility functions across microservices just because the same code happens to be needed in both services. For example, a utility that optimizes the performance of retrieving the User object. Such a utility may start off as being the same across services, but over time, this can introduce an unnecessary coupling as each service's needs evolves.
Examples of the 2nd case:
We prefer to have each micro-frontend make its own domain-specific API calls. Even if multiple micro-frontends are calling the Credentials service, for example, each should have its own client code to make that call. Over time, they may evolve to include their own parameters, optimizations, etc.
We prefer to have each microservice implement its own model of caching data from the discovery service. Each microservice can optimize its own data model according to its own use of that data.
Where DRY makes sense:
Duplicated code within bounded contexts.
Shared utilities that will never need to evolve differently for an individual service.
Architecturally-significant utilities that we believe are necessary to be consistent across all services.
I like the idea of agreeing upon a set of high-level architectural principles, and deriving plans from them. The high-level and second-order principles all look good. The blue bubbles on their own seem a little too abstract to be "plans", but I'm sure that's something we could flesh out.
Why is "Loosely Coupled" our strategy, and not just a plan? Is it the only strategy?
On the Manifesto, I agree with most of Dave's points. I'm curious what the rationale behind the statements are, and if they're supported by experiences edX has had. Maybe I'm missing some history here.
Lastly, I generally disagree with "Simple beats Complex", unless it is accompanied by "Complex beats Complicated". Sometimes the "Simplest" approach (throw everything in edx-platform! plugins/microservices/MFEs are hard!) ends up creating the most Complicated result, and starting with "this will be Complex; let's design for it" would have been better.
1. I like the idea of agreeing upon a set of high-level architectural principles, and deriving plans from them. The high-level and second-order principles all look good. The blue bubbles on their own seem a little too abstract to be "plans", but I'm sure that's something we could flesh out.
Sounds good. I'm looking forward to making further refinements to this with the team's help. Note that these are intended to be Principles, and not an Architecture Roadmap or an Architecture OGSP. So some abstractness will still be there. Perhaps the term "plans" is inappropriate for this.
2. Why is "Loosely Coupled" our strategy, and not just a plan? Is it the only strategy?
As I was putting this together, and thinking about what the main outcomes we were looking for elevating our platform, it seemed that Autonomous Teams and Scalable Architecture were our primary goals. And then thinking about the main Arch Principles we've been pitching as part of our Architecture Roadmap Details, 2018-19 (SOLID, Reactive, DDD, Evolutionary), they all push us toward a loosely coupled and decentralized system and organizational structure. So yes, I do think "Loosely Coupled" seems like our main strategy to get us to where we want to go.
Is it the only strategy? Perhaps not. There may be another one around Defining Core.
3. On the Manifesto, I agree with most of Dave's points. I'm curious what the rationale behind the statements are, and if they're supported by experiences edX has had. Maybe I'm missing some history here.
See my responses to Dave's post. They are based on (1) my own experiences/analysis at edX and elsewhere and (2) tradeoffs resulting from the principles above it.
4. Lastly, I generally disagree with "Simple beats Complex", unless it is accompanied by "Complex beats Complicated". Sometimes the "Simplest" approach (throw everything in edx-platform! plugins/microservices/MFEs are hard!) ends up creating the most Complicated result, and starting with "this will be Complex; let's design for it" would have been better.
See my response to Dave's post. "Simple" does not equate to "Easy". In fact, an Elegant and Simple solution takes multiple iterations and more time. Experienced developers may arrive at Simpler solutions faster than more junior engineers, but only if those experienced developers took the time in their careers to sweat and iterate towards Simpler solutions.
FWIW, overall I like the manifesto approach over the principles map. The principles map has a lot of good terms in it, but it's debatable what SOLID really means for any particular situation, and I wouldn't be able to describe exactly what "High Cohesion" as a plan would translate into. When good sounding architectural principles are presented in isolation, it starts to sounds like "write good code, not bad code" – unobjectionable, but not as illuminating.
The manifesto is great because it gives opinions on which of two plausible paths we're electing to take, and explicitly points out the things we're not doing. I have a whole raft of questions, minor modifications, and explanations I'd like to see in the manifesto, but I think it's a much better vehicle for conveying architectural values in a way that's clear and actionable by teams.
Thanks @Dave Ormsbee (Deactivated).
I'd love to have edX engineers name and articulate the architectural decisions they make. I'd love to walk through our halls and hear teams debate and agonize over their designs. I'd love to teach our engineers about principles, such as SOLID, so when they pause and seek which path to take, they can recollect the principle and apply it to their designs. I have used these terms with the teams and individuals I've worked with over the last 2 years. I have since, heard some of them echo them back.
These terms provide industry names to the intuitions that we develop as we become experts in the field. These terms will align us with achieving the Extensible Core model that will help us succeed.
It's going to be challenging, not easy. But a girl can dream.
The manifesto might benefit from concrete examples.
For instance, "Intentional beats Emergent" might be interpreted to be at odds with "Autonomy beats Governance" when taken at the overall system level.
I'm unclear on what exactly "Coordination beats Collaboration" means.
"Simple beats Complex" is one of those mantras I've always had difficulty wrapping my head around because nobody goes out with the intention of chasing complexity for complexity's sake. One of the reasons I did my whole "Simple is Better" lightning talk is to try to explore what people mean when they say simple (e.g. simplicity of interface vs. simplicity of implementation) and how it trades off against things like completeness and correctness. Again, maybe just a case where concrete examples could help?
Is "Loosely Coupled > DRY" the same as "Duplication > Shared Business Logic"?