Resources for Large-scale and Distributed Systems
Modern
Web
Overambitious API Gateways: anti-pattern per Thoughtworks
Microservices by Martin Fowler
Books
Domain-driven Design by Eric Evans
Notes here: Domain-Driven Design Book Club Notes
Building Microservices by Sam Newman
Notes here: Building Microservices
Michael Nygard
Trainings
Notes from training here: Architecture without an End State with Michael Nygard
Classics
On the Criteria to be used in Decomposing Systems into Modules, Parnas (CMU), ACM, Dec 1972
Simplifying change as a critical criterion for modularizing software.
Using "Information Hiding" as the criteria for modularization bodes better to Requirements Changes than "Flowchart-based componentization" i.e., conventional "Functional decomposition".
Using libraries to share common encapsulated modules.
Additional modularization best practices
Data structures are not shared/passed outside of the module.
The routine and its factory are encapsulated within the module.
Encapsulate "formats of control blocks" - i.e., API interfaces.
Modules should not rely on sequencing of operations.
Clean Decomposition is independent of Hierarchical structure
Lower-levels can be reused independently
Higher-levels are simplified by using lower-levels
Using tool abstractions to compose systems, Garlan, Kaiser, Notkin (CMU/Columbia/UofW), IEEE, 1992
Managing complexity and supporting evolution are fundamental problems with large-scale systems
Data abstraction allows easier change in design decisions about the representation of data structures, while tool abstraction allows easier change in system functions.
Using event-driven integration to implement functional composition on top of data-abstraction-based modularization.
Spreadsheets as an ideal example of this - with additive equations as event-driven control of function - orthogonal to the underlying data.
A Note on Distributed Computing, Waldo, et al, IEEE, 1994
Can't have a fault-tolerant system when remote calls are made as local calls.
Decoupling Change from Design, VanHilst and Notkin (UofW), ACM, 1996
Making a change can be expensive, even if limited to a single module - as modules themselves can be complex and large.
Recursively decomposing using Parnas' techniques is not easy or appropriate.
Decomposing modules around smaller design decisions - encoding one design choice per "submodule".
Example design decisions: data type, algorithm, remote or local procedure calling.
All design choices are considered equally likely to change.
Using C++ template classes
Decision not pre-determined by developer.
Sub-modularize modules to achieve a fine-grained separation of concerns.
Using submodules, the choice of high level modularization could itself be changed without having to rewrite large amounts of code.
Modularity for the Modern World, Mary Shaw (CMU), ACM, March 2011
Why Modularize?
Intellectual control - cognitive independence.
Segmentation of work - distribution of responsibilities.
Evolution - localize changes, substitutions, composition, keep up with marketplace.
Service-oriented architecture