Goal: Gain a strong working understanding of tools and trends for distributed software system architecture and development, to allow the learner to better design and build software systems at edX.
...
(TBD)
IDesign started off as a Microsoft consulting shop but has since branched out to other technologies. Their “Method” documents and related resources are great references: http://idesign.net/
www.infoq.com is a great place to find recordings of pertinent presentations
www.highscalability.com is also a great resource -- lots of case studies and real-world examples available there. Also lots of blog chaff, so be careful :)
Udi Dahan used to have a lot more info on his site, but he still has some great stuff: http://udidahan.com/articles/
Distributed architectures
...
Client-Server
N-tier
Peer-to-peer
Distributed logs
Message-passing
RPC
Bonus: a perspective on a few key architecture patterns. http://www.oreilly.com/programming/free/software-architecture-patterns.csp
Tools (Past + Present)
Goal: Become familiar with the concepts of each of these distributed computing tools. Understand why it was created, what it does, what it’s good at, what it’s not good at, and whether it was successful.
Level of investment: Low (wikipedia / blog posts)
- TCP/IP
- HTTP
- DNS
SQL and Relational Databases
SOAP
DCOM
CORBA
- Web sockets
- REST
- TLS
- Protocol buffers
- Message queuing protocols, reliable messaging, persistent messaging
- Storage
- SQL and Relational Databases
- Document stores
- Object databases
- Graph databases
Distributed Systems
Goal: Gain familiarity with common distributed tools, with a focus on tools used at edX
Level of investment: Medium (blog posts, at least one practical / lab project)
Amazon Web Services (esp. SQS, SES, SNS)
Memcached architecture
Elasticsearch architecture
Cassandra architecture
RabbitMQ architecture
Hadoop architecture (there is an edX course on Hadoop)
Services
Goal: Understand what you need to consider when building a service.
...
AuthZ/AuthN
Service Discovery + Service Registries
Distributed Configuration Management
Continuous integration and delivery
APIs
Design
Publishing
(Eddie still thinks Heroku's "12 Factor App" is a great starting point, even though it’s getting a bit dated…)
...
Level of investment: Medium (blog posts, at least one practical / lab project)
- Concurrency
- Failover
- Circuit breakers
- https://martinfowler.com/bliki/CircuitBreaker.html
- Netflix on Circuit Breakers
- Hystrix (NetFlix open-source circuit breaker library): https://github.com/Netflix/Hystrix/wiki
(Eddie) - https://martinfowler.com/bliki/CircuitBreaker.html
- Eric Brewer’s CAP Theorem
Related themes: eventual consistency, split brain
- http://www.julianbrowne.com/article/viewer/brewers-cap-theorem
- This is a great - if heavy - follow-up written by Brewer earlier this year, on Google Spanner that I’d recommend as supplemental reading: PDF
- Related themes: eventual consistency, split brain
- Command-Query Responsibility Segregation (aka, write/read, aka, CQRS)
- AWS-specific appendix: elastic load balancers, auto-scaling groups
Future
Goal: Understand how the field is evolving; understand current trends such as containerization and serverless computing.
...
Containers (LXC, Docker)
Serverless computing - reading:
Serverless computing - course:
Reference
Randy Shoup talk