Systems Architecture training
Goal: Gain a strong working understanding of tools and trends for distributed software system architecture and development, to allow the learner to better design and build software systems at edX.
Intro
Goal: Understand what distributed computing is, and what’s it’s for.
Level of investment: Low (wikipedia / blog posts)
(TBD)
IDesign started off as a Microsoft consulting shop but has since branched out to other technologies. Their “Method” documents and related resources are great references: http://idesign.net/
www.infoq.com is a great place to find recordings of pertinent presentations
www.highscalability.com is also a great resource -- lots of case studies and real-world examples available there. Also lots of blog chaff, so be careful :)
Udi Dahan used to have a lot more info on his site, but he still has some great stuff: http://udidahan.com/articles/
Distributed architectures
Goal: Become familiar with common distributed computing paradigms
Level of investment: Medium (blog posts / podcasts / talks)
Client-Server
N-tier
Peer-to-peer
Distributed logs
Message-passing
RPC
Bonus: a perspective on a few key architecture patterns. http://www.oreilly.com/programming/free/software-architecture-patterns.csp
Tools (Past + Present)
Goal: Become familiar with the concepts of each of these distributed computing tools. Understand why it was created, what it does, what it’s good at, what it’s not good at, and whether it was successful.
Level of investment: Low (wikipedia / blog posts)
- TCP/IP
- HTTP
- DNS
- Web sockets
- REST
- TLS
- Protocol buffers
- Message queuing protocols, reliable messaging, persistent messaging
- Storage
- SQL and Relational Databases
- Document stores
- Object databases
- Graph databases
Distributed Systems
Goal: Gain familiarity with common distributed tools, with a focus on tools used at edX
Level of investment: Medium (blog posts, at least one practical / lab project)
Amazon Web Services (esp. SQS, SES, SNS)
Memcached architecture
Elasticsearch architecture
Cassandra architecture
RabbitMQ architecture
Hadoop architecture (there is an edX course on Hadoop)
Services
Goal: Understand what you need to consider when building a service.
Level of investment: Medium (blog posts, at least one practical / lab project)
AuthZ/AuthN
Service Discovery + Service Registries
Distributed Configuration Management
Continuous integration and delivery
APIs
Design
Publishing
(Eddie still thinks Heroku's "12 Factor App" is a great starting point, even though it’s getting a bit dated…)
Performance
Goal: Understand performance characteristics of a distributed web architecture, including paradigms and tools.
Level of investment: Medium (blog posts, at least one practical / lab project)
Capacity planning
Scaling
Caching and content delivery networks (e.g. CloudFront)
Network latency
Serialization
Testing
High availability
Goal: Understand the concepts and tradeoffs of a highly-available web architecture, including tools and algorithms for creating a HA system.
Level of investment: Medium (blog posts, at least one practical / lab project)
- Concurrency
- Failover
- Circuit breakers
- https://martinfowler.com/bliki/CircuitBreaker.html
- Netflix on Circuit Breakers
- Hystrix (NetFlix open-source circuit breaker library): https://github.com/Netflix/Hystrix/wiki
- https://martinfowler.com/bliki/CircuitBreaker.html
- Eric Brewer’s CAP Theorem
- http://www.julianbrowne.com/article/viewer/brewers-cap-theorem
- This is a great - if heavy - follow-up written by Brewer earlier this year, on Google Spanner that I’d recommend as supplemental reading: PDF
- Related themes: eventual consistency, split brain
- Command-Query Responsibility Segregation (aka, write/read, aka, CQRS)
- AWS-specific appendix: elastic load balancers, auto-scaling groups
Future
Goal: Understand how the field is evolving; understand current trends such as containerization and serverless computing.
Level of investment: Low (blog posts)
Containers (LXC, Docker)
Serverless computing - reading:
Serverless computing - course:
Other Materials
- Nimisha has another list: Resources for Large-scale and Distributed Systems. We should combine that list into this one.
- Also, see detailed notes from a 2-day architecture training: Architecture without an End State with Michael Nygard
Reference
Randy Shoup talk