Building Microservices

Owned by Nimisha Asthagiri (Deactivated)
Last updated: Dec 02, 2017
9 min read

Notes on Building Microservices, Designing Fine-grained Systems by Sam Newman

Ch 3. Bounded Contexts

  • Seams with Loose coupling and High cohesion
  • Beware of premature decomposition before domains and usages are solidified.

Ch 4. Integration

  • Avoid database integration at all cost
  • DRY and code-reuse can couple microservices
  • Sync vs Async
    • request/response vs event-based vs reactive (observe for results)
    • request/response can be sync or async - async by registering a callback
      • "orchestration" pattern leads to centralized authorities with anemic CRUD-based services
      • these systems are more brittle with a higher cost of change
      • technology choices
        • RPC
          • easy to use
          • watch out for: technology coupling, incorrectly treating remote calls as local calls, lock-step releases
        • REST over HTTP
          • more resilient to changes than RPC - sensible default choice
          • not suited for low latency and small message size (consider WebSockets) 
    • event-based - better decoupling; intelligence is distributed
      • "choreographed" pattern leads to implicit view of the business process (since no centralized workflow)
      • additional work to monitor/track - can create a monitoring system that matches the view of the business process - validates flowchart expectations
      • these systems are more loosely coupled, flexible, and amenable to change
      • technology choices
        • RabbitMQ
        • HTTP + ATOM
      • managing complexities
        • maximum retry limits
        • dead letter queue (for failed messages) - with UI to view and retry
        • good monitoring
        • correlation IDs to trace requests
  • Versioning
    • Postel's Law, a.k.a. Robustness principle: "Be conservative in what you do, be liberal in what you accept from others."
    • consumer-driven contracts to catch breaking changes early
    • semantic versioning - self-documented impact
    • expand and contract pattern when versioning breaking changes.
      • co-existing versions needed for blue-green deployments and canary releases
  • UI as the compositional layer
    • UI Fragment composition
      • server-side rendering of course-grained fragments work well when aligned with team ownership
      • problem: consistency of UX - mitigated with shared CSS/images/etc
      • problem: doesn't support native applications / thick clients
      • problem: doesn't work for cross-cutting features that don't fit into a widget/page
    • Backends for Frontends
      • aggregated backend layers, with dedicated backends serving UI/APIs for dedicated frontend experiences
      • danger: keep business logic within the underlying services.  Aggregated backends should contain only front-end specific behavior
    • Hybrid of both approaches above 
  • Third-party software
    • Build or Buy commercial-of-the-shelf?
      • "Build if it is unique to what you do, and can be considered a strategic asset; buy if your use of the tool isn't that special" - Build if core to your business
    • Problems with COTS
      • lack of control
      • customization - avoid complex customizations - rather, change your organization's functions
      • integration spaghetti - with different protocols, etc
    • On your own terms
      • Hide COTS CMS behind your own web frontend, putting the COTS within your own service facade
    • Use Strangler Application Pattern to capture and intercept calls to the old system

Ch 5. Splitting the Monolith

  • Seams
    • Identify seams that can become service boundaries (not for the purpose of cleaning up the codebase).
    • Bounded contexts as seams, as they are cohesive and loosely coupled boundaries.
  • Reasons to split
    • Pace of change is faster with autonomous units.
    • Team autonomy
    • Replaceable with alternative implementation
  • Tangled dependencies - use a dependency analysis tool to view the seams as a DAG to find the least depended on seam.
  • Coupling at the database layer
    • Examples of database refactoring to separate schemas
    • Transactional boundaries - split across databases
      • Design patterns for failures (success in one db, but failure in another)
        • Eventually consistency - try again later
        • Compensating transaction - abort entire operation
          • more complex to recover
          • need other compensating behavior to fix up inconsistencies
        • Distributed transaction using a transaction manager
          • 2-phase commit
          • Locks on resources can lead to contention, inhibiting scaling
      • Rather than requiring distributed transactions, actually create a higher-level concept that represents the transaction.
        • Gives a natural place to focus logic around the end-to-end process and to handle exceptions
  • Reporting Systems
    • Can use a read-replica to access the data - but couples database technology
    • APIs - don't scale
    • Data pumps to push the data, rather than have reporting system pull the data
      • service owners write their own pump so not coupled to service schemas
      • reporting schema treated as a published API
      • aggregated view of all service pumps within the reporting system
      • need to deal with complexity of segmented schema
    • Event data pump
      • Reporting service just binds to the events emitted by upstream services
      • Looser coupling and fresher data
      • May not scale as well as data pumps though
    • Backup data pump
      • Variant of data pump used by Netflix using Hadoop off of S3-backed Cassandra data
  • Cost of change
    • Make small, incremental changes to understand the impact of each alteration - mitigates cost of mistakes
    • Small cost: Moving code around within a codebase
    • Large cost: Splitting apart a database
    • Whiteboard
      • Make mistakes where the impact will be the lowest: on the whiteboard
      • Go through use cases
      • Class-responsibility cards (CRC) - borrowing from OOP - each card includes name of the class, its responsibility, and its collaborators

Ch 6. Deployment

  • Continuous Integration
    • CI server detects code is committed, verifies code and runs tests
    • Versioned Artifacts are also created for further validation and usage in downstream deployments
      • Confirms that the artifacts deployed are the ones tested
      • Reused without continual recreation
      • Traceability back to the commit
    • 3 questions from Jez Humble on whether you're really doing it
      • Do you check in to mainline once per day?
        • Even if you are using short-lived branches, integrate frequently
      • Do you have a suite of tests to validation your changes?
      • When the build it broken, is it the #1 priority of the team to fix it?
    • Repo
      • Since repo and single CI build for all microservices
        • requires lock-step releases
        • Ok for early stage and short-period of time
        • Cycle time impacted - speed of moving a single change to being live
        • Ownership issues
      • Single repo with separate CI builds mapping to different parts of the source tree
        • Better than the above
        • Can get into the habit of slipping changes that couple services together.
      • Separate repo with separate CI builds for each microservice
        • Faster development
        • Clearer ownership
        • More difficult to make changes across repos - can be easier via command-line scripts
  • Continuous Delivery
    • Treat each check-in as a release candidate, getting constant feedback on its production readiness
    • Build pipeline
      • One stage for faster tests and another stage for slower tests
      • Feel more confident about the change as it goes through the pipeline
      • Fast tests → Slow tests → User acceptance tests → Performance tests → Production
  • One microservice per build
    • Is the goal
    • However, while service boundaries are still being defined, a single build for all services reduces the cost of cross-service changes
  • Deployable Artifacts
    • Platform-specific
    • OS-specific
    • Images
  • Environments for each pipeline stage and different deployments
    • different collection of configurations and hosts
    • Service configuration
      • Keep configuration decoupled from artifacts
      • Or use a dedicated config system
  • Service to host mapping
    • Multiple services per host
      • Coupling, even with Application Containers
    • Single service per host
      • Monitoring and remediation much easier
      • Isolation of failure
      • Independent scaling
      • Automation mitigates amount of overhead
  • Virtualization
    • Vagrant - can be taxing on a developer machine when running a lot of VMs at once
    • Linux Containers
    • Docker - Vagrant can host a Docker instance

Ch 7. Testing

  • Brian Marick's Testing Quadrant from Agile Testing by Lisa Crispin and Janet Gregory
  • Mike Cohn's Test Pyramid from Succeeding with Agile
    • Go up 
      • scope increases, confidence increases, test-time increases; 
      • when test breaks harder to diagnose cause
      • order of magnitude less than previous
    • When broader-scoped test fails, write unit regression test
    • Test Snow cone - or inverted pyramid - doesn't work well with continuous integration due to slow test cycles
  • Unit tests
    • fast feedback on functionality
    • thousands run in less than a minute
    • limiting use of external files or network connections
    • outcomes of test-driven design and property-based testing
  • Service tests
    • bypass UI
    • At the API layer for web services
    • Can stub out external collaborators to decrease test times
    • Cover more scope than unit tests, but less brittle than larger-scoped tests
    • Mocking or stubbing?
      • Martin Fowler's Test Doubles
      • Stubbing is preferable
      • Mocking is brittle as it requires more coupling with the fake collaborators
  • End-to-end (E2E) journey tests
    • GUI/browser-driven tests
    • Higher degree of confidence when they pass, but slower and trickier - especially with microservices
    • Tricky
      • Which versions of the services to test against?
      • Duplicate of effort by different service owners
      • => Single suite of end-to-end tests - fan-in from individual pipelines
    • Downsides
      • Flaky and brittle
        • => Remove flaky tests to retain faith in them
        • => See if they can be replaced with smaller-scoped tests
      • Lack of ownership
        • => Shared codebase with joint ownership
      • Long duration
        • Large end-to-end test suites take days to run
        • Take hours to diagnose
        • => Can run in parallel
        • => Remove unneeded tests
      • Pile-up of issues
        • => release small changes more frequently
      • Don't create a meta-version of all services that were tested together - results in coupling of services again
    • Establish agreed-upon core journeys that are to be tested
      • Each user story should NOT result in an end-to-end test
      • Very small number (low double digits even for complex systems)
  • Consumer-driven contract (CDC) tests
    • Interfaces/expectations by consumers are codified
    • Pact - open-source consumer-driven testing tool
    • Codifying discussions between consumers and producers - a test failure triggers a conversation
    • E2E tests are training wheels for CDC
      • E2E tests can be a useful safety net, trading off cycle time for decreased risk
      • Slowly reduce reliance on E2E tests so they are no longer needed
  • Post-deployment Testing
    • Acknowledge that we cannot find all errors before production
    • Smoke test suite runs once code is on prod
    • Blue/green deployment 
      • allows quick rollback if needed
      • zero downtime deployments
    • Canary releasing
      • to verify new version is performing as expected (error rates, response time, etc) under real traffic
      • version co-exist for longer periods of time
      • either a subset of production traffic or shadow of full production traffic
    • Mean time between failures (MTBF) versus Mean time to repair (MTTR)
      • Optimizing for MTTR over MTBF allows for less time spent on functional test suites and faster time to customer value/testing an idea
  • Cross-functional Testing (a.k.a., nonfunctional requirements)
    • Fall into the Property Testing quadrant
    • Performance Tests
      • Follow pyramid as well - unit test level as well as e2e via load tests
      • Have a mix - of isolated tests as well as journey tests
      • load tests
        • gradually increase number of simulated customers
        • system matches prod as much as possible
        • may get false positives in addition to false negatives
      • Test runs
        • Run a subset of tests every day; larger set every week
        • Regularly as possible - so can isolate commits
        • Have targets with clear call-to-action, otherwise results may be ignored

Ch 8. Monitoring

  • With microservices, multiple servers, multiple log files, etc.
  • Make log files centrally available/searchable
  • Metric tracking
    • aggregated sampling across time, across services
    • but still see data for individual services and individual instances
  • Synthetic transaction (a.k.a., semantic monitoring)
    • fake events to ensure behavior
  • Correlation IDs
    • unique identifier (e.g., GUID) used to track a transaction across services/boundaries
    • one path missing a correlation ID will break the monitoring
      • having a thin shared client wrapper library can help
  • Cascading failures
    • monitor integration points between systems 
  • Service
    • response time, error rates, application-level metrics
    • health of downstream services
    • monitor OS to track rogue processes and for capacity planning
  • System
    • aggregate host-level metrics with application-level metrics, but can drill down to individual hosts
    • maintain long-term data to measure trends
    • standardize on 
      • tools
      • log formats
      • correlation IDs
      • call to action with alerts and dashboards
  • Future - common, generic system for business metrics and operation metrics to monitor system in a more holistic way

Ch 9. Security

Ch 10. Conway's Law and System Design

  • Organizational structure is a strong influence on the structure of the system
  • Microservices are modeled after business domains, not technical ones
    • Teams aligned along bounded contexts, as services are.
  • Align service ownership to co-located teams, which are aligned around the same bounded contexts of the organization.

Ch 11. Microservices at Scale

  • At scale, concerns over failures (statistically likely) and performance
  • Can over-optimize unless know requirements for:
    • response time/latency
    • availability
    • durability of data
  • Graceful degradation
  • Architectural safety measures
    • Anti-fragile organization by Nassim Taleb
      • intentionally causing failures at Netflix and Google
    • Timeouts
      • too long slows down whole system
      • too quick creates false negatives
      • choose defaults and log → monitor → adjust
    • Circuit Breakers
      • Fail fast after a certain number of failures
        • gracefully degrade or error
        • queue for later if async
      • Restart after certain threshold
    • Bulkheads
      • Lose a part of the ship but rest remains intact
      • Separation of concerns via separate microservices
    • Timeouts and circuit breakers help free up resources when they become constrained
    • Bulkheads ensure they don't become constrained in the first place
  • Idempotent operations allow repeatable/replayable messages
  • Scaling techniques
    • Vertical scaling with bigger boxes
    • Splitting workloads with microservices on their own hosts
    • Spreading risk
      • Multiple hosts
      • Across regions/data centers
    • Load balancing
      • Avoids single point of failure
      • Distributes calls to multiple instances to handle load
    • Worker-based systems
      • Message broker is resilient, allowing worker processes to scale to match load and handle failures without loss of work
    • Rewrite/redesign/reimplement as scaling needs change
      • "The need to change our systems to deal with scale isn't a sign of failure. It is a sign of success."
  • Scaling Databases
    • Availability vs Durability of data
    • Scaling for Reads - caching, read replica
    • Scaling for Writes - sharding, though querying across shards become difficult - may scale for write volume, but not improve resiliency
    • CQRS pattern separates state modification from state querying
  • Caching
    • Eliminates round-trips to databases and other services for faster results.
    • Client-side caching with possible hints from the service
      • Reduces network calls
      • Invalidation is trickier
    • Proxy caching in between client and server (e.g., CDN)
      • Generic caching easier to add to existing system
    • Server-side caching via Redis or Memcache
      • Opaque to clients
      • Easier to invalidate, track, and optimize across client-types
    • Knowing requirements for load, freshness, and current metrics
    • HTTP caching with cache-control TTL and ETags
    • Caching for writes, such as write-behind cache
    • Caching for resilience in case of failure by providing stale data
    • Hiding the origin by having the origin populate the cache asynchronously - to protect the origin from cascading load
    • Keep it simple - by sticking to one only if needed
  • Autoscaling
  • CAP theorem
    • Consistency vs Availability vs Partition Tolerance
    • Can mix and match
  • Service Discovery
    • DNS
    • Zookeeper, Consul, Eureka
  • Documenting Services
    • Swagger, HAL

Ch 12. Final

  • Summary
    • Model around business concepts - bounded contexts
    • Automation culture - automated tests, continuous delivery, environment defs, custom images
    • Hide internal implementation details - bounded contexts, hide databases, event data pumps, technology-agnostic APIs
    • Decentralize all things - self-service, team ownership of services, Conway's law, shared governance, choreography over orchestration, dump middleware with smart endpoints
    • Deploy independently - coexisting versioned endpoints, one-service-per-host model, blue/green deployment, canary release, consumer-driven contracts, eliminate lock-step
    • Isolate failure - treat remote calls differently from local calls, anti-fragility, timeouts, bulkheads, circuit breakers, CAP
    • Highly observable - semantic monitoring, synthetic transactions, aggregate logs/stats, use correlation IDs
  • Advice
    • Go incrementally - make each decision small in scope.
    • Change is inevitable. Embrace it.