Domain-Driven Design Book Club Notes

 

Preface

Quotes

"As the team gained new insight into the domain, the model deepened. The quality of communication improved not only among developers but also between developers and domain experts, and the design—far from imposing an ever-heavier maintenance burden—became easier to modify and extend."

"The Extreme Programming process assumes that you can improve a design by refactoring, and that you will do this often and rapidly."

Part 1

Quotes

"It is not just the knowledge in a domain expert’s head; it is a rigorously organized and selective abstraction of that knowledge."

Model Usage

  1. The model and the heart of the design shape each other.
  2. The model is the backbone of a language used by all team members.
  3. The model is distilled knowledge.

Developer motivation

  • "Domain work is messy and demands a lot of complicated new knowledge that doesn’t seem to add to a computer scientist’s capabilities."
  • "Instead, the technical talent goes to work on elaborate frameworks, trying to solve domain problems with technology. Learning about and modeling the domain is left to others. Complexity in the heart of software has to be tackled head-on."
  • "There are systematic ways of thinking that developers can employ to search for insight and produce effective models. There are design techniques that can bring order to a sprawling software application. Cultivation of these skills makes a developer much more valuable, even in an initially unfamiliar domain."

Chapter 1 Crunching Knowledge and Chapter 2 Communication and the Use of Language

Quotes

"Design and Process are inextricable."

“The domain model will typically derive from the domain experts’ own jargon but will have been “cleaned up,” to have sharper, narrower definitions.”

“A document shouldn’t try to do what the code already does well. The code already supplies the detail. It is an exact specification of program behavior. Other documents need to illuminate meaning, to give insight into large-scale structures, and to focus attention on core elements. Documents can clarify design intent when the programming language does not support a straightforward implementation of a concept. Written documents should complement the code and the talking.”

“It takes fastidiousness to write code that doesn’t just do the right thing but also says the right thing.”

Discussion

 Click here to expand...
  • Spoke about examples of various previous experience with DDD.
    • Working with domain experts and using software to solve their problems.
    • Working to define the vocabulary, interactions and contexts for the domain in question.
  • Question: The book presumes there is a domain expert.  What do we do when there is an ideal but not a person that embodies the domain knowledge?
    • Online education is different but it's still teaching.
      • The existing domain experts for teaching is a good starting place for domain expertise
      • Fallacy that any one domain experts knows everything you need to know.
        • All knowledge is based on constraints of the persons experience
      • Current experts can help but there will always be an iterative process on top of that.
    • Domain experts don't know what the software is going to look like when it's "Done", its the people trying to use your software today.
    • To find a domain expert we need to figure out what the job needs to be done first.  Then we can figure out what the domain is and what experts we need.
    • Don't rely on just one domain expert.  No one person will have the full picture.
    Question: Which edX features would have benefited with DDD? (Course modes, cohorts/teams/groups)
    • Course Discussions
      • Discussion IDs, Commentable IDs, etc
        • Different names for the same thing
    • What's the right level to pick a domain?
      • Too close and we make decisions for how instructors teach making it harder to experiment.
      • Discussion vs discussion experimentation platform
      • Are we a course content delivery system or course content delivery experimentation platform?
        • Per recent product discovery, general consensus is that we are a delivery system first and an experiment platform second.
          • Building for generic experimentation is very broad and hard to provide value with
  • DDD could be used to expose gaps where one domain has dependencies on other systems that may not be clear until we think of the problem from a domain view.
  • Question: Could we say that a domain expert for us is our learner base?
    • Using our learners and rapid feedback loops to iterate.
  • Relying on an agreed upon vocabulary is really important. (Not be-all-end-all but crucial none the less).
    • Applying DDD to new things is easier,  how do we do it for existing software.
      • Supposedly a chapter that addresses that we haven't gotten to yet.
  • Question: Looking at the messy areas of our code, would DDD have helped or was the problem space just not clear?
    • We didn't have answers to a lot of basic business questions.
    • How do we go from no answers to having a defined domain?
      • Start with the existing data and define all the existing entities.
      • How do we map them and their interactions together?
      • What assumptions are we making.
      • What data do we have to start with?
  • For us whatever the current implementation is, is the model but not documented.
    • Do we need to document the model of the current implementation or just define what the model should be?
      • We probably need both the model of what we have now and the model we want to move to.
        • Need both so that we can know what we are moving from and what we are moving to.
  • Question: Is there a DDD Skeptic?
    • Many people.
    • Still unsure that DDD would have prevented things that we have done in the past.
    • DDD is supposed to help give us a common language so that we improve time to value.
    • We pay for it later if we don't have alignment.
    • Can we actually put this into practice?
      • The more you practice, the better you get at it.
  • Question: How do we decide what's in and what's out?
    • Conversation with stakeholders and others, realizing that it could change.
  • We were already doing DDD to some extent - even as creating models mentally.
  • Not bringing all legacy code up-to-date with the latest model, helps with moving fast.
    • Which is fine, because the model doesn't need to be knee deep into the entire system.
    • If we're never really touching old code that was built with an older model, then it's Ok to choose not to update it.
      • Analytics team was able to do this because their "workflows" were independent bounded contexts.
  • Question: Are batched workflows in the Analytics pipeline considered a domain model?
    • Since "pipeline" is part of the deliverable, it's reasonable to consider "workflows" to be part of the domain.
    • Consider an "event" from it's baby stage through it's life cycle.
    • For example, when they created the "django framework", a "request" would be part of "django's" domain.
    • There can be multiple "domains" within a system:
      • User domains driven by business needs versus Technical domains driven by technical framework needs.
    • Since the Analytics pipeline has technical customers, the Technical domain models are first-class models.
  • Different edX engineering teams will have different models.  For example:
    • Educator versus Learner on what a "Course" and "Course Run" are.
    • Release pipeline versus Analytics pipeline have a different concept on what "workflows" are.
  • Later in the book, Bounded Contexts should make things a lot clearer, hopefully.

Chapter 3 Binding Model and Implementation and Chapter 4 Isolating the Domain

Quotes

“Software development is all design. All teams have specialized roles for members, but over separation of responsibility for analysis, modeling, design, and programming interferes with MODEL-DRIVEN DESIGN.”

Example: “A user of Internet Explorer thinks of 'Favorites' as a list of names of Web sites that persist from session to session. But the implementation treats a Favorite as a file containing a URL, and whose filename is put in the Favorites list.”

“If the people who write the code do not feel responsible for the model, or don’t understand how to make the model work for an application, then the model has nothing to do with the software.”

“Every developer must be involved in some level of discussion about the model and have contact with domain experts.”

Discussion

 Click here to expand...
  • How would this apply to a functional programming language (author examines procedural and logical paradigms, but not FP)?
    • Artifact of when it was written?
    • Would probably work fine.
    • Not as layered.
    • Examples in JS of functional composition (e.g. React).
    • Django's MVC setup actually makes it harder.
      • Not a place for a lot of domain logic (e.g. utils.py)
      • One strategy: Separate layer to proxy lower level models.
      • Another strategy: Service pattern, service encapsulates business logic.
      • Example of what's talked in the book about frameworks dictating a lot of the design.
      • Grades app does have classes to encapsulate domain, but have to go without using niceties like DRF serializers.
      • Question: Are some of the tools that make it easy to write code making it harder to separate out domain logic?
        • Do we need a layer of our own/special tooling to enable this?
  • Analogs between the Smart UI pattern and Django's "Smart Model" and its conveniences.
    • Django optimizing for building it fast? Easy to leak implementation.
    • Useful middle ground?
    • Enterprise API – package that's installed, but API calls are via REST API as if it were a separate system.
      • Business logic encapsulated in models.
    • How to handle internal Python APIs?
      • Best practice to not pass back models?
      • api.py is the public interface between Django apps (though we're not sure if we want to continue using it)
      • Pass back the model but only allow certain methods to be called?
      • Write business logic as functional modules and use the model methods just as a proxy to that?
    • What is the problem we're solving?
      • Enrollment example
      • Separating domain logic from underlying details
      • Simplifying testing
  • Question: Difference between Application Layer and Domain Layer?
    • Application layer manages jobs and tasks and lifecycle of the application. Very thin.
    • Layer for orchestration.
    • No business logic, just knowing who to call.
    • Initialization of the application, setup, teardown.
    • Django "best practice" as view, but this is often not followed.
      • Need to have a clear location for business logic.
    • In async tasks (grades example), is the thing that kicks off task the app layer and the task itself the domain layer?
      • tasks.py is the Django interface
      • domain logic is in the classes: CourseGrade, CourseGradeFactory, SubsectionGrade, and SubsectionGradeFactory
    • Where does app end and domain begin?
      • Where the framework ends and you create domain layer concepts and return domain layer concepts.
  • Question: Are permissions domain logic or app?
    • The notion of "readable by superusers" (app) vs. domain concept of who should be a superuser?
    • Attached to the View in our code.
    • Mapping of of roles to permissions, permissions to views in Django.
    • Permission a Guard on the view, and therefore a separate Domain?
    • Are permissions for what can be done in a Domain a separate Domain?
    • Implementation of permission is app, where to attach it is Domain.
      • Combination, since there's a lot of permission checking in the domain itself.
    • Artifact of the fact that it's a web app, because the conceptual Domain wouldn't even allow certain methods to be invoked? Different objects for different kinds of people?
    • Purer form of this might be to just pass permissions down to the domain layer and let it make the decisions.
    • When will we know that we've outgrown Django?
      • Many of the things we run into are fundamental to the framework.
      • Gradually outgrow it by pieces (asset pipeline, templates, etc.)
    • Django not strictly object oriented, applying separation of concerns without adhering too strictly to DDD?
      • In the end, it's just Python, we can shape it as we need to.
        • Easy to do in a small team, harder to do at large.
          • Especially at open source.
    • Workflow as business logic, app layer just doing I/O, piping to the next thing.
  • Monolith and Folder Structure: Anyone interested in figuring this out, contact Nimisha Asthagiri (Deactivated)
    • First a template, then a script to generate that structure (manage.py – startapp at a specific URL already supported?)
  • Layers and directionality of layers
    • We should think about separation of communication and not just arrangement of app.
    • "You are talking to way too many places in the code."
      • Compositional components
    • Are we missing anything by having Django apps that cross all four layers?
      • Just a matter of discipline, would be confusing to separate out
      • Cross dependencies without directionality
      • Each Django app as its own Context
        • But part of a higher level tiering (different apps at different layers)
        • Middleware as an example of an app layer concern
          • But can be part of an app that spans all layers, e.g. language preference middleware that also has UI for users and model layer, etc.
            • How does this work for extraction, say pulling language related logic into a separate service?
            • App as glue, so app can get the language preference and pass it to the next thing that cares about language.
            • Common anti-pattern: Passing the request object instead of the things you need from it.
            • Service pattern, letting the Domain query from a service as it's needed.
            • Should we need to pass it down through the layers? Should things near the bottom need to know how to access services?
            • Pass in the service you need to use (for in-process services) to make testing easier
            • Don't like to pass the same parameters over and over again
            • Not opposed to passing the model (User instead of say six fields on the user)
              • If you're always sending the same parameters together, it's probably its own model
              • Difference between passing around simple structure vs. smart object that can make queries
              • Be transparent whether a call is a blocking call to an external service
        • Tracking and analytics are infrastructure layers

Chapter 5 A Model Expressed in Software

Quotes

Intro

Does an object represent something with continuity and identity - something that is tracked through different states or even across different implementations? Or is it an attribute that describes the state of something else? This is the basic distinction between an ENTITY and a VALUE OBJECT. Defining objects that clearly follow one pattern or the other makes the objects less ambiguous and lays out the path toward specific choices for robust design.

Then there are those aspects of the domain that are more clearly expressed as actions or operations, rather than as objects. Although it is a slight departure from object-oriented modeling tradition, it is often best to express these as SERVICES, rather than forcing responsibility for an operation onto some ENTITY or VALUE OBJECT. A SERVICE is something that is done for a client on request. They emerge .. when some activity is modeled that corresponds to something the software must do, but does not correspond with state.

Associations

There are at least three ways of making associations more tractable:

  1. Imposing a traversal direction
  2. Adding a qualifier, effectively reducing multiplicity
  3. Eliminating nonessential associations

It is important to constrain relationships as much as possible. A bidirectional association means that both objects can be understood only together. When application requirements do not call for traversal in both directions, adding a traversal direction reduces interdependence and simplifies the design. Understanding the domain may reveal a natural directional bias.

Constraining the traversal direction of a many-to-many association effectively reduces its implementation to one-to-many - a much easier design.

Entity

An object defined primarily by its identity is called an ENTITY.  ENTITIES have special modeling and design considerations. They have life cycles that can radically change their form and content, but a thread of continuity must be maintained. Their identities must be defined so that they can be effectively tracked. Their class definitions, responsibilities, attributes, and associations should revolve around who they are, rather than the particular attributes they carry. Even for ENTITIES that don't transform so radically or have such complicated life cycles, placing them in the semantic category leads to more lucid models and more robust implementations.

When an object is distinguished by its identity, rather than its attributes, make this primary to its definition in the model. Keep the class definition simple and focused on life cycle continuity and identity.

Value Objects

Software design is a constant battle with complexity. We must make distinctions so that special handling is applied only where necessary. VALUE OBJECTS are instantiated to represent elements of the design that we care about only for what they are, not who or which they are.

When you care only about the attributes of an element of the model, classify it as a VALUE OBJECT. Make it express the meaning of the attributes it conveys and give it related functionality. Treat the VALUE OBJECT as immutable. Don't give it any identity and avoid the design complexities necessary to maintain ENTITIES.

As long as a VALUE OBJECT is immutable, change management is simple - there isn't any change except full replacement. Immutable objects can be freely shared, as in the electrical outlet example. If garbage collection is reliable, deletion is just a matter of dropping all references.

Try to completely eliminate bidirectional associations between VALUE OBJECTS.

Services

A SERVICE is an operation offered as an interface that stands alone in the model, without encapsulating state, as ENTITIES and VALUE OBJECTS do. SERVICES are a common pattern in technical frameworks, but they can also apply in the domain layer.  They are intrinsically activities or actions, not things.

A good SERVICE has three characteristics.

  1. The operation relates to a domain concept that is not a natural part of an ENTITY or VALUE OBJECT.
  2. The interface is defined in terms of other elements of the domain model.
  3. The operation is stateless.

It takes care to distinguish SERVICES that belong to the domain layer from those of other layers, and to factor responsibilities to keep that distinction sharp.  Layers:

  • Application service
  • Domain service
  • Infrastructure service

Granularity

Medium-grained, stateless SERVICES can be easier to reuse in large systems because they encapsulate significant functionality behind a simple interface. Also, fine-grained objects can lead to inefficient messaging in a distributed system.

As previously discussed, fine-grained domain objects can contribute to knowledge leaks from the domain into the application layer, where the domain object's behavior is coordinated. The complexity of a highly detailed interaction ends up being handled in the application layer, allowing domain knowledge to creep into the application or user interface code, where it is lost from the domain layer

Modules (Packages)

It is a truism that there should be low coupling between MODULES and high cohesion within them. Explanations of coupling and cohesion tend to make them sound like technical metrics, to be judged mechanically based on the distributions of associations and interactions. Yet it isn't just code being divided into MODULES, but concepts. There is a limit to how many things a person can think about at once (hence low coupling). Incoherent fragments of ideas are as hard to understand as an undifferentiated soup of ideas (hence high cohesion).

..tiered architectures can fragment the implementation of the model objects. Some frameworks create tiers by spreading the responsibilities of a single domain object across multiple objects and then placing those objects in separate packages. At that point, viewing the various objects and mentally fitting them back together as a single conceptual ENTITY is just too much effort.

Unless there is a real intention to distribute code on different servers, keep all the code that implements a single conceptual object in the same MODULE, if not the same object.

Discussion

 Click here to expand...
  • Entity vs Service discussion:
    • services are stateless
    • entities have an identification and state
  • Service examples:
    • Tracking service: Infrastructure level service
    • Grade service: Domain service
  • Event could be example of entity or value, depending on need.  Like address example in book.
  • Helpful to more explicitly look at the value objects and relationships.
  • Value objects: 
    • Emphasized immutability.
    • Is it just an Enum?  More like a struct.
    • Try to eliminate bi-directional relationships between values objects.
    • Can they do something?  Yes.
  • Opaque keys:
    • Value object themselves.  Intention to be immutable.
    • Used to identify Entities.
  • Granularity
    • Fine grained granularity leads to leakage.
    • Passing between layers, granularity makes a difference.
    • How to combat granularity?  Medium sized services to help combat.
    • Do we think this is true?  
      • REST APIs are difficult to use when too granular, 
        • Having endpoints that return everything, vs endpoints that return pieces at a time.
      • or
      • REST APIs with json.  GraphQL as example of making joins on the server.
        • Front-end vs Back-end knowledge of the needs of the front end.
    • Enrollments example:
      • Ask for all active enrollments of a student?  What are enrollments of this student with scheduling, etc.  Are the implementation details leaked out to the dashboard?  Want to avoid this.
  • Modules
    • Pitfalls of infrastructure driven packaging
    • Tiered architectures can fragment
    • Should tell the larger story of what is going on.
    • Should be defined in terms of the domain, not in terms of the infrastructure.
    • Important to not deprioritize refactoring to have your code match the model.
  • Ubiquitous Language
    • Can start small.
    • Do we want agreement to do for this?
    • Can be difficult when some people are strict and others don't care (past experience of some).
    • Refactor as you go.

Chapter 6  Life Cycle of a Domain Object

Quotes

Aggregates, Factories, Repositories

  • AGGREGATES tighten up the model itself by defining clear ownership and boundaries, avoiding a chaotic, tangled web of objects. This pattern is crucial to maintaining integrity in all phases of the life cycle.
  • using FACTORIES to create and reconstitute
  • REPOSITORIES address the middle and end of the life cycle, providing the means of finding and retrieving persistent objects while encapsulating the immense infrastructure involved.

Aggregates

Cluster the ENTITIES and VALUE OBJECTS into AGGREGATES and define boundaries around each. Choose one ENTITY to be the root of each AGGREGATE, and control all access to the objects inside the boundary through the root. Allow external objects to hold references to the root only. Transient references to internal members can be passed out for use within a single operation only. Because the root controls access, it cannot be blindsided by changes to the internals.

Example: denormalize price to satisfy Purchase Order Aggregate invariant.

Now, to translate that conceptual AGGREGATE into the implementation, we need a set of rules to apply to all transactions.
The root ENTITY has global identity and is ultimately responsible for checking invariants.
Root ENTITIES have global identity. ENTITIES inside the boundary have local identity, unique only within the AGGREGATE.
Nothing outside the AGGREGATE boundary can hold a reference to anything inside, except to the root ENTITY. The root ENTITY can hand references to the internal ENTITIES to other objects, but those objects can use them only transiently, and they may not hold on to the reference. The root may hand a copy of a VALUE OBJECT to another object, and it doesn't matter what happens to it, because it's just a VALUE and no longer will have any association with the AGGREGATE.

As a corollary to the previous rule, only AGGREGATE roots can be obtained directly with database queries. All other objects must be found by traversal of associations.
Objects within the AGGREGATE can hold references to other AGGREGATE roots.

A delete operation must remove everything within the AGGREGATE boundary at once. (With garbage collection, this is easy. Because there are no outside references to anything but the root, delete the root and everything else will be collected.)
When a change to any object within the AGGREGATE boundary is committed, all invariants of the whole AGGREGATE must be satisfied.

Factories

Creation of an object can be a major operation in itself, but complex assembly operations do not fit the responsibility of the created objects. Combining such responsibilities can produce ungainly designs that are hard to understand. Making the client direct construction muddies the design of the client, breaches encapsulation of the assembled object or AGGREGATE, and overly couples the client to the implementation of the created object.

The two basic requirements for any good FACTORY are:

  1. Each creation method is atomic and enforces all invariants of the created object or AGGREGATE. A FACTORY should only be able to produce an object in a consistent state.
  2. The FACTORY should be abstracted to the type desired, rather than the concrete class(es) created.

I refer to the creation of an instance from stored data as reconstitution.  A FACTORY used for reconstitution is very similar to one used for creation, with two major differences:

  1. An ENTITY FACTORY used for reconstitution does not assign a new tracking ID. To do so would lose the continuity with the object's previous incarnation.
  2. A FACTORY reconstituting an object will handle violation of an invariant differently.

Repositories

  1. Abstract the type of the object returned
  2. Take advantage of the decoupling from the client.
  3. Leave transaction control to the client.

The FACTORY makes new objects; the REPOSITORY finds old objects.
These two views can be reconciled by making the REPOSITORY delegate object creation to a FACTORY, which (in theory, though seldom in practice) could also be used to create objects from scratch.
One other case that drives people to combine FACTORY and REPOSITORY is the desire for find or create functionality, in which a client can describe an object it wants and, if no such object is found, will be given a newly created one. This function should be avoided.

A table row should contain an object, perhaps along with subsidiaries in an AGGREGATE. A foreign key in the table should translate to a reference to another ENTITY object. The necessity of sometimes deviating from this simple directness should not lead to total abandonment of the principle of simple mappings.

Questions

  1. I wonder why he recommends that the Repository be the primary interface for reconstituting objects - rather than letting the Factory be the primary.

Discussion

 Click here to expand...


  • find_or_create
    • Client code shouldn't necessarily call it, however it's useful to have
    • Room opinion: Doesn't seem like it has to be a hard or fast rule
    • At the SQL layer, you have to have it to know whether you're violating a constraint
    • We use find_or_create primarily for Value Objects
      • Example: CSM state
  • django
    • is entity focused - not aggregate focused
    • has custom support for repository via models.py and ORM - and model managers
    • but does not have factory support innately
    • factories would allow creation of 
    • Model managers are similar to repositories
      • but you can pass any SQL queries - so easy to break abstractions
      • But then again, his example of custom queries was starting to cross the line
        • should this really belong at the domain-model layer?  Or should the domain layer only contain like 4 high-level methods?
  • It's Ok to go straight to constructor for small cases
  • Repositories
    • Allow us to replace the underlying database easily
    • You're supposed to know about the underlying implementation, but not interact with it directly
    • Maintaining an illusion that things are in-memory
      • Dave Mohr (Unlicensed) says NO!!!  Because that can lead to scalability/performance issues
      • if read out of context, this could be misconstrued
      • In fact, this strongly argues for a Repository - since you can only include performant operations on the public interface
        • like exclude 'getAll' as an option to call
      • Sometimes overly hiding implementation implications from clients can lead to mis-use
      • Does the client need to implement an all-powerful error handling for all types of functions?
    • Repository should be returning errors that are in the domain, right?
    • He does point out that transaction handling should be done by the client
    • Modify the errors as they are propagated up the chain
  • Factories
    • Build a new domain model
    • Not build a new object whether or not it persisted before
  • Example: Registration versus sign-in
    • The registration process is very different from the sign-in process, regarding the 
  • Tour of Block Transformers (re: Factories and Repositories)
    • BlockStructureManager (get_transformed(), get_collected()) – accessed by client  # This is what DDD calls a Repository
    • BlockStructureFactory – internal class (create_from_modulestore, create_from_store)
    • BlockStructureStore – storage interface (crud)
    • Seems like we actually do follow the DDD pattern for Block Transformers, though we use different names. Grades might conflate factory and repository.
  • "Our codebase is not... great... for new developers," – Nimisha

Chapter 7 Example of Using the Language & 8 Breakthrough

Discussion

 Click here to expand...
  • Good examples of domain models:
    • Oscar's payments
  • Caching (p 178)
    • Can cause complexity
    • Do they live within the Repository abstraction?
    • Repository seems like a nice place to keep the technical details of a cache
  • Handling Event
    • What if it were separated out into its own service?
    • It's an interesting thought experiment - if the scale were so big and you needed it to be its own service.
    • The repository can make the external call and handle the caching.
    • Presumably there's a Handling Event factory - are they copied in each service or are they different?
  • Anti-corruption layer
    • Can mitigate discrepancies between interrelated services with different domain concepts
    • Each bounded context would have its own interface of the object
    • For example, the JSON as the messaging object that travels between the boundaries, but then each context forms its own interface
    • But do they share the same terminology - to follow the ubiquitous language?
    • is this always desired, even if "translation" is minimal?
      • also a way to protect the domain's invariants from external systems changing
    • would it be access point specific?
      • when B calls A, B may have multiple Anti-corruption layers to 
    • the bounded context owns this, not the external service
  • https://en.wikipedia.org/wiki/Cynefin_framework
    • Problems can be bucketed into these categories
    • Can apply rigid set of rules
    • Simple, Complicated, Complex, Chaotic, Disorder
    • This book seems to tackle problems in the Simple and Complicated states, but not from the other buckets
      • Tackling shipping, for example, is not as dynamic
    • Can DDD still be able to help us when changes happen rapidly?
      • Communications between Domain experts and Engineers → via Ubiquitous language and model revisioning
      • If changes happen very quickly, can we keep up by refactoring?
        • It may be possible, if you continuously refactor.
    • It's ok if your entire system is complex, as long as each individual part can be complicated/simple so you can reason with each part one at one time.
    • Overtime, your team's velocity will slow down as your system becomes more and more complex.
    • Not 'Truth', with a capital T - but a 'truth' that is relative to today's understanding of the model
  • What happens when the domain is unknown and there's no domain expert (yet)?
    • by the way, edX is not really developing a new science.
  • If our smaller pieces and building blocks can be simple/complicated, then it's possible that the larger system can remain complex and not become chaotic.
  • If we created a domain model for a Notifications service, and then an OSPR comes in - what requirements do we place on it?
  • Business is creating a model via OOUX - how do you develop a ubiquitous language and share it out?
    • glossary/wiki may not be enough
    • it's more about enforcing it informally in communications
    • the DDD model also provides a common mental model - so we are all aligned on the same thing
  • Robert Raposa's proposal for an experiment this week: ubiquitous language - give it a whirl!

Chapter 9 Making Implicit Concepts Explicit

Quotes

  • Experimentation is the way to learn what works and doesn't. Trying to avoid missteps in design will result in a lower quality result because it will be based on less experience. And it can easily take longer than a series of quick experiments.
  • The key to distinguishing a process that ought to be made explicit from one that should be hidden is simple: Is this something the domain experts talk about, or is it just part of the mechanism of the computer program?
  • Constraints and processes are two broad categories of model concepts that don't come leaping to mind when programming in an object-oriented language, yet they can really sharpen up a design once we start thinking about them as model elements.
  • Create explicit predicate-like VALUE OBJECTS for specialized purposes. A SPECIFICATION is a predicate that determines if an object does or does not satisfy some criteria.
  • The SPECIFICATION keeps the rule in the domain layer. Because the rule is a full-fledged object, the design can be a more explicit reflection of the model.

Discussion

 Click here to expand...
  • Digging Out Concepts
    • Robert's example: Progress API (Braden), 2.0 needed for McKinsey, we want to add onto it as well.
      • Conversation with Marco: How they were calculating some things and what Marco thought about that?
        • Marco was thinking about Grades.
        • Completion and Progress are two different things.
        • "On track for passing this course." vs. "Am I 80% done with the things that are in here?"
        • Possibilities: Renaming "Progress API" to "Completion API"? Separating the two concepts more generally?
        • Ubiquitous language experiment?
      • Smells of disconnect: "Progress means how much of the course you've completed."
      • Feanil: "Do you feel that we have a problem committing to a definition because it might change, or because it's true right now?"
        • Delayed defining what a course is with the consequence that we have a bunch of conflicting definitions of course.
    • Changing names is expensive (see: IDAs)
      • Example: Discovery! (Catalog)
      • We have to be able to take on some of these costs and push it out to the edges.
      • Easy to track the cost of changing it, hard to track the cost of the confusion.
      • What's more wasteful?
    • Better to concentrate on the ubiquitous language for new things rather than worry about pulling old things in?
      • RET: As they're designing a "schedule", is an enrollment specific thing that defines that user's schedule in that course. Increased speed of following conversations.
    • Glossary? Robert tried to do this for forums.
      • Verticals
      • Units
      • Chapter vs. Section
      • Multiple names already exist in our language
    • Difficulties in changing
      • API contracts
      • Open source community concerns
        • We're going to cross this eventually
        • How does this compare to other issues though, like Old Mongo.
    • (xmodule comments – write)
  • Constraints
    •  Making them explicit! Woohoo! (Dave is taking notes)
  • Specifications
    • Example: What does it mean for a sequence to be "complete"?
    • Gabe: Doesn't seem as clear a win as Repository
      • Rules that might need to look at other things, but a way to keep it from growing too many dependencies.
      • How deep does it go to get simple answers?
      • Makes sense for more complex things (e.g. progress, gating).
      • RET Example: Based on your schedule, we'll send you emails.
        • "If you're two days before your verified deadline"
        • Is making specifications overkill at this point?
        • Reusability, composability
        • Testability
        • Appropriate for things that you know are going to grow in complexity
  • Processes as Domain Objects

Chapter 10 Supple Design

Quotes

  • Intention-revealing interfaces allow clients to present objects as units of meaning rather than just mechanisms. 
  • Side-effect-free functions and Assertions make it safe to use those units and make complex combinations. 
  • The emergence of Conceptual Contours stabilizes parts of the model and also makes the units more intuitive to use and combine.

Intention-revealing interfaces

  • If a developer must consider the implementation of a component in order to use it, the value of encapsulation is lost. 

  • Name classes and operations to describe their effect and purpose, without reference to the means by which they do what they promise. 

Side-effect-free functions

  • Most operations call on other operations, and those called invoke still other operations. As soon as this arbitrarily deep nesting is involved, it becomes very hard to anticipate all the consequences of invoking an operation. 

  • Functions are much easier to test than operations that have side effects. For these reasons, functions lower risk.

  • Strictly segregate commands (methods that result in modifications to observable state) into very simple operations that do not return domain information. Further control side effects by moving complex logic into VALUE OBJECTS when a concept fitting the responsibility presents itself.

Assertions

  • ASSERTIONS make side effects explicit and easier to deal with.

  • All these assertions describe state, not procedures, so they are easier to analyze.

  • State post-conditions of operations and invariants of classes and AGGREGATES. If ASSERTIONS cannot be coded directly in your programming language, write automated unit tests for them. Write them into documentation or diagrams where it fits the style of the project’s development process.

Conceptual Contours

  • One reason why repeated refactoring eventually leads to suppleness: the CONCEPTUAL CONTOURS emerge as the code is adapted to newly understood concepts or requirements. 

  • Decompose design elements (operations, interfaces, classes, and AGGREGATES) into cohesive units, taking into consideration your intuition of the important divisions in the domain.  

  • The goal is a simple set of interfaces that combine logically to make sensible statements in the UBIQUITOUS LANGUAGE, and without the distraction and maintenance burden of irrelevant options. This is typically an outcome of refactoring: it’s hard to produce up front. But it may never emerge from technically oriented refactoring; it emerges from refactoring toward deeper insight.

Standalone Classes

  • Low coupling is fundamental to object design. When you can, go all the way. Eliminate all other concepts from the picture. Then the class will be completely self-contained and can be studied and understood alone. Every such self-contained class significantly eases the burden of understanding a MODULE

  • The goal is not to eliminate all dependencies, but to eliminate all nonessential ones. If every dependency can’t be eliminated, each one that is removed frees the developer to concentrate on the remaining conceptual dependencies. 

  • Low coupling is a basic way to reduce conceptual overload. A STANDALONE CLASS is an extreme of low coupling. 

Closure of Operations

  • Most interesting objects end up doing things that can’t be characterized by primitives alone. 

  • Where it fits, define an operation whose return type is the same as the type of its argument(s). 

  • If the implementer has state that is used in the computation, then the implementer is effectively an argument of the operation, so the argument(s) and return value should be of the same type as the implementer. Such an operation is closed under the set of instances of that type. 

  • A closed operation provides a high-level interface without introducing any dependency on other concepts. 

Declarative Style of Design

  • NOT Declarative Design because of its many pitfalls (see chapter).
  • Once your design has INTENTION-REVEALING INTERFACES, SIDE- EFFECT-FREE FUNCTIONS, and ASSERTIONS, you are edging into declarative territory. Many of the benefits of declarative design are obtained once you have combinable elements that communicate their meaning, and have characterized or obvious effects, or no observable effects at all.

  • A supple design can make it possible for the client code to use a declarative style of design. 

Discussion

 Click here to expand...
  • How is 'Conceptual Contours" related to Bounded Contexts and Aggregates?  Or are they all similar constructs related to encapsulation, cohesion, and decoupling?
    • Bounded contexts separate objects that use different ubiquitous language.
    • Is there anything to learn from Conceptual contours or is it just a nice phrase?
      • Doesn't seem like it was super important
    • Don't define the contours based on technologies but based on the domain.  Use the concepts of the domain to define the contours of what you are doing.
    • When making refactoring changes, think about doing it from a domain POV not a technical POV.
  • How can we implement Assertions in Python?
    • Type System is an assertion
    • Maybe as a decorator
    • An assertion makes sure that any new client of your code will obey the rule but a unit test is specific to how a specific interface.
    • Assertions in the book are specifically around testing state side-effects not necessarily around typing
    • We do it in test code.  Not something to do in the running code.
    • We can't easily run assertions in production because they are very expensive.
      • PyContracts has previously done damage in production.
    • Summary: Unit Tests + Asserts in production
  • How do we translate the lessons of this chapter to what we do do?
    • How do we make new functions better, make it clear that it doesn't have side-effects.
      • Have to declare both because we have an existing codebase
      • Just need a convention to make this clear.
      • Decorator for side-effect functions
        • Could be used to figure out if a function calls side-effecting functions etc.
        • Seems really complicated.
    • Summary: Docstrings for indicating seems like a good start.
  • What are others' experiences with Declarative Design - did you face the same pitfalls he describes?
    • If you have a declarative language that is not expressive you are forced into the do what the language wants.

Chapter 11 Applying Analysis Patterns

Quotes

  • If you wait until you can make a complete justification for a change, you’ve waited too long.

Discussion

 Click here to expand...
  • Firing types
    • Accrual versus payment types - use different firing types
    • Take the domain approach to something you don't normally think about - like nightly running a script
    • 1. Eager Firing
      • Can be async or sync
      • For example, if we update the learner's grade immediately after the learner submits a problem
    • 2. Account-based Firing
      • Might be akin to a Course in our world
      • Top-level domain model that receives the trigger
    • 3. Posting-Rule-based Firing
      • Closer to how Redux works since the recipient is decoupled from the trigger
      • For example, could be a grading report that gets kicked off on a regular basis
    • The big point here is how decoupled the sender is from the receiver.
  • Chapter 12: Don't just blindly apply Design Patterns to your Domain problems

Part 4 Strategic Design

  • These three principles, useful separately but particularly powerful taken together, help to produce good designs even in a sprawling system that no one completely understands. Large-scale structure brings consistency to disparate parts to help those parts mesh. Structure and distillation make the complex relationships between parts comprehensible while keeping the big picture in view. BOUNDED CONTEXTS allow work to proceed in different parts without corrupting the model or unintentionally fragmenting it.
    • Context, the least obvious of the principles, is actually the most fundamental. A successful model, large or small, has to be logically consistent throughout, without contradictory or overlapping definitions. By explicitly defining a BOUNDED CONTEXT within which a model applies and then, when necessary, defining its relationship with other contexts, the modeler can avoid bastardizing the model.
    • Strategic distillation can bring clarity to a large model. And with a clearer view, the design of the CORE DOMAIN can be made more useful.
    • Large-scale structure completes the picture. In a very complex model, you may not see the forest for the trees. Distillation helps, by focusing the attention on the core and presenting the other elements in their supporting roles, but the relationships can still be too confusing without an overarching theme, applying some system-wide design elements and patterns.

Chapter 14 Maintaining Model Integrity (1st half)

Quotes

  • Total unification of the domain model for a large system will not be feasible or cost-effective.
    • It is necessary to allow multiple models to develop in different parts of the system, but we need to make careful choices about which parts of the system will be allowed to diverge and what their relationship to each other will be.
  • It all starts with mapping the current terrain of the project. A BOUNDED CONTEXT defines the range of applicability of each model, while a CONTEXT MAP gives a global overview of the project's contexts and the relationships between them. This reduction of ambiguity will, in and of itself, change the way things happen on the project, but it isn't necessarily enough. Once we have a CONTEXT BOUNDED, a process of CONTINUOUS INTEGRATION will keep the model unified.
  • Then, starting from this stable situation, we can start to migrate toward more effective strategies for BOUNDING CONTEXTS and relating them, ranging from closely allied contexts with SHARED KERNELS to loosely coupled models that go their SEPARATE WAYS.

  • Explicitly define the context within which a model applies. Explicitly set boundaries in terms of team organization, usage within specific parts of the application, and physical manifestations such as code bases and database schemas. Keep the model strictly consistent within these bounds, but don't be distracted or confused by issues outside.
  • Duplication of concepts means that there are two model elements (and attendant implementations) that actually represent the same concept.
    False cognates is the case when two people who are using the same term (or implemented object) think they are talking about the same thing, but really are not.
  • Institute a process of merging all code and other implementation artifacts frequently, with automated tests to flag fragmentation quickly. Relentlessly exercise the UBIQUITOUS LANGUAGE to hammer out a shared view of the model as the concepts evolve in different people's heads.
  • Describe the points of contact between the models, outlining explicit translation for any communication and highlighting any sharing. Map the existing terrain. Take up transformations later.

Discussion

 Click here to expand...
  • Should CMS and LMS be bounded contexts?  Or are edX bounded contexts our major features?
    • LMS and CMS are 2 different applications, (probably) sharing the same domain.
    • Where one is the editor and the other is a viewer.
    • According to book, different bounded contexts would have different models and different implementation tools.
    • Might be good to experiment with creating a context map to see what models are in each (CMS/LMS).
    • Naturally what's happened is that certain models (such as Course) exist in different places: e-commerce, lms, cms, etc.
      • No unified model across everywhere.  So the concept of Course could be different in each context.
    • edX core domain models - per Slide 38 of the open-edx arch presentation
      • we defined bounded contexts as high-level 'core' concepts such as Course, User, Progress, Discussions
      • while LMS and Studio ended up being simply UI-level applications, with Integration at the UI layer.
      • lower-level contexts (without any UI really) are 'supporting' contexts, such as Grades, Block Structures, Modulestore.
  • What are edX examples of Shared Kernel, Customer/Supplier, and Conformists?
    • Conformist
      • Oscar - where it's a 3rd party provider and we don't have control over the provider's side.
        • Translation must happen at the consumer side.
        • We pulled it off-the-shelf and conforming to it.
      • 3rd party social auth
        • We did contribute a few things back to them, but we're mostly using their package.
    • Shared Kernel
      • Unintentionally: Monolith!!!
      • We considered Shared Kernel approach for the Insights stack, but decided not to pip install edx-platform.
    • Customer/Supplier
      • Analytics pipeline is a customer of the edx-platform, with the tracking layer as the translation layer.
      • Video pipeline (VEDA) is a supplier to the edx-platform - although one could debate whether VEDA is 'supporting' or 'core'.
      • Comments Service is a supplier to LMS - unclear whether it's 'supporting' or 'core'.
      • Discovery Service is a supplier to various Bounded Contexts - as a 'supporting'.
      • e-commerce Service is a supplier.
        • supporting or core?
          • According to our OKRs, it's core.
          • But, we could also substitute it with Stripe and make it 'generic'.
        • Coupons relationship to Enterprise learners
          • there's business logic there that is custom today because of Oscar, could have been different if we used Stripe.
          • we may be bending the model to fit the tool.
  • Process by which you define bounded contexts.
    • For example, learner and educator divide.
    • According to the book, alignment of teams to domain models.
    • Currently, it's around functional boundaries rather than domain boundaries.
    • Chicken and egg problem?  Have code match the team?  Or, have the team match the code?
    • Today, when we try to align with product divisions, it gets difficult since developers are all over the place - and there's no long-term ownership of the domain.
  • Later chapter will distinguish between core (80% of the system that's part of the overall secret sauce), support, and generic.
  • This chapter was probably the most useful so far.  His relationships and architectural patterns really resonated.
  • Supplier gets to dictate the shape of what they provide - they may also have many consumers.
    • Can become conformist then.
    • For internal teams within companies, one would expect/hope for more shared-kernels and customer/supplier relationships.
  • It's unclear whether Course Content would be its own bounded context.
    • Since you can't do anything by itself.
    • Can distinguish this (Course Creation) from Course Taking and from Course Buying and possibly have them have a Shared Kernel.
  • Does a domain contain multiple bounded contexts, or does a bounded context contain multiple domains?
    • A bounded context has its own unified domain model with its own ubiqutious language.
      • Within a bounded context, everything needs to be consistent (model, language).  Across contexts, all bets are off.
    • There's a single domain in which Courses can exist, but the models within each Context can differ.
    • A Shared Kernel is an exception here, because it's shared and ubiquitous across contexts - at least between the contexts that use the Kernel.
      • For example, if User were a Shared Kernel.
    • The domain is visualized by the Context Map - which shows the high-level contexts and their relationships.
    • domain = A sphere of knowledge, influence, or activity.
    • model = A system of abstractions that describes selected aspects of a domain and can be used to solve problems related to that domain.

Chapter 14 Maintaining Model Integrity (2nd half)

Quotes

  • ANTICORRUPTION LAYER
    • Create an isolating layer to provide clients with functionality in terms of their own domain model. The layer talks to the other sys- tem through its existing interface, requiring little or no modification to the other system. Internally, the layer translates in both directions as necessary between the two models.
    • One way of organizing the design of the ANTICORRUPTION LAYER is as a combination of FACADES, ADAPTERS, and translators, along with the communication and transport mechanisms usually needed to talk between systems.
    • The FACADE belongs in the BOUNDED CONTEXT of the other system.
  • SEPARATE WAYS
    • Integration is always expensive. Sometimes the benefit is small.
      Declare a BOUNDED CONTEXT to have no connection to the others at all, allowing developers to find simple, specialized solutions within this small scope.
      The features can still be organized in middleware or the UI layer, but there will be no sharing of logic, and an absolute minimum of data transfer through translation layers—preferably none.
    • Taking SEPARATE WAYS forecloses some options. Although continuous refactoring can eventually undo any decision, it is hard to merge models that have developed in complete isolation.
  • OPEN HOST SERVICE
    • Define a protocol that gives access to your subsystem as a set of SERVICES. Open the protocol so that all who need to integrate with you can use it. Enhance and expand the protocol to handle new inte- gration requirements, except when a single team has idiosyncratic needs. Then, use a one-off translator to augment the protocol for that special case so that the shared protocol can stay simple and coherent.
  • PUBLISHED LANGUAGE
    • Use a well-documented shared language that can express the necessary domain information as a common medium of communication, translating as necessary into and out of that language.
  • Model Context Strategy
    • Generally speaking, there is a correspondence of one team per BOUNDED CONTEXT. One team can maintain multiple BOUNDED CONTEXTS, but it is hard (though not impossible) for multiple teams to work on one together.

Discussion

 Click here to expand...
  • The anti-corruption diagram above is amusing
    • showcases the engineer's mindset about how "the system under design" is expected to be the new good stuff compared to the older legacy "other subsystem".
  • Anti-corruption layer example from previous job
    • legacy code that integrated with Facebook's AD API, which constantly changed and difficult to keep up-to-speed
    • so they created an anti-corruption layer as an intermediary, as they were a Conformist
    • but sometimes it still broke down since there sometimes wasn't a mismatch in the model.
    • every quarter they had to update the translations - and tried to contain the logic within that layer
    • couldn't really be a Conformist - since they were in a legacy system that was hard to refactor
  • Tradeoffs in the size of the bounded contexts
  • Typically, a bounded context is owned by a single team
  • Enterprise repo
    • More Conformist than Anti-corruption or Separate Ways
    • For platform, try to be Conformist in order not to change the core course-experience
    • For discovery-service, could be Customer/Supplier
      • things are specific to edx.org and may not apply to enterprise or affiliates
      • it's not clear who owns discovery-service, but enterprise is currently a consumer
  • Unifying an elephant
    • Different perspectives → can have individual different bounded contexts → can strive for a common model → may or may not arrive at the elephant
  • Model Context strategy
    • not a how-to-guide, but provides guidelines for trade-offs that would need to be made
  • Transformations
    • If we had bounded contexts with consistent models and isolation, then:
      • Generally speaking, breaking up CONTEXTS is pretty easy, but merging them or changing the relationships between them is challenging.
  • What should be the next steps?
    • openedx/Features directory versus LMS directory
      • unclear on which code should go in the Features directory
      • not just the newest place to put stuff
    • current issues
      • poor organization
      • split across modules
      • cognitive load
      • hard to discover/find code
      • multiple implementations of features
      • large modules
    • when people encounter things that don't belong somewhere, why don't they move/refactor it?
      • is it because they don't know what the vision is?
    • do we need to first align on a domain, before defining the bounded contexts?
    • if we have edx-platform be a pip-installable app, then it will help splitting up modules into their own repo.
    • proposal:
      • finish the book
      • follow-up with discussions with this group to develop the domain and bounded contexts (start with 1 or 2 more meetings)
      • will help us cement this topic and give us a jump-start
      • start at the top-level to determine the higher-level context map

Chapter 15 Distillation

Quotes

  • A simple DOMAIN VISION STATEMENT communicates the basic concepts and their value with a minimum investment. The HIGHLIGHTED CORE can improve communication and help guide decision making—and still requires little or no modification to the design. More aggressive refactoring and repackaging explicitly separate GENERIC SUBDOMAINS, which can then be dealt with individually.
  • COHESIVE MECHANISMS can be encapsulated with versatile, communicative, and supple design. Removing these distractions disentangles the CORE. Repackaging a SEGREGATED CORE makes the CORE directly visible, even in the code, and facilitates future work on the CORE model. And most ambitious is the ABSTRACT CORE, which expresses the most fundamental concepts and relationships in a pure form (and requires extensive reorganizing and refactoring of the model).
  • Factoring out GENERIC SUBDOMAINS reduces clutter, and COHESIVE MECHANISMS serve to encapsulate complex operations. This leaves behind a more focused model, with fewer distractions that add no particular value to the way users conduct their activities. But you are unlikely ever to find good homes for everything in the domain model that is not CORE. The SEGREGATED CORE takes a direct approach to structurally marking off the CORE DOMAIN.

Core Domain

  • Boil the model down. Find the CORE DOMAIN and provide a means of easily distinguishing it from the mass of supporting model and code. Make the CORE small.
  • Spend the effort in the CORE to find a deep model and develop a supple design—sufficient to fulfill the vision of the system. Justify investment in any other part by how it supports the distilled CORE.
  • But scarce, highly skilled developers tend to gravitate to technical infrastructure or neatly definable domain problems that can be understood without specialized domain knowledge. Apply top talent to the CORE DOMAIN, and recruit accordingly.
  • If you need to keep some aspect of your design secret as a competitive advantage, it is the CORE DOMAIN. And whenever a choice has to be made (due to time limitations) between two desirable refactorings, the one that most affects the CORE DOMAIN should be chosen first.

Generic Subdomain

  • Identify cohesive subdomains that are not the motivation for your project. Factor out generic models of these subdomains and place them in separate MODULES. Leave no trace of your specialties in them.
  • Once they have been separated, give their continuing develop- ment lower priority than the CORE DOMAIN, and avoid assigning your core developers to the tasks (because they will gain little do- main knowledge from them). Also consider off-the-shelf solutions or published models for these GENERIC SUBDOMAINS.
  • Options:
    • Off-the-shelf
    • Published design or model
    • Outsourced
    • In-house (possibly by temps)
  • Though you should seldom design for reusability, you must be strict about keeping within the generic concept.


Domain Vision Statement

  • Write a short description (about one page) of the Core Domain and the value it will bring, the “value proposition.”
  • A DOMAIN VISION STATEMENT gives the team a shared direction. Some bridge between the high-level STATEMENT and the full detail of the code or model will usually be needed.

Highlighted Core

  • The CORE DOMAIN must be made easier to see. The mental labor of constantly filtering the model to identify the key parts absorbs concentration better spent on design thinking, and it requires comprehensive knowledge of the model.

  • Significant structural changes to the code are the ideal way of identifying the CORE DOMAIN, but they are not always practical in the short term. In fact, such major code changes are difficult to un- dertake without the very view the team is lacking.

  • 2 options:
    • Distillation Document - Write a very brief document (three to seven sparse pages) that describes the CORE DOMAIN and the primary interactions among CORE elements.
    • Flagged CORE - Flag each element of the CORE DOMAIN within the primary repository of the model, without particularly trying to elucidate its role. Make it effortless for a developer to know what is in or out of the CORE.

Cohesive Mechanisms

  • Hide complex algorithms in methods with intention-revealing names, separating the “what” from the “how.”

  • Computations sometimes reach a level of complexity that begins to bloat the design. The conceptual “what” is swamped by the mechanistic “how.”

  • Partition a conceptually COHESIVE MECHANISM into a separate lightweight framework. Expose the capabilities of the framework with an INTENTION-REVEALING INTERFACE.

Segregated Core

  • Refactor the model to separate the CORE concepts from supporting players (including ill-defined ones) and strengthen the cohesion of the CORE while reducing its coupling to other code.
  • Factor all generic or supporting elements into other objects and place them into other packages, even if this means refactoring the model in ways that separate highly coupled elements.
  • Steps:
    1. Identify a CORE subdomain (possibly drawing from the distilla- tion document).
    2. Move related classes to a new MODULE, named for the concept that relates them.
    3. Refactor code to sever data and functionality that are not directly expressions of the concept. Put the removed aspects into (possibly new) classes in other packages. Try to place them with conceptually related tasks, but don’t waste too much time being perfect. Keep focused on scrubbing the CORE subdomain and making the references from it to other packages explicit and self-explanatory.
    4. Refactor the newly SEGREGATED CORE MODULE to make its relationships and interactions simpler and more communicative, and to minimize and clarify its relationships with other MODULES. (This becomes an ongoing refactoring objective.)
    5. Repeat with another CORE subdomain until the SEGREGATED CORE is complete.

Abstract Core

  • Identify the most fundamental concepts in the model and factor them into distinct classes, abstract classes, or interfaces.
  • Design this abstract model so that it expresses most of the interaction between significant components.
  • Place this abstract overall model in its own MODULE, while the specialized, detailed implementation classes are left in their own MODULES defined by subdomain.

Choosing Refactoring Targets

  1. In a pain-driven refactoring, you look to see if the root involves the CORE DOMAIN or the relationship of the CORE to a support- ing element. If it does, you bite the bullet and fix that first.
  2. When you have the luxury of refactoring freely, you focus first on better factoring of the CORE DOMAIN, on improving the segregation of the CORE, and on purifying supporting subdomains to be GENERIC.

Discussion

 Click here to expand...
  • What is segregated core?
    • Move the core into an isolated location so that its very clear what it is
  • We are thinking this chapter should have come earlier in the book
    • This chapter is the core domain!
    • It did feel verbose
  • Choosing refactoring targets
    • Identify the core domain and start there
    • Focus on the core
  • Developers solving generic problems are not delivering nearly as much value to the business as those who are working on the core
  • Do we have a course model?
    • Kinda? CourseOverviews? CourseModule? Catalog/Publisher?
  • Where do we want to be flexible and where do we want to be fixtured?
    • Too much flexibilty
  • Question on page 398 figure 15.1
    • Is all of this a bounded context? Should we be doing this at the level of our domain? Or our business?
    • Is there one core per bounded context?
    • Take e-commerce
      • You might have core things you really care about within e-commerce
      • Everything about e-commerce might be outside of our core
      • George thinks its a supporting function - not core
      • We can be less rigorous, outsource etc
    • Figure out what our core domain is
    • A core domain might span multiple bounded contexts
    • Split up edX into a core domain, supporting domain, generic domain
      • Each one of these might have multiple bounded contexts
      • A bounded context would not span across core and generic for example
    • Rule of thumb: core should be 20% of your code base
    • You can break things up by modules
  • Figuring out what's core and not core could be done at a macro or micro level
  • Domain vision sounds useful to us
  • There is a more detailed document that talks about the domain 
    • Highlighed core tells us what the true core is within the domain
  • Extracting the core should be a goal in-and-of itself to clarify what is the core
  • You often have to add in a bunch of cruft in order to separate stuff out
  • Having a segregated core is more important than everything else
    • Pay the price! It's worth it.
    • Moving it into its own bounded context is important since it allows you to aggressively work on the core and limit the splash damage.
  • Core, Supporting and Generic are separate domains
  • Having a model that spans the bounded contexts is a problem that needs to be solved
  • This chapter is mostly about how to get started with where we are with the ultimate goal of having domain models that match your code
  • Your core must be beautiful
  • Try to identify the core so that you can put most of your effort into that
    • Hard to figure out what's valuable and prioritize
  • What and where should edX core domain go?
    • Should it be in edx-platform? Or should we put it somewhere else?
  • What is edX's core?
    • What's edX's competitive advantage? What does our business think is our differentiator?
    • Proposal: we provide high quality, rigorous education to a massive audience
    • We need the business and development to be aligned about what the core actually is
    • Could help us build business cases for refactoring
    • We aren't behind a paywall
      • Why aren't our courses indexed by google?
      • We could change our URL structures to make our content rank better in search results
    • Bug triage becomes a simpler
      • Is it core? → CAT-1 otherwise CAT-3
    • Let's take a cut at it - strawman
  • Take discussions
    • Some people might say it's not in the core
    • would that be useful for making certain decisions? if we defined it?
    • How do we resolve disagreements
  • It's easy to get distracted by generic and supporting stuff
  • Generic Subdomain versus Cohesive Mechanism
    • Cohesive mechansim is essentially removing any complicated algorithms (into even helper functions) from the core domain
    • Generic Subdomain is like the modulestore, which is a generic domain that can be used by the core domain or multiple domains
  • Abstract Core
  • Re-read the Segregated Core
    • severing relationships
    • will impact many developers
    • end-goal is to have a module that is your core and nothing else
  • Discussions around tradeoffs between extracting the segregated core versus removing generic domains; whether we could use both tactics at once.

Chapter 16 Large Scale Structures

Quotes

  • A “large-scale structure” is a language that lets you discuss and understand the system in broad strokes.
  • Devise a pattern of rules or roles and relationships that will span the entire system and that allows some understanding of each part’s place in the whole—even without detailed knowledge of the part’s responsibility.
  • In a large system without any overarching principle that allows elements to be interpreted in terms of their role in patterns that span the whole design, developers cannot see the forest for the trees.
  • One key to keeping the cost down is to keep the structure simple and lightweight. Don’t attempt to be comprehensive.

Evolving Order

  • Let this conceptual large-scale structure evolve with the application, possibly changing to a completely different type of structure along the way.
  • Don’t overconstrain the detailed design and model decisions that must be made with detailed knowledge.

System Metaphor

  • When a concrete analogy to the system emerges that captures the imagination of team members and seems to lead thinking in a useful direction, adopt it as a large-scale structure. Organize the design around this metaphor and absorb it into the UBIQUITOUS LANGUAGE.

  • But because all metaphors are inexact, continually reexamine the metaphor for overextension or inapt- ness, and be ready to drop it if it gets in the way.

Responsibility Layers

  • Look at the conceptual dependencies in your model and the varying rates and sources of change of different parts of your domain.
  • If you identify natural strata in the domain, cast them as broad abstract responsibilities. These responsibilities should tell a story of the high-level purpose and design of your system.

Knowledge Level

  • In an application in which the roles and relationships between ENTITIES vary in different situations, complexity can explode. Neither fully general models nor highly customized ones serve the users’ needs.
  • KNOWLEDGE LEVEL untangles things when we need to let some part of the model itself be plastic in the user’s hands yet constrained by a broader set of rules.
  • Create a distinct set of objects that can be used to describe and constrain the structure and behavior of the basic model.
  • Keep these concerns separate as two “levels,” one very concrete, the other reflecting rules and knowledge that a user or superuser is able to customize.
  • Like all powerful ideas, REFLECTION and KNOWLEDGE LEVELS can be intoxicating. This pattern should be used sparingly.

Pluggable Component Framework

  • Distill an ABSTRACT CORE of interfaces and interactions and create a framework that allows diverse implementations of those interfaces to be freely substituted.
  • Likewise, allow any application to use those components, so long as it operates strictly through the interfaces of the ABSTRACT CORE.

Discussion

 Click here to expand...
  • Layering
    • Do these things actually change at different rates?
    • What would these layers even be in the edX context?
    • Why is it easy for us to break up technical things into layers? But domain stuff is harder?
    • Possible layers
      • Potential - Course Content?
        • Do we have a potential layer?
  • Certificates could be it's own application (maybe an IDA)
    • It owns everything related to certs
      • Authoring
      • Viewing
      • Analytics
      • etc
  • Where do courses fit?
    • Different apps would have their own views of the course
    • Just the parts they need
  • We would probably want some kind of API gateway - like api.github.com
    • We can pull data without caring where it's coming from
  • Having course be a overloaded concept probably works to our detriment
    • We may want to have content not in courses
    • Our problem banks are courses
  • How would we do it if we did it again?
    • Maybe leaf nodes should be different from other blocks in our course
    • Having everything be generic might be holding us back
    • Limits us to traditional, fixed, course structures
    • We probably have an implicit concept around "leaf blocks" vs. our "structural blocks"
    • XBlocks and XModules were designed to be very flexible becuase we didn't know what people were going to want to do
    • If we had "leaf blocks" we could sandbox them more easily
  • We should use these structures when we are missing the forest for the trees
  • The structures span bounded contexts and domains
  • Feels premature for us to be thinking about the large-scale structure
  • The layering could be useful for us?
    • Particularly when there are multiple ways of organizing things... it could provide guidelines
    • The layering should reveal itself to you after you rigorously practice the domain and the bounded contexts
  • Pluggable component framework
    • downsides exist as well
    • reminded of Jira - since trying to code the workflow
    • ultimately flexible, but completely static framework is hard to maintain
      • for example, xblocks
        • modelled the interface and let the plugin to implement it
          • invoked at query time
        • over-time we had a design where the data from the plugin gets pushed to other services
        • for example, score could be plugged in and could be implemented however it wanted
          • over-time, we pulled back from it
    • enables distributed devleopment, but makes it harder to change things afterwards
    • Performance versus flexibility is hard
  • At ADG, they found it difficult because of the tangling between the rules and the code.

Chapter 17. Bringing the Strategy Together

Quotes

  • ..evolution means that your final structure will not be available at the start, and that means that you will have to refactor to impose it as you go along. This can be expensive and difficult, but it is necessary.
  • A large- scale structure can exist within one BOUNDED CONTEXT, or it can cut across many of them and organize the CONTEXT MAP.
  • When you are tackling strategic design on a project, you need to start from a clear assessment of the current situation.

    1. Draw a CONTEXT MAP. Can you draw a consistent one, or are there ambiguous situations?

    2. Attend to the use of language on the project. Is there a UBIQUI- TOUS LANGUAGE? Is it rich enough to help development?

    3. Understand what is important. Is the CORE DOMAIN identified? Is there a DOMAIN VISION STATEMENT? Can you write one?

    4. Does the technology of the project work for or against a MODEL- DRIVEN DESIGN?

    5. Do the developers on the team have the necessary technical skills?

    6. Are the developers knowledgeable about the domain? Are they interested in the domain?

  • An architecture team can act as a peer with various application teams, helping to coordinate and harmonize their large-scale structures as well as BOUNDED CONTEXT boundaries and other cross-team technical issues. To be useful in this, they must have a mind set that emphasizes application development.
  • Six essential for strategic decision making
    • Decisions must reach the entire team.
    • The decision process must absorb feedback.
    • The plan must allow for evolution.
    • Architecture/infrastructure teams must not siphon off all the best and brightest.
    • Strategic design requires minimalism and humility.
    • Objects are specialists; developers are generalists.
  • Technical frameworks can greatly accelerate application development, including the domain layer, by providing an infrastructure layer that frees the application from implementing basic services, and by helping to isolate the domain from other concerns. But there is a risk that an architecture can interfere with expressive implementations of the domain model and easy change. Evolution, minimalism, and involvement with the application development team can lead to a continuously refined set of services and rules that genuinely help application development without getting in the way.
  • Don’t write frameworks for dummies.

  • Advocate a set of principles for all community members to apply to every act of piecemeal growth, so that “organic order” emerges, well adapted to circumstances.

Discussion

 Click here to expand...
  • This last diagram should have been presented in the beginning of the book.
    • from his perspective, he may have wanted the reader to first understand all the terminology beforehand.
  • The Assessment First section is good about how to get started
  • Architecture handed down doesn't usually work
  • Don't put all your senior engineers on the platform team
    • partly because of interests of people on the team
  • Don't have a top-down ivory-tower approach 
  • The books seems to taper off.
  • The epilogue has a bunch of war stories - with further perspective on the models he used from earlier in the book.
  • Would we ever get to the point of getting to the details (e.g., value objects versus entities, repositories, etc)?
  • We started with the django framework → overtime, we've migrated several things into our custom implementation (e.g., asset-handling, distributed config)
    • Vistaprint started in a framework, then replaced with a custom framework, then needed to replatform again to get back into the framework.
      • Probably need a hybrid - going either all framework or all custom is too extreme
  •  Don’t write frameworks for dummies.
    • Some DSLs do this.
    • Even xBlocks may be guilty of this.
  • Level of refactoring effort may depend on the maturity of the company.
    • Per the 20/80 rule, our focus on refactoring only needs to be within the segregated core part of the codebase
    • He also talks about how multiple refactorings lead to more suppleness in the code - so future refactorings would be easier.
    • But what about if the company pivots?  Then their core may change.