Lean Architecture: for Agile Software Development by James O. Coplien and Gertrud Bjornvig
A well-considered system keeps evolving so everybody wins.
It is imperative because, on a human scale, it is a scandal to sacrifice development staff comfort to end user comfort.
Just because things are simple doesn’t mean they are simplistic.
Lean’s primary focus is the enterprise value stream. Lean grabs the consumer world and pulls it through the value stream to the beginnings of development, so that every subsequent activity adds value. Waste in production reduces value; constant improvement increases value.
Architecture represents the significant design decisions that shape a system, where significant is measured by cost of change.
However, one of the great paradoxes of Lean is that such intensity at the beginning of a project, with heavy iteration and rework in design, actually reduces overall life cycle cost and improves product quality. Apply those principles to software, and you have a lightweight up-front architecture.
Lean architecture is about doing what’s important now that will keep you in the game for the long term.
Lean architecture also adopts the perspective that standards are valuable, but again: at the level of form, protocols, and APIs, rather than their implementation.
The code is the design.
Agile understands that nothing lasts forever, and it instead focuses explicitly on what is likely to change.
System architecture should reflect the end users’ mental model of their world.
Deferring interactions with stakeholders, or deferring decisions beyond the responsible moment slows progress, raises cost, and increases frustration. A team acts like a team from the start.
Going beyond this expression of form with too much structure (such as class implementation) is not Lean, slows things down, and leads to rework.
If we were to look at Lean and Agile through a coarse lens, we’d discover that Agile is about doing and that Lean is about thinking (about continuous process improvement) and doing.
Everybody, All together, Early On
We go to the code for what and how, but only the authors or their documentation tell us why.
People appreciate being invited to the party early. And remember the Lean Secret: everybody, all together, from early on.
The what-the-system-is part relates to what is commonly called the architecture or platform: the part that reflects the stable structures of the business over time.
We like Jerry Weinberg’s definition of problem: the difference between the current state and a desired state.
great projects start with a visionary and vision rather than a market analysis.
It’s not that form follows function, but function and form weave together into a real structure that we call member data and methods.3
Think of domain expertise as a broadening of user expertise into the realms of all stakeholders taken together. Expert developers learn over time what the fundamental building blocks of a given system should be.
Maybe half of software development is about nerd stuff happening at the whiteboard and about typing at the keyboard. But the other half is about people and relationships.
Architecture is an ongoing discipline with a substantial up-front planning component.
The key to success in both Lean and Agile is the Lean Secret: Everybody, all together, from early on.
Much of the challenging part of design lies in partitioning a system into modules and collections of related APIs. That is exactly the problem that up-front architecture addresses.
A true software architect is one who is a domain expert, who knows how to apply the domain expertise to the design of a particular system, and who materially participates in implementation
User stories help end users think in a concrete way, and if we feed back our understanding of their expectations, we can gain foresight into how they will react if the system were deployed according to our understanding. If we do that job well we can build a system that technically meets end user needs.
One problem with requirements is that they never anticipate all the scenarios that the user will conceive.
Another way of talking about user expectations is to note that, unless we are designing video games, the end user rarely finds surprises to be pleasant. Therefore, it is crucial that the system architecture reflect the end user cognitive model.
But when we are laying out the product architecture we want the end user’s cognitive model of the system.
Software development is rarely a matter of having enough muscle to get the job done, but rather of having the right skill sets present.
Developers are the prime oracles of technical feasibility. They are the primary solution domain experts. They should be active experts.
Developers and testers should be friends.
Working independently avoids groupthink, and avoids one personality overpowering the other with arguments that things must be thus-and-so
Martin Fowler writes (Fowler 2006) that multi-site development requires more written documentation and, in general, more formal communication styles, than co-located teams do.
A problem definition is an explicit, written statement of a problem: the gap between the current state and the desired state.
Software systems are not only complicated: they are complex. Though you can track a system effect back to set of causes you can’t always chart a path from cause to effect.
As a stopgap measure, anyone can receive a “request to fix something” that originated somewhere along the lines of power in the organization, re-write it as a good problem definition, and feed it back to the requestor. Such feedback ensures that you are solving the right problem, that you together understand what the solution criteria are (they are measurable), and that the problem doesn’t have a built-in trap that will lead to failure.
But we also face the stark reality that it’s becoming increasingly difficult to find the ideal single person to master plan the system. Instead, we recognize that together we know more than any one of us.
Time hardens the interfaces and the entire system slowly hardens.
However, re-factoring is almost always a local activity, whereas architecture is a global concern. There is a serious mismatch between the two.
A second common belief of Agile is that we should do things at the last responsible moment (Cohn 2004, p. xv). It’s a bit of a problematic formulation because one never knows exactly when the magic moment has slipped past. It might be better said that we shouldn’t make decisions irresponsibly early - or late. If you have enough information to get started, then get started.
The fundamental form of a business often repeats itself in system after system. This constancy means that we know much of it at the beginning of every project. To defer the decision of what essential domain form to lay as the system’s foundation is irresponsible because it has a high chance of creating waste. Therefore, we embrace domain knowledge - stuff that we truly know about the business - at the beginning of product construction.
The compensation must come from a historical perspective. Domain knowledge integrates many past nows into a “long now” (Brand 1999). That’s why it’s better to form modules around domain knowledge than around the current end-user view. However, sometimes the end-user view is all you have. This is particularly common for new features. Use domain knowledge when you can get it, and the current end-user mental model when you can’t.
To embrace change requires the discipline of understanding what is likely to change, and what will likely remain stable.
So the first rule of thumb for architecture documentation is: Don’t sweat the small stuff. Focus on what matters.
Lean architecture avoids producing wasteful artifacts simply by refusing to produce many historic artifacts of software architecture.
If the domain analysis finds a recurring, common, stable form, capture it somewhere, even if it supports only a single stakeholder.