Why Software Can't Tell You It's Wrong

The Unfalsifiability Problem

Software architecture debates have a problem that most other engineering disciplines don't: the alternative was never built. When a bridge fails, the failure is physical, attributable, and measurable against every other bridge that didn't. The engineering decisions that caused it can be isolated, traced, and corrected - not just in theory, but in the next bridge, because the material itself produces feedback that no amount of professional opinion can override. Steel deflects. Concrete cracks. Physics doesn't care what the architect believed.

Software produces no equivalent feedback. A system built around the wrong abstractions compiles, runs, ships, and passes its tests just as readily as one built around the right ones. A bug introduced by a misaligned domain model looks identical, from the outside, to a bug introduced by a typo. A feature that took three times longer than it should have, because the structure made it harder than the business logic warranted, produces no artifact that distinguishes it from a feature that was simply difficult. The cost is real. The cause is invisible.

This is the unfalsifiability problem, and it runs deeper than "we can't measure everything." It means that when a system becomes expensive to change, the diagnosis almost always lands on the wrong variable. The domain is complex. The requirements changed. The previous team was careless. Almost never: the structure was wrong, and the structure was wrong because nobody ever built the other version of it to compare against. That version doesn't exist, it never will, and every architectural argument in the industry is conducted in its absence.

This would be a purely philosophical problem if there were nothing to do about it. There is something to do about it - but it requires accepting that the standard metric for software quality, whether it works, is measuring the wrong thing entirely.

The Metric That Hides the Problem

The natural substitute for "is this good engineering" is "does it work," because working is the only property the system demonstrates directly and continuously. A feature either satisfies the requirement or it doesn't. Tests are either green or they aren't. The sprint either shipped or it didn't. These are checkable facts, and in an environment with no physical feedback to measure against, checkable facts become the entire basis for evaluation.

The problem is that working is compatible with almost any underlying structure, good or bad, for a surprisingly long time. A system can satisfy every functional request in its backlog while its essential complexity - the actual rules and relationships of the business - is scattered across service methods, duplicated across layers, encoded differently in three places because three developers made the same inference independently without any structural pressure to put it in one place.

None of that appears as a failure. It appears as a working system, until the day a new requirement needs to reach across all three of those places simultaneously, and what should be a small change becomes an investigation into how many versions of the rule actually exist and which ones are authoritative. By then the cause is undiagnosable. The system is large. The rule has accreted across two years of sprint reviews. The developers who wrote the original three versions are gone. The structure that caused the problem will be attributed to enterprise complexity, changing requirements, accumulated technical debt - anything except the decision, made early and silently, to let working stand in for well-structured.

When behavior is the only metric, behavior is what gets optimized. Frameworks that produce correct behavior fast become the natural answer. Templates that generate working code without requiring structural reasoning become best practice. The question that would actually determine the ten-year cost - what are the mechanics that produce this behavior, and do those mechanics live in a place that makes them findable, correctable, and legible - never gets asked, because asking it requires a comparison that isn't available.

The Mechanics, Not the Behavior

The substitute for physical falsifiability isn't a better way to measure behavior. It's a deliberate shift in what gets examined - from the surface of the system, what it does, to the structure underneath it, what it is. Every piece of business behavior is produced by mechanics: relationships between concepts, rules that govern those relationships, responsibilities that belong to one concept and not another. A checkout process is behavior. The mechanics underneath it are what actually need to be represented correctly.

Consider a library lending system. The behavior is simple: a borrower takes an item, returns it later. The mechanic, examined carefully, is that a loan is a distinct event - something that happened between a person and an item, at a point in time, with its own start, end, and history. It is not a property of the item itself. A book does not become a different kind of thing when it is borrowed. The loan is a separate concept with a separate lifecycle.

That distinction is not an implementation preference. It is what is actually true about the business - and a system that puts loan dates on the Book entity has encoded a false mechanic, one that will prevent the system from answering basic questions about lending history without structural rework. The behavior can be implemented many ways. The mechanic is either represented correctly or it isn't, and unlike behavior, a misrepresented mechanic produces a specific, locatable kind of wrongness: the rule exists in the wrong place, the concept carries responsibility it was never supposed to own, the assumption is encoded somewhere it can't be found when it turns out to be wrong.

This is not a theoretical exercise. It is a structuring and learning activity. The goal is not to produce a perfect model before a line of code is written - it is to make the structure of the business explicit enough that building it becomes a matter of finding the right place for each piece, rather than making an independent decision every time about where things should go.

A user story is not a specification to implement. It is information about what the business is - a piece of evidence that either fits the current understanding of the mechanics or reveals that the understanding was incomplete. Both outcomes are useful. The first confirms the model. The second improves it.

Once the mechanics are explicit, simplicity becomes achievable in a way it never is when the focus stays on behavior alone. The simplest thing that works is not the simplest thing to type - it is the implementation that puts each responsibility exactly where it belongs, with no duplication, no misdirection, no logic living in a place that requires explanation rather than being found by intuition. That kind of simplicity is legible to a new developer on their first day. It is transferable to a team member who didn't write it. It is correctable when an assumption turns out to be wrong, because the assumption lives in one place that can be questioned directly rather than in a diffuse implication scattered across the system.

Essential and Accidental Complexity

Fred Brooks named this distinction decades ago, and it has lost none of its precision: essential complexity is the difficulty that comes from what the problem actually is; accidental complexity is the difficulty introduced by the tools and representations chosen to solve it.

Essential complexity cannot be removed without changing the problem. The fact that a loan involves a lender, an item, and a time period - and that these are three separate things with separate lifecycles - is not a decision anyone made about how to model a library system. It is what a loan is. Strip it out and the system no longer represents what the business does. Every piece of structure that reflects this accurately is essential complexity made explicit.

Accidental complexity is everything else: the framework, the deployment topology, the persistence layer, the test harness, the CI pipeline, the service boundaries drawn for organizational rather than domain reasons. None of this exists in the business. All of it exists to make the essential complexity operational. It is supposed to serve the essential complexity, not the other way around.

The moment a framework's conventions, a deployment constraint, or an organizational boundary starts dictating what the domain is allowed to look like, accidental complexity has taken the lead, and the essential complexity is being shaped to fit it rather than the reverse. This inversion is the likely trajectory of most software projects within a few years of starting - unfalsifiability means it can be neither proven nor disproven with certainty, but a perfect structural alignment between code and business mechanics from day one, sustained without a mechanism to detect and correct drift, requires a precision that makes winning the lottery look achievable.

It happens quietly because it produces no visible symptom. The system still works. The tests still pass. The sprints still ship. The essential complexity didn't disappear - it was dissolved into the accidental complexity surrounding it, encoded in service method names and transaction annotations, still doing its job but no longer findable as a thing you can look at, question, or change in one place.

The Cogwheels of the Business

A mechanical watch is a useful image here. The cogwheels of a watch are its essential complexity made physical - the gear ratios, the relationships between wheels, the escapement that governs the release of energy. These cannot be simplified away without the watch losing the ability to tell time. They are what the watch fundamentally is. The casing, the hands, the crystal, the materials - these are accidental complexity: real, necessary, but entirely in service of the mechanism underneath.

A watchmaker looking at a malfunctioning watch does not need to guess where the problem is. The mechanism is visible. Each part has a known responsibility, and a part that has the wrong shape for its position will produce a specific, locatable failure rather than a general degradation of the whole. The casing does not malfunction. The mechanism does - and it can be found.

A rich domain model is the cogwheels of the business - and the automation of it. Not data with operations attached, which is what most systems produce, but responsibilities that carry their own data and enforce their own rules. An account is not a row in a table with balance and status fields. An account is something a customer has, that carries a balance, that knows what it means to deposit and withdraw, that enforces the rules governing when a withdrawal is and isn't permitted.

Those rules belong on the account - not because of aesthetic preference, but because that is where they would be found by anyone who thought to look for them by asking what an account is. When those rules live on the account, a wrong rule is a local problem, visible and correctable in one place. When those rules live in a WithdrawalService and a BalanceCheckService and a TransactionValidationService distributed across a microservice boundary, a wrong rule is a distributed problem, findable only by someone who already knows which services encode the assumption and can read them all simultaneously.

The practical difference is not academic. On a system where the essential complexity is explicit, a new developer can read the model, understand what the business is, and contribute meaningfully in days - because the structure of the code reflects the structure of the business, and understanding one is understanding the other. On a system where the essential complexity has been dissolved into accidental layers, a new developer learns how to navigate the framework, traces several services to reconstruct a rule nobody wrote down, makes a change, breaks something else that encoded the same rule differently, and starts the cycle again. The business is still there - it is just no longer visible as the thing organizing the code.

Correctability as the Partial Answer

This is where the unfalsifiability problem becomes something other than a purely philosophical concern. Unfalsifiability means the alternative was never built, so no direct comparison is available. It does not mean nothing can be done. The absence of a physical feedback mechanism can be partially compensated for by a structural property: if wrong assumptions live in one legible place, they can be corrected when they turn out to be wrong.

Correctability does not prove the original model was wrong - it cannot, because unfalsifiability is still in effect. But it prevents the cost of being wrong from compounding indefinitely, which is the actual damage the problem causes. A wrong assumption that gets corrected in year two costs a week. The same wrong assumption that is only discoverable by reading fourteen service methods, in year five, after three team changes, costs months - and the correction itself has to find and fix every place the assumption was encoded, which are no longer guaranteed to be consistent with each other.

When essential complexity is kept explicit, a misaligned model is a correctable one. When it has been dissolved into accidental layers, a misaligned model is a permanent one - not because it cannot technically be corrected, but because the correction cost grows faster than anyone is willing to pay it, and the system calcifies around the wrong assumptions instead. Every new feature gets built on top of them. Every new developer inherits them. The system is not failing. It is just becoming progressively more expensive to change, in a way nobody can trace back to a cause because the cause is structural and the structure is no longer visible.

Focusing only on whether the system works skips this entirely. Frameworks that deliver working behavior fast are not the problem. The problem is that working is the only thing being measured, and the structure that determines the ten-year cost is invisible to the measurement.