09. There Is No Perfect System
The Problem: The Illusion of a Day Without Failure
A service has been running reliably for several months. The monitoring screen is green, and failure alerts are silent. During the retrospective, the team says, “It feels almost perfect now.” A week after those words are spoken, a telecommunications outage in one region brings the payment service down. A disaster-recovery environment exists, but it has not been checked in a long time. Automatic failover works, but an old certificate rejects the connection. During recovery, some customers are charged twice, and the support center is flooded with the same question.
A failure may appear suddenly, but most failures are not created suddenly. Small exceptions accumulate. Manual procedures with no clear owner remain in place. The statistics of successful days conceal the conditions under which the system can fail. Believing that a system has become perfect is more dangerous than having a defect. It tells us that there is no longer any reason to doubt or test it.
The Philosophical Question: Recognizing Vulnerability Instead of Pursuing Perfection
We imagine systems as closed machines. Input goes in, output comes out according to fixed rules, and if the design is good enough, the system will remain in a predictable state. Actual systems, however, are open structures entangled with people and organizations, external services, laws and markets, weather and chance. Events that occur beyond the boundary can enter from outside at any time.
Philosophy interrupts the illusion of perfect control. The important thing is not to eliminate failure, but to imagine the forms failure might take and make it possible for people to recover afterward. This involves more than resilience in the technical sense. It also raises questions about who can reverse a decision, who is told what went wrong, who receives compensation, and who gets another chance to try.
In organizations that pursue perfection, bad news arrives late. Metrics must remain clean, plans must proceed as scheduled, and mistakes are reduced to individual carelessness. In organizations that acknowledge vulnerability, strange logs and uncomfortable reports from users are not annoying interruptions but inputs through which the system learns. Safety is less like a state without defects than a state in which defects do not have to be hidden.
An Engineering Scene: Quiet Bias in a Recommendation System
A video platform’s recommendation team is running a model designed to increase the time users spend on the service. The model shows videos that hold a user’s attention for longer, and the overall metrics continue to rise. One day, an operator receives reports that extreme videos on a particular subject are being repeatedly shown to some users. The model has merely linked videos that do not individually violate the rules. Each recommendation is a small continuation of an existing interest, but by the end of the chain, the user is trapped in an information environment they never anticipated.
The team does not discard the model immediately. Doing so could also remove useful recommendations and increase the effort users need to find content they actually want. Instead, it exposes some of the reasons behind recommendations, places interest-reset and feed-switching features where they are easy to find, and adds metrics that monitor repeated exposure and topic concentration in addition to watch time. When a particular type of content begins to trigger a rapid chain of recommendations, the system automatically slows the process and routes it for human review.
These measures are not perfect solutions. They can be exploited, create new biases, and fail to explain every user’s experience. But they differ from what came before in one important respect: the system reveals its limitations and gives users a way to intervene. Rather than promising perfect recommendations, the product makes the possibility of being wrong part of the recommendation system itself.
Counterargument and Tension: Do Not Use Uncertainty as an Excuse
The statement “nothing is perfect” can be used in two ways. One is the starting point for humble design. The other is a shield for avoiding responsibility. “It can never be perfect, so let’s launch it anyway,” or “Every system has errors, so there is nothing we can do,” often appears when the cost of failure is being passed on to users.
Acknowledging imperfection does not mean lowering standards. It means deciding more clearly which failures will not be tolerated. Payment errors should trigger automatic refunds; medical-advice systems should hand a case over to a person when confidence is low; and an account suspension should begin as a temporary state open to review, not as an immediate permanent punishment. If we cannot prevent everything, we should at least reduce irreversible harm and secure time to correct a mistaken judgment.
This requires tools such as incident drills, chaos testing, audit logs, and rollback buttons. But tools alone are not enough. We must also treat as part of the system whether someone who sees an alert actually has the authority to stop operations, whether an operator has the documentation and contact list needed to make a decision at two in the morning, and whether the person who reported a failure will be punished. A robust structure is not one that is hard as steel. It is one that knows where it will bend under pressure.
A Question to End With
If your system fails, what will be the first thing to shake: the data, trust, or one particular user’s life? And where is the mechanism that will expose the failure rather than conceal it, allowing people to choose again? Instead of building a system that merely looks perfect, we should ask whether we are building one that can recover in an imperfect world.