04. Do models describe reality?

The question: The graph rose, so why did people leave?

The line on the dashboard climbs smoothly. After the new recommendation model is deployed, click-through rate rises by 12 percent, and prediction accuracy is higher than that of the old system. Relief spreads through the meeting room. It seems natural to conclude that the model has become better at understanding users' preferences.

A few weeks later, customer support fills with a different kind of message. "It keeps showing me only what I already watch." "I don't know why this was recommended." "It was convenient at first, but now I'm tired of it." The numbers have improved, but users are leaving. Did the graph lie? Or did we simply believe too quickly that the graph was a miniature version of reality?

Models are powerful engineering tools. They compress complex phenomena into variables and relationships, turn them into forms that can be computed, and help guide future decisions. But to compress something is also to leave things out. A model does not show reality itself. It shows one part of reality, shaped by a particular purpose, dataset, and method of measurement.

The philosophical point: Are explanation and prediction the same?

A model that predicts well does not necessarily explain reality well. A model that accurately predicts tomorrow's traffic does not thereby understand the cause of congestion. A model that captures many strong correlations does not thereby know the causal relationships. The fact that two variables move together is a different kind of knowledge from the claim that changing one will produce a different result in the other.

As debates about causation since Hume have repeatedly shown, we cannot establish a cause from records of events occurring together alone. Contemporary machine-learning systems add a problem of a new scale. Thousands of features and weights may combine to produce high performance, but complexity does not guarantee understanding. Explainability is not a magic trick that extracts the model's inner truth intact. Deciding what explanation to give, to whom, and for what purpose is another design problem.

A model does not copy reality. It cuts reality into the shape of a question. A model asking "Who is likely to churn?" creates a very different system from one asking "Who is bearing the risk of churning?" Both may start with the same data. The first can lead to campaigns designed to retain users; the second can lead us to examine structural causes such as pricing, accessibility, and service quality. Before examining a model's equations, we should ask what sentence the model is putting to reality.

An engineering case: When one prediction score changes a person's path

Imagine a loan-review system that calculates an applicant's likelihood of repayment as a score. The model learns rules from variables such as past repayment records, income, transaction patterns, and address. High accuracy is attractive to a financial institution. But when an applicant receives a low score, what does that score mean? Does it really mean that the applicant has a low ability to repay, or does it mean that people with similar circumstances did not receive enough access to credit in the past?

A model can learn past imbalances along with past decisions. If a particular area has had few financial services, its transaction data may be sparse, and that sparsity may then be translated into low trustworthiness. The absence of data is not a neutral gap. It is the record of a history in which someone was not included in the system. Once the model reads that gap as risk, past exclusion becomes evidence for the future.

Recommendation systems create similar problems. The fact that a user clicked on a piece of content may mean they liked it, but it may also mean that the title was provocative or that no other option was easy to see. A click is evidence of interest, but it is also a trace of behavior shaped by the interface. If the system optimizes only for click-through rate, it may increase immediate reactions rather than user satisfaction. Easy-to-measure behavior pushes harder-to-measure experience aside.

That is why models need several layers of validation. We need to examine not only performance metrics, but also how the data was produced, how predictions are used in actual decisions, and who bears the loss when the model is wrong. A model card or evaluation report can be more than decorative documentation; it can constrain where a model is used. Marking uncertainty, giving people a way to challenge a result, and distinguishing automated decisions from human decision support are also parts of performance.

Counterpoint and tension: Can we make decisions without models?

Criticism of models can easily slide toward the opposite extreme: the belief that human intuition is more fair and more humane. But people have imperfect memories, repeat the biases of their groups, and often cannot properly explain why they made a judgment. Not using a model does not automatically produce a better understanding of reality. Unexamined human judgment may simply operate more quietly.

The issue, then, is not whether to use a model. It is where to place the model's judgment and with what authority; whether there is a process for correcting or rejecting it; and whether we can question the model again when its data or goals change. Numbers do not remove human judgment. They move it somewhere else. Humans still decide when they set the objective function, create labels, choose thresholds, and approve exceptions.

The risk is especially high with generative AI, where plausibility can look like explanation. A model can offer reasons in smooth, fluent sentences, but fluency does not prove that evidence exists. Systems need procedures for checking real sources when citations are requested, limits that allow them to say when they do not know, and human review for consequential results. Explanation should be closer to traceability than to tone.

A question to leave with

What does the model you built show clearly about reality, and what does it make harder to see? As the accuracy score rises, are there blind spots growing alongside it? Engineering is responsible not only for giving people more reasons to trust a model, but also for building structures that show them when not to trust it.