12. Can Humanity Be Automated?
The Problem: The Office Where the Empathy Button Rings
A new feature has been added to a customer-support screen. When a customer's message is entered, the system analyzes their emotional state and recommends the most appropriate reply. The sentence “We're sorry you've had to go through this inconvenience” appears at the top of the screen, ready for an agent to send with a single click. One day, a customer who is about to attend a funeral requests a refund. The system detects sadness in the message and recommends an empathetic phrase. Amid a busy queue, the agent sends it. The customer replies immediately: “It feels like no one even read that before sending it.”
What failed here was not the grammar of the sentence. The sentence was polite, and the emotional classification may not even have been wrong. The problem was that humanity had been reduced to the ability to produce an appropriate sentence. Automatically adding the word “empathy” does not mean that anyone has taken in another person's circumstances. Humanity is less a decorative tone of voice than the ability to adjust one's actions in the presence of another person's specific situation.
As AI and automation expand, we are tempted to turn humanity into a feature list: empathetic chatbots, friendly voices, personalized recommendations, notifications tuned to emotion. But what does it mean to “respond in a human way”? Is humanity an essence that cannot be automated, or is it a pattern of behavior that can be implemented with sufficient sophistication?
The Philosophical Question: Humanity Lies in the Way We Relate, Not Only in the Result
People have long taught humane behavior as a set of rules: ask how someone who is ill is doing, give someone who made a mistake another chance, and admit when you do not know something. Some of these behaviors can be automated. We can build systems that express consideration under certain conditions and add filters that reduce discriminatory language. Automation does not only undermine humane values. It can also support forms of care that would otherwise be missed because of the limits of memory and attention.
But humanity cannot be exhausted by executing rules. It also includes the ability to pause or reinterpret a rule according to the situation. The same sentence—“A refund is difficult under our policy”—cannot be applied identically to someone asking for a refund because they changed their mind and someone who lost their store in a disaster. Allowing exceptions can unsettle a standard, but allowing no exceptions at all turns the standard into a wall that can no longer see people.
Philosophy stops us at the assumption that “being efficiently kind is enough.” Engineering translates that question into design. Is the system merely imitating emotion, or is it helping people exercise better judgment? Does an automated sentence replace review by the person actually responsible, or is it only a draft for that review? Can users tell whether they are speaking with a person or a machine? And when they encounter a machine's error, can they be connected to a responsible human being?
An Engineering Case: What Disappears When Work Is Broken Apart
Suppose a company wants to automate employee interviews for its human-resources team. The system analyzes survey responses, attaches labels such as “high burnout risk” and “likely to leave,” and even generates interview questions for managers. On the surface, it is an efficient system. A manager can quickly scan the state of hundreds of employees and identify people who might otherwise be overlooked.
Yet the meaning of an interview can change the moment the work is divided into “calculate risk score,” “generate questions,” “send notification,” and “summarize interview.” An employee discovers that their distress was read first as a score and a set of tags, rather than through a conversation. Before actually listening, the manager has already received a conclusion. A system-generated summary is convenient, but it can erase hesitation, silence, and confessions that do not quite fit together.
A better design limits the score to a signal for asking questions, not a verdict. A high risk score should not automatically lead to an evaluation or personnel action, and the person concerned should be told what information is collected and how it is used. Interview records should include space for the employee to request corrections or deletions, not only the manager's summary. Above all, the system should say not “This person is burned out,” but “Recent responses mention workload and sleep problems. Would you like to check this directly?” The goal of automation shifts from judging a person on someone's behalf to helping a person avoid judging too quickly.
Humanity in recommendation systems is not only a matter of adding more choices. We need mechanisms that pause the stream of provocative content when someone may need care, and allow users to say, “I don't want these kinds of recommendations right now.” Predicting a user's past behavior as accurately as possible is not always the same as respecting them. Sometimes a more humane design gives people the right to decline prediction, the right not to be remembered, and the opportunity to encounter something unfamiliar.
Objection and Tension: Is Human Judgment Always Warm?
We should not romanticize human judgment while criticizing automation. People get tired, carry biases, and grow indifferent after hearing the same question again and again. One rude remark from a single support agent can leave a deeper wound than hundreds of automated replies. Emotion analysis and notification systems can help detect signals humans miss and guarantee a consistent minimum level of respect.
So it is not quite accurate to frame humanity and automation as opposites. The important question is whether automation conceals human vulnerability or supplements it while acknowledging that vulnerability. The mere fact that a human agent remains in the loop is not enough. What matters is whether the agent can reject a suggested phrase, take time to handle an exception, and apologize and repair the situation after making a mistake. Leaving a human in place as the system's final button and handing that person every judgment is not the preservation of humanity either.
Humanity can be automated. But that is different from automating output that merely looks human. What can be automated is remembering the conditions for care, checking for dangerous bias, and calling back people who have been left disconnected. Lowering one's certainty in the face of another person's words, bearing the weight of an exception, and taking responsibility and repairing a relationship after doing harm still require a human decision beyond the system.
Questions to Leave With
Does the system you are building imitate humanity through its tone, or does it help people listen longer and judge more carefully? Is there a real person behind the automated sentence of empathy who will take responsibility? And even when a machine eventually produces an answer that is sufficiently human, will we be ready to ask who sent it—and whether that relationship should remain in the machine's care?