All articles

The Empathy Gap in AI

Why empathy isn't something you can automate away

Your AI will never be able to empathize.

It's not built for empathy. It can approximate, and imitate, but it is fundamentally incapable of caring.

That absence of empathy becomes a product problem the moment you stop accounting for it.

A Growing Dependency

As product builders, be it designers, engineers, product managers, or an amalgamations of all three of those roles that seem ever more pervasive these days, we've come to rely on AI more and more over the past few years. It's inevitable, and inescapable. AI is genuinely better than we are at some things. It can analyze data at scale, digest enormous volumes of text, and synthesize information in seconds.

Be it Claude, Perplexity, Cursor, or whichever form your daily artificial assistant takes, it's undeniable that we've all grown accustomed to leaning on it. It's easier to let it stew on a question while you work on a different task, let it compile a slide deck for a presentation, or have five different instances work through your task list in parallel.

The danger isn't using AI. It's forgetting which problems were never really about information in the first place.

And that danger arises when we start handing over tasks that are unquestionably human in origin. When we forsake things that have traditionally landed in the domain of conversation. Of empathy.

Think of user interviews. Of customer support calls. Of answering your in-product chat inbox. Or of writing help center articles about commonly (or uncommonly) occurring issues.

Those are by their very nature inextricably human endeavours. And yet we're increasingly finding opportunities to outsource these to our artificial companions.

Stuck

A while back, I gave a talk at a conference, titled "Where Are The Humans?" And the fundamental issue at hand here was similar, yet different, than the one outlined above.

Humanity is quickly disappearing from our products, and is being replaced efficiency.

Or so we like to tell ourselves. Instead, in my experience over the last two to three years, we've shifted the onus of finding a path forwards in unexpected situations from our knowledge bases, Support staff, and Customer Success teams, to the user themselves. The difficulties that one encounters using our products hasn't gone anywhere. It's just been quietly re-routed and diluted to a "them" problem, instead of an "us" problem.

And the result of this, is that our users feel more stuck than ever. Isolated even.

In that talk, I mentioned several instances where I, as the end-user of a product, had gotten stuck. And the only way forward was through a series of increasingly frustrated interactions with artificial support systems. Electronically voiced agents over the phone repeatedly echoing "Please tell us how we can help", with no actual capability of helping.

Since then, it's gotten worse.

These instances of being left at the whims of an AI that is incapable of understanding the intricacies of human emotion, and was never intended to do so, have grown exponentially. From being stuck at an airport at 5:17 in the morning because an "AI-first fraud detection algorithm" suddenly decided an Uber ride was an abnormal spending pattern amongst a history of hundreds of similar rides, to having a company-issued laptop shipped out to my home and back to the supplier because the automated agent on the other end of the conversation misconstrued my support request, I've felt more lost than ever.

And that's the thing that strikes me every time.

None of those situations required a more capable model. They required someone to recognize that I wasn't asking the right question. That I was frustrated. That I was tired. That I wasn't looking for another explanation of policy, but for a way forward.

The failure wasn't one of intelligence.

It was one of empathy.

Staying Close

So where does that leave us?

Do we reject AI and return to a world before the advent of LLMs? Probably not. A blanket rejection of all things AI likely results in our roles (and products) getting cannibalized by someone who does embrace the things that it is specifically good at. Assuming we can travel back to a time before conversational UI became mainstream would be just as naive as believing AI will eventually replace every human task imaginable.

Instead, I think there's something to be said for being deliberate. For consciously being slower. For being deliberate about what we choose to delegate.

Not everywhere.

Just in the places where speed costs us understanding.

There's a time for speed. For efficiency. When you're summarizing an interview, sure. Or writing the first draft of a help centre article. Or for identifying patterns across five hundreds support conversations.

But sitting in the interview itself? Writing the support ticket response? Watching someone struggle the workflows that you and your team have just spent 6 months refining?

No.

Those aren't outsourceable tasks. Those are where empathy originates. Where it is cultivated, understood, and embeds itself in the way you build experiences.

Those are the moments that remind us our users aren't personas, metrics, or dashboards. They're people. Standing on a train platform on their way to work. Half-listening in a Zoom meeting while trying to fix an issue before the next agenda item. Trying to get through their day. Those moments are uncomfortable. But they're also irreplaceable.

Because they teach us what it actually feels like to use the thing we've built. That isn't something you can generate after the fact.

The interview summary tells you what was said. It doesn't tell you why someone hesitated before answering. It doesn't tell you that they apologized before asking a question. It doesn't tell you that they were clearly frustrated, but trying not to sound rude.

Those details are easy to omit because they don't look like data.

They're also where empathy comes from.

Beyond Dogfooding

For years we've been told to dogfood our own products.

“Dogfooding” refers to the practice of using your own company’s product internally. Companies dogfood to catch bugs before customers do, build genuine product empathy across teams, and test whether the product holds up under real daily use.

Dogfooding has been one of those ideas in product development that has become almost universally accepted. And for good reason. If we aren't willing to use the things we build, why should anyone else?

But I think we've slowly started asking dogfooding to answer a question it never really could.

The goal was never to prove that we could use our product. It was to understand whether the people we built it for could.

Those are fundamentally different things.

When I use my own product, I bring months, sometimes years, of context with me. I know why a feature exists. I know what compromises were made to get it shipped. I know which workflows are still rough around the edges, and which ones we simply haven't gotten around to improving yet. I know what an error message is trying to tell me, even when it does a poor job of saying it.

My users know none of that.

Instead, they bring an entirely different kind of context. The meeting they're already late for. The train they're trying not to miss. The frustration of having already spent twenty minutes trying to solve a problem before finally reaching out for help. The expectation that this should have been straightforward, and the creeping suspicion that perhaps they're the one doing something wrong.

Dogfooding will never show me that.

The closest I'll ever get is by spending time with the people using my product. By watching them hesitate before clicking a button. By seeing the workaround they invent because the intended workflow wasn't obvious. By noticing the questions they apologize for asking because they assume they should already know the answer.

The Long Way Around

When I first started putting these thoughts to paper, I thought it was going to be about AI.

It isn't. And it is.

But more fundamentally it's about distance.

Over the years we've become remarkably good at insulating ourselves from the uncomfortable parts of building products. We have dashboards instead of conversations. Personas instead of people. AI-generated summaries instead of sitting through the interview where someone spent the first ten minutes convinced the problem was their fault, before quietly admitting they'd nearly given up altogether.

None of those things are inherently bad. In fact, they're often incredibly useful.

I've lost count of the number of times I've asked Claude to summarize an interview transcript before going back through it myself. I'd be lying if I said I was about to stop doing that. But there's a difference between making sense of an experience, and replacing it.

One saves time, the other costs perspective.

I worry that we're becoming so good at removing ourselves from the messy parts of product development that we're also removing ourselves from the very things that taught us empathy in the first place. The awkward interview where nothing quite goes to plan. The support conversation that turns out not to be about the bug you thought it was. The customer who spends fifteen minutes describing the wrong problem because they don't yet have the vocabulary to describe the real one.

Those moments are inefficient. They're also where the work happens. Not the work of shipping features. The work of understanding the people you're shipping them for.

Somewhere tomorrow morning, someone will open your product between two meetings. They'll have a coffee in one hand and a Slack notification demanding their attention in the other. They'll click the wrong thing. They'll misunderstand what you meant. They'll blame themselves before they blame you.

No dashboard will ever tell you what that felt like.

You have to go and see it for yourself.