When looking deeper is the mistake
The hard part of user research is knowing how to interpret what you hear
“You know what they’ll tell you: they’ll ask for a faster horse.” - the fallacy is about literal interpretation: users describe solutions, not needs.
Plenty has been written about that already.
What gets less attention is the opposite failure: going so deep into interpretation that you start finding things that were never there.
Occasionally what users say they want is exactly what they want, and the most useful thing we can do is believe them.
What comes to mind when we hear the word ‘pain’
The word pain means something different to everyone.
We don’t know how others feel pain, and we may default to our past experiences: a broken knee, pain of missing someone, mental issues etc.
There is an artist that dedicated part of his artistic career to the interpretation of language.
Mladen Stilinović.
His retrospective “Sing!” was hosted in The Ludwig Museum in Budapest in 2011.
If we entered the museum then we would see his work Dictionary – Pain hanging on the walls: framed pages, row after row, running the full length of the gallery.
Only when we moved closer would we see the content: each frame holding a single page from an English dictionary, 523 pages in total, every definition covered in white and replaced with a single word, so that ability became pain, abolish became pain, abdomen became pain, and every entry through to the end of the alphabet became pain too.
Stilinović had spent decades examining what language does to people, he took a dictionary - a book whose entire purpose is to stabilize meaning, and reduced it to only one definition.
His work operates on several levels at once:
it collapses the dictionary’s core promise by making every entry return to the same definition
it names what political language under Yugoslav ideology actually felt like from the inside, where official speech crowds everything else out until only the sensation remains
it uses English, pointing at the language that defines access to art, legitimacy, and the international stage
and it holds a mirror to us - standing in the room, because the word pain arrives loaded with our own experience.
Stilinović said that people would not stop asking him:
What pain?
Who is in pain?
- always needing answers and interpretations.
But there was nothing to be explained or analysed. The pain was just there - he responded.
What does it mean for us?
When we run conversations, we are trained to look underneath what people say. We dig for motivations, hidden needs, systems, tensions, the story behind the story. Often, that makes sense. Users describe solutions, and we need to understand the need underneath.
But sometimes the opposite happens: we keep digging after the useful meaning has already arrived.
We turn a clear sentence into a theory.
We treat someone’s words as raw material for our interpretation, instead of taking their words literally.
When we hope to see what’s not there
I was organizing a stakeholder immersion in Delhi, small part of which was a voluntary project exploring appetite for a sustainability shopping platform in India. We invited potential customers to a space we rented for the study - a narrow room that nobody had touched in months, dusty, no windows, slightly airless. Participants often came from far away, visibly uncertain about the whole study.
In a couple of those sessions, some of them mentioned unprompted reducing plastic usage. I was invested in the project and I built a thread around it. I connected the plastic comments to a broader willingness to embrace sustainability, to buy secondhand, to shift consumption habits in ways that would make the platform viable.
I missed the fact that the Indian government had recently introduced pressure on reducing plastic use. The participants were describing a regulation they were adjusting to, they haven’t chosen that value themselves.
Eventually the existing external data confirmed what I hadn’t wanted to see.
It offered a more sober picture: although some Indian second-hand clothing buyers were motivated by environmental impact, the formal market was still at an early stage of adoption.
The appetite I had identified wasn’t yet there: what I had found was my own interpretation, dressed up in their words.
Over-interpretation is hard to catch from the inside because it feels like due diligence. We are going deeper, making connections, seeing the pattern underneath. The participants gave us one data point and we built a narrative around it.
Checking whether the pattern is real
It’s worth to check with ourselves after every session what the signal is telling us, and what would prove our interpretation wrong.
The problem with over-interpretation is that it feels indistinguishable from good research. We are listening for the need underneath the solution, looking for the thread that connects individual moments into something actionable.
Before we do anything else, we should ask ourselves whether we want this pattern to be true.
I skipped it in Delhi because the answer was uncomfortable: I was personally drawn to the sustainability angle, I believed in the project, and I wanted the research to bring positive signal to something I thought was worth building.
Maybe there is a stakeholder we respect and don’t want to disappoint, a feature that was our idea in the first place, that we championed in a couple of meetings before we got approval to test it, a topic that connects to something we care about personally, sustainability, accessibility, related to our own values that we want the evidence to go a certain way.
In all of these cases the bias is not cynical, it is investment that makes we good at our job but also may make us see things that are not there.
It’s good to be self aware of it before we start writing up our findings, and run our interpretation through that awareness before we present it as neutral.
I usually run two practical checks:
The first is scale: qualitative research surfaces patterns but cannot confirm them. If we are hearing something in sessions that feels significant, find out whether it shows up anywhere else, in behavioral data, in usage metrics, in industry reports, in consumer trend research. If we cannot find any corroborating signal at scale, it’s an important information. It does not invalidate what we heard - it may mean it’s a novel topic that hasn’t been studied yet but it should make us hold our interpretation more lightly until we can explain the gap.
The second check is past behavior. Stated preferences in a research session are the most optimistic version of what someone believes about themselves. They are not lies, they are aspirations, shaped by the context, the presence of a person listening, the implicit social contract of wanting to give useful answers. What cuts through that is evidence of what people have actually done: meaning whether they have changed a purchasing habit because of it, sought out an alternative when the convenient option was not available, or absorbed a cost they did not have to absorb.
Motivation that has already produced behavior is a different category of signal from motivation that exists only as a preference. Participants in Delhi were telling me how they reduce plastic usage, but they did not tell me what motivated them to do so.
Neither of these checks eliminates interpretation. Qualitative research is inherently interpretive and that is its value, surfacing things that no metric can show you. But these checks can force our interpretation to survive contact with evidence that exists independently of our reading of the room.
If it holds up, you have something worth building on. If it doesn’t, we have a hypothesis worth testing further.
I did not run any of these checks in Delhi. The research confirmed what I wanted it to confirm. Running them earlier would not have saved the project, but it would have saved me several months of wasting time promoting something that was not there.
Tip: If other people watched the sessions, ask them what they noticed before sharing your own reading, and approach their answers with curiosity rather than looking for the version that confirms what you already think.
Then check whether the pattern holds outside the room: whether participants have actually behaved in line with what they said, and whether you can find the corroborating signal somewhere in the data, or whether its absence is itself worth noting.
So my colleague was probably wrong about wanting to skip the conversations. He was not wrong that sometimes we come back from it with a horse.
It comes with practise which one is which.
Disclaimer: The views and opinions expressed on this Substack and in its related posts and articles are my own and do not necessarily represent those of any current or former employer.