Part 5 of the “UX × AI” series.

In Part 1, we reframed the fear. In Part 2, we recognized the skill. In Part 3, we protected the thing that cannot be delegated. In Part 4, we got practical—a week-by-week guide to building genuine AI fluency in your real workflow.

This article is a myth-buster. And the myth it addresses is one of the most quietly dangerous ideas circulating in product and design organizations right now.

The idea is that more data produces a better understanding of users.

It does not. And the belief that it does—amplified enormously by AI's ability to process data at a previously impossible scale—is leading design and product teams to make decisions backed by the largest datasets they have ever worked with and to be wrong about the people those datasets are supposed to represent.

This is not an argument against data. I have spent 25 years working with data in design and research practice, and I believe in its power when it is used correctly. This is an argument about what data can and cannot tell you—a distinction that AI's scale has made more urgent, not less.

Where the myth comes from

The belief that more data produces better insight has a seductive logic. In a world where design decisions were historically made based on small samples—twelve user interviews, a usability study with eight participants, and a survey with a few hundred responses—the arrival of AI tools capable of processing millions of data points feels like a revolution. And in some respects, it is.

A design team that previously had to make decisions about a checkout flow based on 50 session recordings can now have AI process 50,000. A research team that previously coded three hundred survey responses manually can now have AI synthesize thirty thousand. A product team that previously relied on a handful of support tickets to understand a feature's problems can now have AI analyze every support interaction from the past three years.

The quantity is genuinely new. The belief that quantity equals understanding is not true. It is a very old mistake dressed in new technology.

The mistake is this: data tells you what happened. It does not tell you why. It tells you that users dropped off at Step 3. It does not tell you what they were thinking when they dropped off, what prevented them from continuing, or whether their abandonment was a signal of confusion or a rational decision given their specific situation. It tells you that a feature has a low engagement rate. It does not tell you whether low engagement means users do not value the feature, cannot find it, have found an alternative way to accomplish the same goal, or simply have not yet discovered it.

AI does not resolve this limitation. It amplifies it. When you process fifty thousand session recordings through AI and receive a pattern map showing that most users drop off at Step 3, you have a very large-scale confirmation of something you could have learned from fifty recordings. What you still do not have—what no amount of additional data will give you—is the why. And the why is where the design insight lives.

"Analytics mostly tell you what happened—not why it happened. That's why they work best alongside qualitative research to explain the reasons behind the behavior."— Medium/Saisuryakarthik (2026)

The Walmart lesson: What automated data gets wrong

There is a case study from retail that has become one of the most instructive examples of the gap between data and insight—and it is worth examining in detail because it illustrates exactly the kind of mistake that AI-scale data processing is now enabling design teams to make more efficiently.

Walmart ran large-scale automated surveys asking customers to rate their agreement with statements about their shopping experience. The surveys produced enormous datasets—hundreds of thousands of responses—and the AI-processed patterns showed high agreement rates across most dimensions. The data looked positive. The insight was absent.

What the automated surveys had done was ask for agreement, not understanding. They had asked customers to evaluate their experience against the dimensions Walmart had decided were important—not against the dimensions that customers themselves would have identified as relevant. The customers who rated their experience positively were rating it positively on the terms the survey provided. They were not revealing what actually mattered to them, what they would have changed, or what was driving the decisions—including the decision to shop elsewhere—that the agreement ratings did not capture.

The misstep, as Ethnio's research on UX automation identified, was treating automated data collection as a substitute for human insight. The data was real. The sample was large. The processing was efficient. And the understanding of why customers behaved the way they did was absent.

This is the pattern that AI-scale data processing is replicating in product and design organizations. The data volume is unprecedented. The understanding of why users behave the way they do is often no better—and sometimes worse—because the false confidence that large-scale data produces is harder to challenge than the honest uncertainty that small-scale research generates.

The difference between pattern and meaning

Let me make the distinction precise, because it is the conceptual foundation of this entire article.

A pattern is a regularity in data. It tells you what has occurred consistently across a dataset. A million users navigate from the homepage to the product page before converting. Sixty percent of users who reach the checkout abandon at the payment step. Users who engage with the onboarding tutorial have a 40% higher 30-day retention rate. These are patterns. They are real. They are useful. And they are not insight.

Insight is meaning—the explanation of why a pattern exists and what it implies for design. Why do 60% of users abandon at the payment step? Is it because the payment form is too long? Because it asks for information users do not have readily available? Because it triggers a trust concern that the earlier product experience did not resolve? Because the users who abandon have a specific characteristic—they are mobile users, they are first-time purchasers, they are users from a specific region where a specific payment method is not available—that the aggregate pattern conceals?

The answer to that question is not in the data. It is in the mind, and the situation of the person who abandoned—and the only way to access it is to engage with that person directly, with the skill and the presence that qualitative research requires.

Market Xcel's 2025 research on human versus AI insight generation identified a case study that makes this vividly concrete. AI-based behavioral analytics in a UX testing context misinterpreted hesitation—a pause before a user action—as confusion and a design failure. Human researchers in the same study recognized the hesitation as deliberate exploration—a positive engagement indicator. Same data. Same pattern. Completely different meaning. The difference between them was not more data. It was human judgment applied to the pattern.

AI cannot make that judgment. It can identify that hesitation occurred. It cannot know whether that hesitation is confusion, deliberation, exploration, or the particular way that a specific person interacts with a digital interface because of an accessibility need the AI has no way to model. The judgment belongs to the researcher. And the researcher needs to be in the room—or on the screen, with the real user—to exercise it.

"Data without meaning is noise, and AI without human oversight is a risk. Two companies with identical analytics can arrive at opposite strategic decisions because human judgment defines interpretation."— Market Xcel (2026)

Why scale specifically makes this problem worse

Here is the counterintuitive element of this argument, and the one I want to make most carefully.

The problem with the "more data equals more insight" belief is not just that it is wrong in the way that all data-without-interpretation is wrong. It is that the scale of AI data processing makes the specific errors it produces harder to detect and harder to challenge.

When a design decision is based on fifty user interviews, the researchers who conducted those interviews carry a specific memory of the uncertainty in their findings. They know which participants were atypical. They know which insights were unexpected. They know which questions their data did not answer. They hold the complexity of what they learned alongside the patterns they identified. And when a stakeholder challenges their conclusions, they can articulate the limits of their evidence with specificity.

When a design decision is based on AI-processed patterns from fifty thousand session recordings, no researcher holds that memory. The AI has summarized the patterns and discarded the complexity. The fifty thousand individual user experiences—each one a specific person in a specific situation making a specific decision for a specific reason—have been reduced to a thematic map that appears authoritative because of its scale and loses the uncertainty that the scale should have preserved.

The UX researcher who says, "We saw this pattern in twelve interviews, and we are uncertain whether it generalizes," is expressing the honest epistemic position of someone who knows the limits of their evidence. The AI system that says "this pattern appears in 73% of user sessions" presents the same uncertainty with none of the caveats that would allow a decision-maker to evaluate it appropriately. The 73% looks more authoritative than the twelve interviews. It may be less informative.

This is the scale problem. Not that large-scale data is useless—it is not. But the confidence it produces in decision-makers is frequently disproportionate to the understanding it provides. And AI, which processes that data at unprecedented speed and presents its patterns with confident authority, amplifies the confidence without resolving the understanding gap.

What large-scale data is actually good for, and what it needs

I want to be precise about this because the argument is not that data is bad or that scale is worthless. It is that data is one component of a research practice—a component with specific strengths and specific limitations—and that treating it as a substitute for the other components produces the specific failures this article is describing.

  • Large-scale behavioral data—session recordings, analytics, A/B test results, and survey responses—is genuinely valuable for identifying where problems exist. The pattern that most users drop off at Step 3 is a real finding. It tells you where to focus your research attention. It does not tell you what to find there.
  • It is genuinely valuable for validating qualitative hypotheses at scale. When qualitative research generates a hypothesis—"users are abandoning at Step 3 because the payment form requires information they do not have readily accessible"—large-scale data can test whether that hypothesis is consistent with the patterns in the broader dataset. Hypothesis validation is a legitimate use of a scale. Hypothesis generation is not.
  • It is genuinely valuable for tracking the impact of design changes over time. After a design intervention, large-scale data provides the most reliable signal about whether the intervention has changed user behavior in the intended direction. This is a retrospective measurement—understanding what happened after a design decision—which is a different and valid use of data than understanding why users behave as they do before a design decision.

What large-scale data needs, in all of these legitimate uses, is the qualitative research that provides the meaning the data cannot generate itself. The interviews and observations that explain why the pattern exists. The usability studies that reveal what users are thinking at the moment the pattern occurs. The contextual inquiry that situates the data in the specific lives and situations of the specific people whose behavior it represents.

This combination—qualitative understanding and quantitative confirmation—is what the Loop11 research on UX research trends for 2025 and 2026 identifies as the hybrid methodology that characterizes the most effective research practices. Not AI data processing replacing qualitative research. Both, each doing what it does best, in a research practice that is designed to integrate them.

The Indian context: When missing data means missing people

In India, the "more data equals more insight" myth carries an additional and specific risk that the global conversation about AI and data frequently overlooks.

The users who are most extensively represented in digital data—in session recordings, analytics, app usage patterns, support interactions, and online surveys—are the users who are already well-served by digital products. They have reliable internet connections. They have smartphones capable of the interactions the product requires. They are digitally literate enough to use the product in ways that generate usable data. They are comfortable enough with digital interfaces to give feedback through the channels the product provides.

The users who are least represented in digital data are the users who face the greatest barriers—limited connectivity, limited device capability, limited digital literacy, and limited comfort with the dominant language of the interface. And in India, these users are not a small minority. They are a significant proportion of the actual population that digital products are supposed to serve.

When a product team makes design decisions based on AI-processed patterns from their existing user dataset, they are making decisions based on the behavior of users who have already successfully navigated enough of the product to generate data. The users who abandoned before they could generate meaningful data—the users who could not load the app on their device, who could not read the interface, who gave up before reaching the first screen—are invisible in the dataset. The AI that processes the data has no way to represent it. They are absent from the patterns.

The product that is designed based on that dataset is a product designed for users who are already succeeding. It is not designed for the users who are failing—and failing silently, in ways that no amount of AI data processing will surface.

This is why qualitative research in the Indian context—research conducted with users who are at the edges of digital access, in the field, in their actual environments, with their actual devices and their actual connectivity—is not a complement to data-driven design. It is a prerequisite for design that serves the full population of users the product is supposed to reach.

Applying LucyUX: The right role for data at each stage

The LucyUX framework—Listen, Understand, Conceptualize, Yield—provides a precise map of where data belongs in the research and design process and where it does not.

  • Listen: The listening stage of LucyUX is qualitative. It is the stage where you encounter real users in real contexts and allow what you observe and hear to shape your understanding of the design problem. Data has no role in listening. Data is a record of what has happened. Listening is an encounter with what is happening—with the specific, situated, contextually embedded reality of a specific person's experience. You cannot listen to a dataset. You can only listen to a person.
  • Understand: The understanding stage integrates data with qualitative insight. Having listened—having built a qualitative understanding of the user's experience—you use data to test whether what you observed is consistent with what has happened at scale. Does the pattern in the session recordings confirm the hypothesis that qualitative research generated? Does the analytics data validate the findings from the interviews, or does it challenge them in ways that require further investigation? Understanding integrates both. It is not achievable from either alone.
  • Conceptualize: The conceptualization stage uses both qualitative understanding and quantitative validation to generate design directions that are specifically responsive to the specific reality of the specific users you have researched. The data tells you where to focus. The qualitative understanding tells you what design response the focus demands. Both are necessary. Neither is sufficient without the other.
  • Yield: The yield stage measures impact on real users—not on the data patterns that preceded the design decision. Did the design change improve the experience of the specific users whose qualitative research informed it? Did it serve the users who were previously underrepresented in the data? Did it improve the product for the users who were failing silently before the design intervention gave them a voice?

"Even with AI and new tools, traditional methods remain the core of good UX research. AI handles the heavy lifting of data processing and pattern recognition, while human researchers bring irreplaceable skills like empathy, contextual understanding, and ethical judgment."— Optimal Workshop (2026)

Your action this week

Take one data-based design decision that your team has made or is currently making. A decision supported by analytics, A/B test results, session recordings, or AI-processed survey data.

Ask two questions about that decision. First, what does the data tell us happened? Name the pattern specifically—not "users have trouble with checkout" but "63% of users who reach the payment step do not complete a purchase." Second, what does the data not tell us? What is the why that is absent from the pattern? What would we need to know—from what source, through what research method—to understand why this pattern exists and what design response it demands?

The gap between those two questions is the gap between data and insight. Making that gap visible—naming it specifically, in the context of a real decision your team is making—is the beginning of a research practice that uses AI-scale data for what it is genuinely good for, without mistaking it for the understanding that only direct human research can produce.

My perspective: What I actually believe

The design community is at risk of making a specific and consequential error in its adoption of AI: mistaking the scale of data processing for the depth of user understanding. These are not the same thing. They have never been the same thing. And the arrival of AI tools that can process user data at an unprecedented scale has not changed that—it has only made the error more seductive and its consequences more difficult to detect.

I believe in data. I believe in AI's ability to process data at a scale that genuinely advances design practice. And I believe, with equal conviction, that data without interpretation is noise, and that interpretation requires the human researcher—present, attentive, and willing to be surprised by what a real person reveals about their experience of a product that was designed without sufficient understanding of who they actually are.

More data is not more insight. More insight is more insight. And insight comes from the encounter with the real human being, not from the scale of the data that human being has contributed to.

Up next in the "UX × AI" series: “Designing with AI, Not for It.“ The tools you use shape what you make. Every medium has a bias—toward certain kinds of outputs, certain aesthetic conventions, and certain design directions. AI design tools are no different. In Part 6, we examine how AI tools are shaping design outputs in ways that designers are not always aware of and how to stay the author of your work rather than becoming its editor.

References & further reading