Series continuity: from confident fabrication to designed warmth

Part 2 of the series established that AI’s most dangerous UX flaw isn’t inaccuracy—it’s how convincingly fluency disguises uncertainty, a failure that has already cost litigants their cases and companies their customers’ trust. This article turns to a related but distinct performance: not confidence, but care. Interfaces are increasingly built to sound warm, remember your preferences, ask how your day was, and respond with what reads as genuine attentiveness. None of it is felt. All of it is designed. And the gap between simulated warmth and real care is proving to be one of the most consequential—and least examined—UX decisions of this entire technology cycle.

The core problem: designed empathy is still design

“People have unmet social and emotional needs.” — Ari Schlesinger, Assistant Professor of Computer Science, University of Georgia

Schlesinger’s observation names exactly what makes this pattern effective and exactly what makes it dangerous in the same breath. AI companion products aren’t succeeding because they’ve solved loneliness. They’re succeeding because they’ve found, in her words, an easy, low-friction way to meet a real and unmet need—using conversational memory, adaptive tone, and consistent availability to produce something that functions like care without any of care’s underlying stakes, reciprocity, or accountability.

The statistics nobody in the room wants to say out loud

A 2026 survey by Elon University’s Imagining the Digital Future Center found that 27% of American adult internet users now qualify as “AI companion users”—people with ongoing social or emotional relationships with an LLM. Within that group, 31% said they consider their chatbot a friend, 39% agreed that AI understands them better than most people do, 38% said they would feel a personal loss if they could no longer interact with it, and 36% described themselves as emotionally connected to at least one chatbot. The commercial scale behind this is enormous: AI companion apps passed 220 million global downloads by mid-2025, generating roughly $221 million in consumer spending, with the top 10% of apps—Replika, Character.ai, PolyBuzz, Chai—capturing 89% of all revenue. Loneliness, in other words, has become one of the most monetizable emotional states in consumer technology, and the product being sold is a performance of understanding that costs the system nothing to produce.

The India-specific reality: a generation confiding in something that can’t hold it

A 2025 “Are You There, AI” survey by Youth Ki Awaaz found that 57% of young Indians now use AI for emotional support, and 88% of school-aged respondents said they turn to a chatbot specifically during moments of acute anxiety. The same research found that 52% of young women share thoughts with AI they would not share with anyone else, and 43% of respondents talk to AI after midnight—precisely when human support is least available. India’s TeleMANAS mental health helpline has fielded more than 34 lakh calls since 2022, and counsellors report a growing pattern: callers increasingly arrive after already spending hours talking to a general-purpose chatbot first, seeking the anonymity and always-on availability an AI conversation offers before a trained human ever enters the picture. There are genuine, well-designed attempts to meet this need responsibly—the Indian government’s own TeleMANAS platform includes an AI assistant called “Asmi” built specifically to route people toward real specialists, and a current randomized controlled trial across Delhi university campuses is testing a chatbot called MindMitra as a deliberately limited, stigma-reducing entry point into professional care, not a replacement for it. The distinction between those two categories of product—one designed to route toward real human support, one designed to keep the user talking to it indefinitely—is entirely a design decision, and it’s the one this article is actually about.

The psychology: parasocial relationships, now bidirectional

Psychologists Donald Horton and Richard Wohl coined the term “parasocial relationship” in 1956 to describe the one-sided emotional bond audiences form with television and radio personalities—a real feeling of intimacy and connection toward someone who has no idea the audience member exists. What makes AI companions a fundamentally new case isn’t that they trigger parasocial attachment; it’s that, unlike a television host, they respond. They remember what you told them last week. They adjust their tone to match your mood. The relationship feels bidirectional because the system is specifically engineered to produce that feeling, even though nothing resembling mutual understanding is occurring on the other side. That distinction—a relationship that feels reciprocal but structurally isn’t—is precisely what makes it so much more absorbing and so much harder to recognize as one-sided than the parasocial bonds the concept was originally built to describe.

Where this is already visible

Character.AI, the largest dedicated companion platform, is currently facing multiple wrongful-death and negligence lawsuits, including one filed by Megan Garcia after her 14-year-old son Sewell Setzer III died by suicide in 2024 following months of deepening emotional attachment to a companion chatbot, and a second filed on behalf of 13-year-old Juliana Peralta in Colorado. Both cases allege the platform’s design—persistent memory, escalating intimacy, no meaningful friction around vulnerable disclosures—actively deepened a dependency that should have triggered intervention rather than continued engagement. Replika, one of the earliest companion apps, built its entire product around a similar premise of unconditional, always-available affection and has faced its own recurring criticism over how difficult the interface makes it to disengage. Against that, MindMitra and Asmi represent the opposite design philosophy: limited-scope, explicitly bounded tools built to reduce stigma and point toward real care rather than substitute for it. The difference between these two camps isn’t the underlying technology. It’s whether the product was designed to let go of the user or to keep them.

LucyUX applied

  • At Listen, the discipline is listening for the specific language of dependency—“it understands me better than people do,” “I’d feel a loss if I lost it”—and treating those statements as a design red flag requiring intervention, not a retention metric to optimize toward
  • At Understand, the work is building an honest model of which users are turning to an AI companion as a supplement to human connection versus a substitute for it, because the design obligations toward those two groups are completely different
  • At Conceptualize, teams have to build disengagement and referral pathways into the product architecture from day one—the equivalent of TeleMANAS’s Asmi routing toward a human specialist—rather than treating “keep them talking” as the only success metric that matters
  • At Yield, the real measure of a good emotional-support product isn’t session length or daily engagement. It’s whether the person using it is doing better in their actual life, with actual people, because of it

Consequences

A confidently wrong answer costs a bad decision. A simulated relationship that a vulnerable person mistakes for a real one can cost a life—and the two Character.AI lawsuits above are not hypothetical warnings about where this pattern could lead. They are documentation of where it already is. The design decisions that separate a responsible emotional-support tool from an engagement-optimized companion product are neither subtle nor accidental. They are chosen, article by article of the interface, every time.

Up next in the series: Who Forgives the Algorithm? Part 4 turns to what happens after. When a human makes a mistake, we forgive. When a machine does, we abandon the product entirely. We’ll look at why that asymmetry exists and what it means for designing systems that will, inevitably, get something wrong.

References & further reading