Flag

We stand with Ukraine and our team members from Ukraine. Here are ways you can help

Home ›› Artificial Intelligence ›› How Can User Experience Research (UXR) Help Build Users Trust in AI Systems and Increase Engagement?

How Can User Experience Research (UXR) Help Build Users Trust in AI Systems and Increase Engagement?

by Celine Lenoble
4 min read
Share this post on
Tweet
Share
Post
Share
Email
Print

Save

With ML facing so many users, there is a case to approach the conception and design of ML-powered applications from a UX research perspective. Read more to find out.

Today, Artificial Intelligence (AI) and, more specifically, Machine Learning are pervasive in our daily lives. From Facebook ads to YouTube recommendations; from Siri to Google Assistant; and from automated translation of device notice to marketing personalization tools; AI now deeply permeates both our work and personal lives.

This article is the first in a series of three that advocate for renewed UX research efforts in ML apps.

With ML facing so many users, there is a case to approach the conception and design of ML-powered applications from a UX research perspective.

This lies on three main reasons:

  1. Mental models of users haven’t caught up with how ML and AI truly work.
  • UXR can uncover existing mental mentals and help design new ones that are more suited to this new tech.

2. ML and AI can have an insidious and deep impact on all users’ lives

  • UXR reveals the myriad of intended and unintended effects of apps on people’s life — and help build more ethical AI.

3. ML and AI can have disparate impacts on individuals based on their ethnicity, religion, gender, sexual orientation:

  • UXR can also help address some of the sources of bias.

In this episode, we will focus on the first reason: How can UXR help build trust in AI systems and increase users’ engagement?

ML and Real Users

Users’ attitudes towards ML-powered apps are complex. Algorithm aversion has been well studied and documented:

In a wide variety of forecasting domains, experts and laypeople remain resistant to using algorithms, often opting to use forecasts made by an inferior human rather than forecasts made by a superior algorithm. Indeed, research shows that people often prefer humans’ forecasts to algorithms’ forecasts (Diab, Pui, Yankelevich, & Highhouse, 2011; Eastwood, Snook, & Luther, 2012), more strongly weigh human input than algorithmic input (Önkal, Goodwin, Thomson, Gönül, & Pollock, 2009; Promberger & Baron, 2006), and more harshly judge professionals who seek out advice from an algorithm rather than from a human (Shaffer, Probst, Merkle, Arkes, & Medow, 2013).

Dietvorst, B. J., Simmons, J. P., & Massey, C. (2015). Algorithm aversion: People erroneously avoid algorithms after seeing them err. Journal of Experimental Psychology: General, 144(1), 114–126. 

However, their research shows that this algorithm aversion phenomenon appears only once humans witness, or are made aware of, forecasting errors. In 2019, Logg J.M., Minson J.A., Moore D. A. demonstrated the contrary, that humans show an initial appreciation towards algorithm advice compared to fellow humans:

Our participants relied more on identical advice when they thought it came from an algorithm than from other people. They displayed this algorithm appreciation when making visual estimates and when predicting: geopolitical and business events, the popularity of songs, and romantic attraction. Additionally, they chose algorithmic judgment over human judgment when given the choice. They even showed a willingness to choose algorithmic advice over their own judgment.

Logg J.M., Minson J.A., Moore D. A. (2019). Algorithm appreciation: People prefer algorithmic to human judgment. Organizational Behavior and Human Decision Processes, vol. 151, March 2019, 90–103.

ML and The Theory of Machine

One possible explanation still being investigated is “The Theory of Machine” (equivalent to the “Theory of Mind” for humans) that people operate with. The Theory of Machine, or more simply, mental models, as designers call it, is a series of assumptions humans make on how an application works internally.

One such assumption is the idea of a fixed mindset. Having a fixed mindset in psychology means you believe people have a certain amount of intelligence or skills, and they can’t do anything to increase that amount. Applied to a Theory of Machine, it means that people believe that a computer program output is fully determined by the initial input and not capable of learning or evolving.

The fixed mindset applied towards traditional software was appropriate for a long time. Your typical software, word processor, or spreadsheet was not capable to improve on its own and learn from its mistakes. The user might expect changes following an update, but otherwise, they expect the program to behave consistently over time.

When confronted with ML-powered applications, users continue to apply the classic fixed mindset mental model. So, once they experience what they perceive as the app making a mistake, they completely lose trust in the system’s ability to give accurate results. This is possibly what triggers the shift to algorithm aversion, after an initial appreciation.

Numerous ML apps presents themselves as an assistant. They draw on the mental model of a relationship with a person, hoping to change the assumptions users make on how the program works.

This choice of mental model presents several challenges:

  • AI is not (yet) powerful enough to pass for a human: Users’ expectations are shaped by how they expect a human to respond, and users typically end up extremely disappointed, if not infuriated, by the AI behavior.
  • Even for their fellow humans, people tend to apply a fixed mindset and rarely allow for the possibility of growth and change in capabilities, at least not in any short time frame.
  • If users do have a growth mindset in relation to humans, meaning that they believe humans can improve provided they are given the opportunities to learn or they are taught what to do, this mindset doesn’t transfer well to AI assistants, because the learning modalities of humans and AI are so different.

Mental Model and User Engagement with ML Apps

What mental model should you use then? There is no one-size-fits-all answer to this question. This is where User Experience Research is required:

  • to uncover the existing mental models associated with specific tasks,
  • to experiment with multiple UI metaphors beyond the assistant, and
  • to help users adjust their existing mental models and expectations to the reality of ML-powered apps.
post authorCeline Lenoble

Celine Lenoble

I am Director of UX Research at brainlabs where I lead a team of CRO analysts and UX researchers with diverse backgrounds. I believe in a holistic approach to UX research and design combining all perspectives: Human Computer Interaction, design thinking, psychology, sociology, anthropology and all methods: from big data to ethnographic study. I am particularly interested in the UX of ML-powered products & services. Disclaimer: opinions represented here are personal and do not represent those of brainlabs.

Tweet
Share
Post
Share
Email
Print
Ideas In Brief
  • The article covers the conception and design of ML-powered applications from a UX research perspective.
  • The author unpacks the following ideas:
    • Machine Learning and Real Users
    • Machine Learning and The Theory of Machine
    • Mental Models and User Engagement with Machine Learning Apps

Related Articles

Curious about the next frontier in AI design? Discover how AI can go beyond chatbots to create seamless, context-aware interactions that anticipate user needs. Dive into the future of AI in UX design with this insightful article!

Article by Maximillian Piras
When Words Cannot Describe: Designing For AI Beyond Conversational Interfaces
  • The article explores the future of AI design, moving beyond simple chatbots to more sophisticated, integrated systems.
  • It argues that while conversational interfaces have been the focus, the potential for AI lies in creating seamless, contextual interactions across different platforms and devices.
  • The piece highlights the importance of understanding user intent and context, advocating for AI systems that can anticipate needs and provide personalized experiences.
Share:When Words Cannot Describe: Designing For AI Beyond Conversational Interfaces
21 min read

Discover how Flux.1, with its groundbreaking 12 billion parameters, sets a new benchmark in AI image generation. This article explores its advancements over Midjourney and Dall-E 3, showcasing its unmatched detail and prompt accuracy. Don’t miss out on seeing how this latest model redefines what’s possible in digital artistry!

Article by Jim Clyde Monge
Flux.1 is a Mind-Blowing Open-Weights AI Image Generator with 12B Parameters
  • This article examines Flux.1’s 12 billion parameters and its advancements over Midjourney and Dall-E 3. Highlights its superior image detail and prompt adherence.
  • The piece explores the shift of developers from Stability AI to Black Forest Labs and how this led to Flux.1. Analyzes the innovation impact.
  • It compares Flux.1 with Midjourney V6, Dall-E 3, and SD3 Ultra, focusing on visual quality, prompt coherence, and diversity.
  • The guide explains how to access Flux.1 via Replicate, HuggingFace, and Fal. Covers the different models—Pro, Dev, Schnell—and their uses.
  • The article investigates Flux.1’s capabilities in generating photorealistic and artistic images with examples of its realism and detailed rendering.
Share:Flux.1 is a Mind-Blowing Open-Weights AI Image Generator with 12B Parameters
5 min read

Is true consciousness in computers a possibility, or merely a fantasy? The article delves into the philosophical and scientific debates surrounding the nature of consciousness and its potential in AI. Explore why modern neuroscience and AI fall short of creating genuine awareness, the limits of current technology, and the profound philosophical questions that challenge our understanding of mind and machine. Discover why the pursuit of conscious machines might be more about myth than reality.

Article by Peter D'Autry
Why Computers Can’t Be Conscious
  • The article examines why computers, despite advancements, cannot achieve consciousness like humans. It challenges the assumption that mimicking human behavior equates to genuine consciousness.
  • It critiques the reductionist approach of equating neural activity with consciousness and argues that the “hard problem” of consciousness remains unsolved. The piece also discusses the limitations of both neuroscience and AI in addressing this problem.
  • The article disputes the notion that increasing complexity in AI will lead to consciousness, highlighting that understanding and experience cannot be solely derived from computational processes.
  • It emphasizes the importance of physical interaction and the lived experience in consciousness, arguing that AI lacks the embodied context necessary for genuine understanding and consciousness.
Share:Why Computers Can’t Be Conscious
18 min read

Tell us about you. Enroll in the course.

    This website uses cookies to ensure you get the best experience on our website. Check our privacy policy and