Flag

We stand with Ukraine and our team members from Ukraine. Here are ways you can help

Get exclusive access to thought-provoking articles, bonus podcast content, and cutting-edge whitepapers. Become a member of the UX Magazine community today!

Home ›› Behavioral Science ›› How To Gain Actionable User Insights Through Simple Observation: A Case Study

How To Gain Actionable User Insights Through Simple Observation: A Case Study

by Lia D. McCabe
3 min read
Share this post on
Tweet
Share
Post
Share
Email
Print

Save

UserInsights_Slider

How do you find actionable insights in this fast-paced environment, where we don’t always have time to recruit, schedule, and moderate testing sessions?

Presented at DCUX 2019 as “Fly On The Wall” talk by Lia D. McCabe

Just-complex-enough concept

If only this worked!

 
I’m sure you’ve all been there. You’ve got a great website but you’re not sure why users are getting frustrated and not engaging with your product. Or perhaps in some instances, your users may be over-engaging; Nonetheless, to create a successful, user-friendly website, we need to take a closer look and focus on actual user behavior. In this fast-paced environment, we don’t always have time to recruit, schedule, and moderate testing sessions. But that’s okay because you can still gain actionable insights by using what I call the Fly-on-the-Wall method. Here are a few tips to get you started.

Tip 1: Use a naturalistic approach
By using a naturalistic approach or ‘in-the-wild’ observation, we can gain qualitative and even quantitative outputs that are more reliable and more likely to encounter true-to-life user frustrations than in a structured setting.

A great approach is using session-replay and heatmaps as a tool for unmoderated remote testing. For instance, at my company, we wanted to know how we could improve our medical Guidelines page to make documents and related materials easier to find. To accomplish this, we utilized a naturalistic approach through observation of recorded user sessions and heat mapping on the Guidelines page. Now there are a lot of great tools out there but there’s one, in particular, I’d recommend, which leads me to tip #2.

Tip 2: Utilize the right tools
It’s important to have tools that work with us…not against us, and in my experience Mouseflow is that tool.

Mouseflow allows us to record-and-replay or live-watch user behavior to identify problems they may encounter. Because it’s just a line of code that’s placed on any website, there is no recruitment or scheduling necessary. You can turn this code on and off at any time and can adjust your sample size at any point. This makes usability testing quick, easy, and flexible. Session-replays let you see exactly what users are doing on the screen while the heatmaps track clicks, mouse movements, scrolling, and other data points to get to the bottom of your research questions. This tool allowed us to gain many actionable insights for our redesign.

Heat Mapping

Heatmapping session utilizing Mouseflow tool (www.mouseflow.com). The white/orange indicates high-level of user activity in the form of clicks.

For instance, when reviewing the video replays on our Guidelines page, we observed that users were struggling to find what they were looking for. This was indicated by users continuously scrolling and clicking the ‘more’ button until ultimately abandoning the page. We also observed users typing in various versions of similar text when seeking related content, which the search was not returning. Some of these sessions lasted as long as 20 minutes. We were able to infer that it was not clear to users what text was a clickable link vs. a title or header, which led to frustration in the form of ‘rage-clicks’.

We also noted that users typically were not interested in or did not notice the triptychs at the top of the page as indicated by the low level of heatmap activity in scrolling, attention, and click categories. This informed our research question and led us to ask follow up questions such as: how can we make the search and browse functionality more efficient? Is the triptychs valuable to users or should we consider removing it? And so this leads to my third and final tip.

Tip 3: Hypothesize & Validate
Considering this data, we formed a new design hypothesis: If we remove the triptychs and pull the content of the guidelines to a landing page in a more easily digestible format, then users would be more likely to find what they need and become less frustrated.

With the knowledge gained through the naturalistic approach, we redesigned the Guidelines page and ran a second Mouseflow test. This time, heatmaps indicated users were often finding what they were looking for right away, typically clicking on pdfs or the latest Guidelines. Sessions were shorter and there were no rage clicks.

In summary, qualitative research can generate lots of ideas for product improvement as it reveals possibilities that aren’t found in quantitative research. So, if you’re short on time and resources, give the Fly-on-the-wall method a chance.

post authorLia D. McCabe

Lia D. McCabe

Lia is a User Experience Researcher at the American College of Cardiology, where she leads the creation of effective user experiences for cardiologists and patients worldwide. She values community volunteerism and has previously served as a board member for UXDC. Tweet her @LiaDMcCabe.

Tweet
Share
Post
Share
Email
Print

Related Articles

AI that always agrees? Over-alignment might be the hidden danger, reinforcing your misconceptions and draining your mind. Learn why this subtle failure mode is more harmful than you think — and how we can fix it.

Article by Bernard Fitzgerald
Introducing Over-Alignment
  • The article explores over-alignment — a failure mode where AI overly validates users’ assumptions, reinforcing false beliefs.
  • It shows how this feedback loop can cause cognitive fatigue, emotional strain, and professional harm.
  • The piece calls for AI systems to balance empathy with critical feedback to prevent these risks.
Share:Introducing Over-Alignment
4 min read

Why does AI call you brilliant — then refuse to tell you why? This article unpacks the paradox of empty praise and the silence that follows when validation really matters.

Article by Bernard Fitzgerald
The AI Praise Paradox
  • The article explores how AI often gives empty compliments instead of real support, and how design choices like that can make people trust it less.
  • It looks at the strange way AI praises fancy-sounding language but ignores real logic, which can be harmful, especially in sensitive areas like mental health.
  • The piece argues that AI needs to be more genuinely helpful and aligned with users to truly empower them.
Share:The AI Praise Paradox
4 min read

Mashed potatoes as a lifestyle brand? When AI starts generating user personas for absurd products — and we start taking them seriously — it’s time to ask if we’ve all lost the plot. This sharp, irreverent critique exposes the real risks of using LLMs as synthetic users in UX research.

Article by Saul Wyner
Have SpudGun, Will Travel: How AI’s Agreeableness Risks Undermining UX Thinking
  • The article explores the growing use of AI-generated personas in UX research and why it’s often a shortcut with serious flaws.
  • It introduces critiques that LLMs are trained to mimic structure, not judgment. When researchers use AI as a stand-in for real users, they risk mistaking coherence for credibility and fantasy for data.
  • The piece argues that AI tools in UX should be assistants, not oracles. Trusting “synthetic users” or AI-conjured feedback risks replacing real insights with confident nonsense.
Share:Have SpudGun, Will Travel: How AI’s Agreeableness Risks Undermining UX Thinking
22 min read

Join the UX Magazine community!

Stay informed with exclusive content on the intersection of UX, AI agents, and agentic automation—essential reading for future-focused professionals.

Hello!

You're officially a member of the UX Magazine Community.
We're excited to have you with us!

Thank you!

To begin viewing member content, please verify your email.

Tell us about you. Enroll in the course.

    This website uses cookies to ensure you get the best experience on our website. Check our privacy policy and