UX Magazine

Defining and Informing the Complex Field of User Experience (UX)
Article No. 546 August 17, 2010

Doing More with Remote Research

:
Exploring Physical and Social Contexts

Chances are, you've encountered remote user experience research. Perhaps you've read about it in an article or have seen the book. Everyone from the scrappiest startup to the most established corporation is beginning to incorporate remote methods into their research arsenal.

Currently, the most common remote research protocols incorporate screen sharing and task analysis, often employed at a late stage in the product development process. While screen sharing is a relatively new technology, task analysis has been the standard usability method for 30 years. But since then, user habits and behaviors have changed significantly to accommodate the tech-filled world we live in.

Instead of buying a washing machine by walking into Sears, today people can choose from many different washing machines by reading online reviews, asking friends on Facebook for recommendations, or doing head-to head-comparisons across manufacturers. Thus doing a remote research study focusing solely on the Sears website would not capture these other activities that are part of the buying process, even if the purchase is eventually completed on sears.com.

We see real value in the ways remote research can contribute to developing a broader view of what people actually do in their natural environments. Think of it like low-cost field-testing for the purpose of understanding rich and complex real-world experiences. This can be done to a limited degree through screen sharing of website interactions, but as interactions span the physical and digital worlds and have myriad social influences, remote research can be a valuable contributor to understanding these touchpoints as well.

Thus, remote methods are particularly suited for understanding context of use. We should be asking questions that reveal the broader context in which customers are using a website. For example:

  • What brought users to the website, and what will they be doing when they leave?
  • How long have they been doing a given task?
  • What other sources do they use for getting information and how does that influence their experience on our website?
  • Are they making decisions with their friends?
  • Are they doing things offline that contribute to the experience?

These are hard questions to answer even in full-blown field studies. Because remote methods give you access to users in their native environments, it's much more likely that you'll gain insight about physical and social contexts in a remote study than in a laboratory study.

Remote research for physical contexts

We recently used remote methods to help LendAround, a London-based startup, learn about potential customers' DVD sharing habits. LendAround's platform for peer-to-peer physical media sharing (primarily of DVDs) was in early beta. Needing to move quickly and be frugal meant that in-person field research was not feasible, particularly since the team wanted to focus on participants who lived outside of the Bay Area tech bubble where the research project was based.

The team developed a study that involved remotely recruiting and interviewing people in their homes where their DVD collections were kept. To begin, they posted a survey on Mechanical Turk and paid "Turkers" to complete it. The survey results were used for two purposes: to identify target interview participants and to address some initial research questions.

In the next phase, we conducted research using that most basic of tools—the phone. Our questions covered many aspects of how participants share and borrow DVDs. And since DVDs are physical products that are kept in people's homes, it was important to interview participants in the environment where they had their DVD collections. When we asked about their sharing habits, they could simply walk over to where they kept their DVDs to look around and estimate how many they had and how many they'd lent to friends. This led to insights about the rich context of how people use and share DVDs that wouldn't have been captured in an in-person laboratory study.

Remote research for social contexts

We have also used remote methods to learn about online social interactions. One study in particular was focused on how services like Twitter, Facebook, and Flickr help friends connect, discover content, and synthesize new information. Our goal was to understand how social interactions affect perceived content value within a set of friends.

We set up a remote study in which three friends submitted desktop recordings of their online interactions over the course of three weeks. We paid particular attention to times when one participant was referenced by or communicated with another participant, and we asked both of them about the interaction. Also, when multiple participants read the same news item we asked each person whether they remembered the content, whether it was meaningful, and whether other people had talked or commented on that story since.

It became clear that people's interactions and the value they found in social content were dependent on a number of factors, from context cues they picked up in the interface to social factors such as reputation and trust. For example, participants skimmed Twitter focusing only on avatars, and when they saw an avatar of someone they respected they read the associated tweet more closely and were more likely to share it with others.

Doing this study remotely allowed us to uncover social interaction patterns that would not have been captured in an hour-long screen share/task analysis of a single participant's desktop.

Conclusion

Though late-stage remote task analysis is valuable, remote methods can also establish an understanding of the unspoken and often unseen contexts, both physical and social, of a product, program, website, interaction, or decision-making process. Creative and thoughtful use of remote research methods can improve product development practices across the board—for startups and large, established companies alike.

ABOUT THE AUTHOR(S)

User Profile

Juliette Melton is a user experience researcher and design strategist based in San Francisco. You can find her on Twitter at @j.

User Profile

Brynn (@brynn) is obsessed with the intersection of social networks and human behavior. At first she shunned social psychology, finding real joys in neuroscience and dissecting brains. But after a 6-year stint as a neuropsychologist, she switched to understanding how people act and behave (with each other!) in the real world. And when Digg and Facebook and Twitter entered the scene, her work dovetailed again into social interaction design. Today she's the Social Interaction Lead and UX Researcher at Bolt | Peters.

Add new comment

Comments

38
44

This seems like a great alternative to diary studies > asking users to install silverback to capture specific types of interactions. I love this idea!

A practical question: How did you arrange to have silverback installed on your participants computers? Did you have to buy three licenses and then give them a log in?

Thanks for a great article!

Chamblis

40
41

@Mary — Good questions about the logistics of the 3-friend study. Each participant installed screen recording software (Silverback) on their machines and turned it "on" in 30-90 minute blocks when they went online for a period of time. There were intentionally no clear instructions on what to capture, but we asked that they record whenever they were in "information mode": reading or finding articles, or sharing info with others. It was also not real time screen sharing, but as the videos rolled in, we reviewed them one at a time.

We also had the participants review their own videos with us and describe what was going on in each. This was extremely useful for understanding the context of interactions that didn't make sense taken as a stand-alone scene.

Hope that helps!

44
38

Thanks for an insightful article!

The study that recorded three friends' social interactions over 3 weeks was a smart way to examine that space.

I'm curious about the practical matters of conducting that study: What did you use for screen recording? Did it record 24/7 over a 3-week period? Did you review the footage regularly during the study, or sort through it all when recording was complete?

40
48

Thanks for the comments! To your question @Christopher, undoubtedly yes! Avatars help to personalize our otherwise plain text in a place like Twitter — a real person's face versus a company's logo does affect people's perceptions. But also the fact that avatars are used as an anchor to identity and reputation — frequently changing your avatar may throw people off.

33
49

That's a really interested way of performing your remove research. There's something gratifying about the fact that all research isn't performed in a "tech bubble" as you call it.

41
45

The focus on avatars when browsing twitter is interesting. I wonder how this relates to bigger brands and if brands who tweet with a voice from within the company (an owner or spokesperson) have greater success than brands who tweet in a general manner.