In the last 1 year, companies from different industries are forced to rethink ways of working due to the social distancing measure to prevent the spread of COVID-19. This is not an exception for those working in the UX research field. In the current pandemic days, gone are the days when you can gather strangers in a closed room to have 1–2 hours of focus group discussion together. Say goodbye to arranging a visit to the respondent’s house and have a conversation in their living room. Interaction with strangers seems very unnerving and even considered taboo these days. Zoom call is the new normal, it’s the new working reality for everyone.
“Most people are familiar with WhatsApp call or google meet these days, it should be easy to conduct online interview session!”
That was my initial thought when my office Somia Customer Experience decided all team members should resume working from home until further notice.
But after conducting dozens of remote interviews in the last 12 months, I change my mind: “Remote interviews are harder to conduct.”
Yes, it’s painstakingly hard and let me tell you 6 reasons why based on my experiences and quick tips I’ve picked up from handling different situations:
Reason #1: Respondent recruitment poses more challenge during the pandemic
Before the pandemic, if you need to find small business owners to talk about their challenges when running the day-to-day business, the team can simply visit different stores and intercept to scout for potential respondents to be interviewed. But nowadays, people get agitated when they are approached by stranger due to the spread of COVID-19. Thus, recruiting interview respondents requires more precaution and effort.
Quick tips: Set a longer duration to find respondents. Be flexible with your recruitment criteria and be creative in ways of recruiting. When intercept method isn’t possible, reach out to various online network, e.g go to relevant online communities and spread a google survey with screener question to recruit respondents.
Reason #2: There are higher chances of people not showing up to the online session
I don’t know about you, but I’ve experienced online ghosting to another level in the last 12 months. Obviously not from unreliable guys from dating apps, but from respondents who failed to show up to the interview session without any explanation. A ‘no show’ in a face-to-face session is indeed common, but I’ve had more ‘no show’ in a remote interview setup.
I guess it’s probably easier to disappear into thin air and bluetick someone on WhatsApp, especially if you never meet in real life and only communicated online for a few days. I notice this was more likely to happen when I was in a project where the team must conduct respondent recruitment independently and didn’t have a recruitment team who helped to maintain communication with the potential respondents.
Quick tips:
- Always recruit a backup respondent to make sure you have enough respondents to gather insights and ensure the research timeline is still on track. In our case, we always aim to recruit 20–30% more respondents. For example: if you need to interview 10 respondents, always recruit 12–13 respondents.
- Build relationships with respondents by maintaining communication before the actual interview. It can be via WhatsApp by simply reminding them of the session time, or asking about their days a few days before the session to make sure that they will attend the session or should there be any rescheduling needed.
Reason #3: Remote interview demands more focus
Unfortunately, there are times when respondents take remote interview sessions less seriously. This is shown when the respondent is multitasking while doing the interview. I once had a respondent who insisted on doing the 1-hour Zoom interview while he was driving a van in the middle of nowhere in South Kalimantan with his entire family.
Not only was the respondent not focused and only gave short answers, but the session quickly turned into focus group discussion as other family members in the van also added irrelevant responses because our conversation was put on loudspeaker. It was an amusing scene to witness and I was worried about the respondent’s safety at the same time. You won’t encounter this kind of scene when conducting face-to-face interviews.
Quick tips:
- Brief your respondent as detailed as possible about the interview set up before the actual session. This includes what devices are needed, who will moderate and how many observers will be in the session. Make it clear they need to allocate an uninterrupted time for the session. This will set the tone that the interview session is important and their response is crucial to the research.
- During the interview, create an environment to ensure respondents engagement during the session. Ask the respondent to sit in a well-lit room (avoid backlight!), turn on the camera and put on earphones. This also applies to the moderator. In a remote setting, the moderator needs to establish his/her presence as strong as possible to earn the respondent’s full attention.
Reason #4: Internet connection is not always on your side
This is the biggest hurdle of all when conducting online sessions, especially when you are based in a developing country like Indonesia. Despite paying for fibre broadband connection and living in the nation’s capital Jakarta, we don’t always have the smoothest internet connection and we can’t predict when the bad connection will hit us. It’s even more tricky when it is the rainy season.
It is even more challenging if you’re planning to interview respondents who live in rural villages where internet connection is very limited. For instance, I once had to interview a 65 years old woman who lives in a small village in Central Java. I thought I had everything prepared for the interview session: we arranged a local fixer to visit the respondent’s house, asked him to set up a laptop with a webcam, pocket wifi and a spare smartphone with a personal hotspot as a backup connection.
But things went south very quickly on the day of the interview: the pocket wifi did not work, the spare smartphone had no signal in the area even though we had purchased a sim card with the best network coverage in Indonesia. In the end, we had to change our plan. We ditched the Zoom interview and called via landline to the respondent’s Nokia feature phone to conduct the interview. It worked out even better than expected! The respondent was more comfortable chatting as she is more familiar with the phone call format.
Quick tips: When the internet fails,adopt other ways to communicate with the respondent. Most times going back to basic is still a way to go. Always be flexible because there is no such thing as perfect research planning.
Reason #5: Remote session is more exhausting by nature because you need to over-communicate to prevent misunderstanding
“Can you hear my voice clearly?”
[When showing concept testing] “Can you see the image on the screen?”
“Do you need me to repeat the question?”
[respondent image freezes] “Umm…. Hello?”
I bet you have asked these questions at least once when running an online interview. Sometimes it might be the crappy internet connection that breaks the respondent or moderator’s voice, or perhaps the respondent is too shy to ask clarifying questions. We don’t have to ask these questions when we meet respondents face-to-face. As a moderator, I also realise that I need to be more specific when throwing my questions during the remote interview session.
Quick tips: Never assume, always confirm. It’s better to over-communicate your statement/question and ask whether the respondent is on the same page to avoid misunderstanding.
Reason #6: It’s harder to gain context
The biggest disadvantage of a remote interview is that you can’t easily capture nuances or how respondents interact in context. Most times we have no choice but to rely on what respondents’ say and do our best as a moderator to dig deeper by clarifying and paraphrasing our questions to make sure we get the right information from the respondent. But there are tendencies when respondents forget certain information or subconsciously want to appear a certain way and don’t reveal true behaviour.
One of my previous projects was to understand people’s relationship with food and how people prepare daily meals at home. One of the respondents, a mom of 2 kids, got really excited and told me that she had adopted a new routine to do meal prep and created different categorisations for storing food in the fridge. As she was telling me in her own words, it was hard for me to imagine without the visual cue.
Since the respondent was at her home, I finally asked her to show me her fridge and do ‘live vlogging’ style to confirm whether my understanding was correct. While she was standing and showing items on her fridge, she began to tell her story more vividly and it all made more sense to me. Through observing items on her fridge, I even got additional data about respondent’s frozen food restocking behaviour and how she occasionally makes shopping list and stick them on the fridge. Those were new information she didn’t mention when we had the conversation earlier.
Quick tips: Be creative in different ways to capture additional data beyond the interview session because you might get valuable insight from observing. There are different ways you can do this:
- Prepare a shoot list of things that respondents can capture or screenshot
- Ask respondents to bring items they can show on cam during the interview (e.g if you want to know respondent’s skincare routine, ask them to bring the 3 most used skincare products during an interview).
When respondent interacts with artefact during the interview, you can uncover stories and additional data that might be valuable for the research. I also notice that when respondents move around the house or interact directly with objects, it gets the respondent more excited to tell their story.
I don’t hate running remote interview session, but I’m simply sharing the challenges I’ve faced when running these sessions (and as the title of this article suggests, one year of Zoom clearly drains my mental capacity). I’ve unlocked new skills from conducting remote interviews that I might not be able to develop when running face-to-face user interview. As a user researcher, I’m challenged to be more collaborative, adaptive and meticulous in the research planning to make sure that the remote session runs smoothly.
I’ve also had a fair share of great experiences from running remote interviews, and I believe that there are still few positive advantages from conducting online interview sessions. You can cut down travelling time, more clients can observe the session and you can actually squeeze in more interview sessions in a day.
At the end of the day, I still long for the days when I finally can go back to the field without worry, observe the way people live and establish human connections from doing face-to-face user interviews with the respondent. This seems like a lifetime away, for now. But until then, stay safe and always keep adapting, fellow UX researchers. We got this!
