Need faster recruiting for user research and testing? Try building your own UX research participant community. This is becoming easier to accomplish and more important due to the need for increased agility in development. At the most basic level, a panel is a pre-recruited list of people who have opted in to participate in your research. The panel can take different forms, depending on your resources for funding and day-to-day management, and for the needs of your overall research program.
For companies that can afford professional-level panel management platforms, these participant communities enable some interesting new kinds of interactions with customers. A panel can eliminate as much as two weeks from the traditional recruiting process for a round of research.
Your panel can be a community of thousands with advanced features for member friending and co-design, something as simple as a spreadsheet, or something in between. Members can be customers who regularly use your products and services, non-customers (people who aren’t aware of the brand that sponsors the panel they joined) who want the type of solutions offered by your products and services, or both. In any case, a distinguishing feature of a panel is that the members are pre-registered and expect you to call on them. Marketing groups have been doing this sort of thing for years, and UX groups can benefit from their experience while adapting panels to our needs.
Let’s review some of the pros and cons of panels and look at their make-up and capabilities.
Benefits and Drawbacks of Having Your Own Panel
Pros:
- Speed: Panel members tend to be more engaged than your average customer, and therefore respond to your requests for participation quickly. If you have panel software (from providers such as Toluna or Vision Critical) with built-in survey features, you can filter, recruit, deploy, and show results in the system within a few days. If you are working with an internal or external recruiter to do manual screening, your panel data can give them a head start. For simple recruiting needs, panel profile data may eliminate the need for manual screening entirely.
- Cost: There is a basic cost (as little as $0) and level of effort to setting up and maintaining a panel, but every time you use this in place of a recruiting agency you will save on fees. Some panels can attract members with minimal or no payment (incentives for facilitated research still apply).
- Loyalty: Some people want to be associated with your brand and appreciate having an inside voice.
- Ad hoc opportunities to get close to your users: Panels can give you spontaneous or unplanned chances to interact with members. For example, I have reached out to individual members of my panel when I’ve noticed an interesting response to a survey that I wanted to learn more about, and this led to some rich conversations.
- Deep profiles: Some panel platforms enable profiles to automatically build over time as individual members respond to research, giving you increasingly rich contextual information surrounding future responses. This can be used for longitudinal research.
- Company buy-in: Faster, simpler ways to get close to customers may, in some companies, be the key to getting acceptance for integrating a customer voice into the development process.
Cons:
- Bias: A panel is just one slice of your audience, and not the whole picture. People who register for your panel tend to be engaged with your brand, and may give you a more positive reading than those who don’t. You should make sure not to oversample a narrow subset of people, and continually recruit new members. You can track the history of individual responses manually, if you use a spreadsheet, or through query filters in a panel system.
- Limited use: You may want people in a study who use a certain part of your product or service, and this particular usage may not be part of your profile information for members. In order to find these people you can integrate some panel platforms with your behavioral metrics system (such as Omniture) or else use an intercept tool such as ethnio to present invitations at the point of use.
Panel Capabilities
There are different types of panels for different uses. The second and third types in the list below require a professional panel platform.
- Branded panels: The members know the panel is for your company. These panels are branded, designed, and named. For example, The Economist Digital reader’s group or the NASCAR Fan Council. Members want to contribute to improving products and services for the brand, and like being associated with the brand.
- Unbranded panels for your competitive niche: These members care about the problems you are trying to solve and use your competitor’s products or services.
- Unbranded generalist panels: Members are queried or tested on a variety of topics for marketing or usability testing.
Members in a branded panel may not want or require financial compensation to join in some types of research. Members in the two unbranded panels typically do get paid, which can be expensive for surveys. Testers who expect to be paid may give opinions that they think will get them asked back, rather than their genuine responses. Some panels are managed for continuous or longitudinal research, and others are short-term, where a company conducts a brief, intense period of research and then disbands the panel.
All panel systems should offer most or all of the features below:
- Member registration and profiles (and the ability to filter and invite based on profile data)
- Simple, reliable means for opting out
- Branded emails and other types of messaging
- Member website including a CMS for posting member-only content
- Embedded research tools, such as surveys
- Tracking of member responses over time, for gradual enrichment of profiles and longitudinal data
- Social-media interactions, with or without facilitation
- Multimedia uploading from members in both quantitative and qualitative research
- Co-creation and beta-testing tools (Vision Critical has observed that panel groups tend to have a subset of creative people who can articulate their ideas clearly, and has a methodology for identifying and recruiting these people into co-creation activities)
Panel providers may also offer personalized services to help recruit members, filter and recruit for studies, or even conduct studies. There is a trend toward networking panel data with data from other systems, such as CRM or metrics systems, that were previously siloed. This can allow you, for example, to see what someone said they did on your website, compared with what they actually did (the Big Data effect).
What to Consider
Your objectives: Will you target paying customers or a broader group of people? Are you focused on improved usability, new product development, or both? What groups in your organization want the data – is this mainly UX and product development, or does it include marketing, sales, or editorial?
If you want a representative cross-section of your users you should recruit from every touchpoint
Filling your panel: If you want a representative cross-section of your users you should recruit from every touchpoint possible. This includes your website, mobile site and apps, other sites that your users visit, your social media channels, your printed publications, your physical sites or events, your help pages, and your customer service interactions. If you use intercepts outside of the panel to invite people to a study, they can be invited to join the panel once they have completed the study.
Things to consider as part of the sign-up experience:
- Participant profile data: What core demographics or other information from members is relevant for a broad range of future studies?
- Instituting clear rules: Who is welcome? How often will members receive invitations to join a study? Are there requirements for how often they must accept invitations? What are social, financial, or other rewards for joining the panel or joining an individual study? How does privacy work? How do you opt out?
- Participation options: Are there options for joining in different types of research? When you sign up for the FT.com panel you area shown a list of different research types and can select the ones you want to get invites to (eg, surveys, field visits, beta tests).
- Your values: What aspirations will you convey about the role of members in contributing to product and service improvements? How comfortable are you about sharing research results with members?
- Research invitations: How will you brand and design your invitation template? Does your legal department require any particular language for describing incentives, such as random prize drawings? Invitations should include a way to opt out of the panel.
- Rewards for joining the panel: If your brand is well-liked you may get enough members without paying people to join, but be clear during sign-up about how you may or may not reward participation in different types of research studies (e.g. surveys or usability testing). The Official NASCAR Fan Council does not pay, there is a waiting list to join, and response rates to studies are high.
Keeping the group engaged and fresh:
- Share back: People want to know how they made a difference. What did you learn that you can share? Can you tell them anything about upcoming changes? Internal groups may be cautious about sharing some information, but it’s important to show members that they are special and are getting information that non-members are not privy to.
- Make the community visible: Being a community member is more interesting than being on a list. Can you share demographic information that describes the community? FT.com offers a panel member website where you can log in, read articles and summaries of completed research, and see names of individuals who won a prize drawing.
- Apply feedback: Enable members to report issues and know they are getting taken care of. You will help panel morale if you periodically survey members to ask how the group can be improved and then act on their suggestions.
- Periodic recommitment: In time, member profile data reported at sign-up time may get outdated; you can periodically ask members to confirm/update their profile information. This helps keep the data accurate and may identify people who are no longer motivated to remain active.
- Continual recruitment: Your panel recruitment channels should remain open and active, in order to replenish people who leave and to include new voices.
Starting Small, on a Budget
If you aren’t ready to sign up for a professional panel platform, you can start simply by inviting people to sign up with a Google form. These forms allow some limited branding, and they place form answers (profile data) into a spreadsheet, which you can filter for specific studies. Add a page to your Help pages explaining how your panel works—at a minimum people need to know how to opt out, so make that information clear at the bottom of every email to members.
If you use email for invitations, be careful to write subject headers that minimize your chances of landing in spam filters. MailChimp has pointed out that 10-20% of emails suffer this fate. Never, ever expose member email addresses to other members in email invites.
Even these simple steps can help ramp up your recruiting process. As your list grows in size, it may eventually seem too unwieldy to filter, or not secure enough. But by then, your panel approach will hopefully have shown its worth and make it easier for you to get support for further improvements.
Image of paper box doll courtesy Shutterstock.
Acknowledgements:
- Monica Todd, Senior Customer Research Manager, FT.com
- Andrew Reid, President, Founder and Chief Product Officer, Vision Critical
- Peter Shafer, Senior Vice President, Toluna
- Building and Managing Your Own Online Survey Panel by Vivek Bhaskaran
