This is a continuation of a two part article on mobile testing tools. In Mobile Testing Toolbox: Part 1, I talked about tools that can be used for moderated remote usability testing and for recording in-person mobile usability studies.
In part two of this article I will share my knowledge of unmoderated mobile tools. Unmoderated tools are good for collecting behavioral data about what people are doing with a mobile website or an app.
If a researcher wants to know why people are doing certain things on a mobile site or app they should use the moderated tools discussed in my previous article.
Please note: this is not a complete list of all mobile remote usability testing tools. It is just a list of tools that I have reviewed. These tools are constantly being updated with additional features and functionalities, so please visit the tool’s website for the most up-to-date information.
Unmoderated tools are good for collecting data about what people are doing with a mobile site or app
An Overview Of Features By Tool
What Tools Can I Use For Unmoderated Remote Usability Testing?
Unmoderated remote usability testing tools provide users with a link to a website evaluation. The website evaluation includes set of tasks that the user can complete at any time. The task instructions serve as the usability test moderator. A survey can be shown after the tasks to gain additional information. Userzoom and userlytics are two products on the market that can provide unmoderated remote usability testing.
UserZoom
UserZoom is an enterprise level software tool that enables you to conduct unmoderated remote usability testing and integrate with a suite of other mobile tools, such as mobile Voice of Customer (VoC) and real-time recruiting from your mobile sites. Mobile VoC is a feedback feature that allows users to comment on various portions of the mobile website. The comments are connected to a page or feature and can be sorted and analyzed. Real-time recruiting allows the researcher to intercept a user who is browsing on the site and invite them to participate in a survey or an online study. UserZoom is a quantitative tool that records analytical data such as clickstreams, time-on-task, and completion rates, offering a video session replay. (The video session is a screen capture with no picture-in-picture or audio.) The user accesses the study by downloading an app and using the app to access a mobile website.
Works on iOS and Android; mobile website.
Features/Functionality
- Unmoderated usability testing of mobile websites
- Email invitation to download app, allowing for screening of certain types of users
- Clickstreams (no tagging), time-on-task, and net promoter scores
- Video session replay and the ability to create video clips
- Mobile surveys and voice of client
Limitations
- Mobile websites only; no app capabilities
- User must install UserZoom app
- No picture-in-picture or audio in video
- Videos stored on UserZoom servers
Userlytics
Userlytics is a more qualitative tool that allows for unmoderated remote usability testing. The tool provides video files to analyze and annotate. This tool does not provide any analytical data such as clickstreams and time-on-task, it only provides the raw video file of the session. Userlytics requires the user to use their mobile device as well as a laptop and webcams—one pointed at the mobile device and a second to record the picture-in-picture of the user’s face. Again, similar to UserTesting, this setup can get complicated, therefore Userlytics offers the use of a panel of users who have their systems configured to be able to access any remote mobile studies. If using a panel for a study is a viable option for your business, the tool can be a good fit. If not, recruiting your own users is possible, but it requires you to send your users a webcam they will have to configure to their systems in a specific way.
Works on iPhone or iPad; mobile app or websites.
Features/Functionality
- Unmoderated usability test of an iPhone or iPad app or website
- Utilizes a panel
- Record the iPad/iPhone screen, picture-in-picture, and user’s audio
- Download the videos for advanced editing and analysis
- Ability to annotate videos and request transcripts of the recording
Limitations
- iPhone and iPad only
- If you test with your own users, they must have laptop, iOS device, and webcam
- Apps only, no mobile website capabilities
- In order to ask follow up questions after the tasks, you must either create a zip file with a presentation or ask the questions after the recording part of the videos
- The videos are stored on the Userlytics servers
What Tools can I use for Analytical Data?
Unlike research tools where you have to recruit participants, analytical data tools run in the background for everyone who accesses your site or app. All of these tools are for apps only and additional code must be built into your app’s SDK.
Appsee
Appsee (formerly UserVOD) is an analytic tool that provides both quantitative data as well as qualitative video files of the user’s interaction with your app. Reports include touch analysis heat maps, unresponsive gestures, and crash records. The unresponsive gestures report can help you understand what parts of the screen users are tapping, swiping, or pinching that are not currently active. This can help identify ways to improve usability by potentially enabling gestural interactions that users expect. In addition to its usability benefits, the app has some quality assurance features because it includes crash records. Video files are stored on Appsee’s servers, but they have the ability to set private screens that will not be recorded to ensure user privacy.
Works on iPhone or iPad; mobile app.
Features/Functionality
- Unmoderated click stream recording; records application screens; and captures all gestures
- Touch analysis heat maps, crash records, and unresponsive gestures are recorded
- Ability to set private screens that will not be recorded
- In order to preserve data useage the app has an option to select if the sessions should only be recorded and uploaded through Wi-Fi
Limitations
- iPhone or iPad only; Appsee is developing an Android solution they hope to have ready soon
- Apps only, no mobile website capabilities
- Appsee code must be built into the app SDK
- The videos are stored on Appsee servers
Delight
Delight is an analytics tool that provides qualitative video files of the user’s interaction with your app. Delight also has the capability to capture picture-in-picture and audio by having the user consent to a recording of their session. Delight does not provide quantitative reports of the data, but it provides a dashboard to watch recordings of specific user flows that can enable you to gain insights from actual user sessions. Video files are stored on Delights servers, but Delight has the ability to set private screens that will not be recorded to ensure user privacy.
Works on iPhone or iPad; mobile app
Features/Functionality
- Unmoderated click stream recording, records application screens, and captures all gestures
- Ability to capture picture-in-picture and audio
- Ability to set private screens that will not be recorded
- An option to select if the sessions should only be recorded and uploaded through Wi-Fi or not
- Ability to track events
Limitations
- iPhone or iPad only, Delight is developing an Android solution they hope to have ready soon.
- Apps only, no mobile website capabilities
- Delight code must be built into the app SDK
- The videos are stored on Delight.io servers
Swrve
Swrve is an A/B testing tool that measures the differences between two variations of an app. The tool is tailored to marketing or return on investment (ROI) tasks such as revenue, conversion, user retention, and engagement, but can also be used to determine the best designs for improving traffic or usability as well. Swrve has the ability to target specific users for A/B tests by using their interface to send the versions to specific users.
Works on iPhone or iPad; mobile app
Features/Functionality
- Mobile app A/B testing tool
- Full key performance indicator (KPI) reporting (revenue, conversion, user retention, engagement, etc)
- Full direct marketing platform delivers in-app messages to target customer groups with relevant offers and content
Limitations
- iPhone or iPad only
- Apps only; no mobile website capabilities
- Swrve code must be built into the app SDK
Arise
Similar to Swerve, Arise is an A/B testing tool that measures the differences between two variations of an app. For example, it provides data that makes it possible to decide between button color, font size, and text labels. It can show you which version of the app has the best results based on pre-determined goals such as return on investment, new account creation, or traffic to certain pages. Arise is still in the beta version of its product, although they claim to be production ready.
Works on iOS or Andriod; mobile app
Features/Functionality
- Mobile app A/B testing tool
- Simple, clear graphs allow you to instantly see which version of your app is driving results
- Can export test data to other software programs for viewing
- iOS and Android
Limitations
- Apps only; no mobile website capabilities
- Arise code must be built into the app SDK.
I hope this article was educational and helpful in distributing the knowledge of what tools are available to assist with mobile usability testing. I expect it at least serves as a good overview and starting point for researchers looking for tools in the mobile space.
Image of person using smartphone courtesy Shutterstock
