User experience design and human–computer interaction (HCI) emerged in a world of desktop computers. But our experience of computing has changed radically in the past 10–15 years. Many of our interactions now take place on mobile phones, tablets, e-readers, and smart TVs. And it’s common to use one service across multiple devices with different form factors.
BBC iPlayer can be used on connected TVs, smartphones, tablets, PCs, game consoles, and set-top boxes (image: BBC)
We’re still figuring out the best ways to design for new devices and experiences. Interactions can happen in a wide variety of contexts, especially for mobile devices. They can happen on a variety of scales, from tiny wrist-tops, to smartphones, to TV user interfaces (UIs) viewed from 10 feet away. Even academic researchers in HCI have published relatively few papers on cross-platform design.
The “Internet of Things” (IoT) refers to the growing range of everyday objects acquiring connectivity, sensing abilities, and increased computing power. In consumer terms, some common categories currently include:
- Connected home technology (such as thermostats, lighting, and energy monitoring)
- Wearables (such as activity/fitness trackers and “smart” watches)
- Medical/wellness devices (such as bathroom scales and blood pres- sure monitors)
- Connected cars (which may provide access to smartphone apps via dashboard controls, engine diagnostics, and automatic alerting of authorities in case of a crash)
- Urban systems (such as air quality sensors, city rental bikes, and parking meters/sensors)
Designing for the Internet of Things (IoT) raises all the challenges of cross-platform design, and more. An obvious difference is the much wider variety of device form factors, many without screens. Less obvious differences include the effects of many IoT devices being only intermittently connected. And even a simple task, like unlocking a door, can quickly become complex when it forms part of a system spanning many interconnected devices, services, and users.
IoT is still a technically driven field. At the time of writing, the UX of many IoT products is some way off the level expected of mature consumer products. For example, the UK government commissioned a study on the usability of connected heating systems in late 2013. They found that none of the five major connected heating devices on the market in the UK offered a good UX.
The Lockitron connected door lock is one of a huge number of connected devices with no screen (image: Lockitron)
Functionality Can Be Distributed Across Multiple Devices With Different Capabilities
IoT devices come in a wide variety of form factors with varying input and output capabilities. Some may have screens, such as heating controllers or washing machines. Some may have other ways of communicating with us, such as flashing LEDs or sounds.
The Honeywell evohome connected radiator valve has a basic LCD screen (image: Honeywell Environmental Controls)
Some may have no input or output capabilities at all and are unable to tell us directly what they are doing. Interactions may be handled by web or smartphone apps. Despite the differences in form factors, users need to feel as if they are using a coherent service rather than a set of disjointed UIs. It’s important to consider not just the usability of individual UIs but interusability: distributed user experience across multiple devices.
The GlowCaps connected pill bottle lid uses light and sound notifications to remind the user to take medication (image: GlowCaps)
The Nest Learning Thermostat can be controlled by the on-device UI, a smartphone app, or a web app (image: Nest)
The Focus Of The User Experience May Be In The Service
When we talk about IoT, we tend to focus on the devices, particularly those with striking or novel forms. But the behavior of the device might be generated by a program that lives on another device on the network (i.e., a server). We call this the Internet (or “cloud”) service.
This means that the service around a connected device is often just as critical in delivering the user experience, if not more so, than the device itself. For example, the smart travelcards such as the London Oyster and Hong Kong Octopus are often thought of as the focus of the payment service. But the services can be used without a card at all via an NFC-enabled smartphone or bank card. The card is just an “avatar” for the service (to borrow a phrase from the UX expert Mike Kuniavsky).
Hong Kong’s Octopus payment service can be used with an NFC phone as well as a smart card (Octopus reader on KMB bus by Ka890, CC licence via wikimedia Commons; Octopus app: Octopus Cards Ltd.)
We Don’t Expect Internet-Like Glitches From The Real World
It’s frustrating when a web page is slow to download or a Skype call fails. But we accept that these irritations are just part of using the Internet. By contrast, real-world objects respond to us immediately and reliably.
The shift from desktop to mobile computing means we use computers in a wide variety of situations
When we interact with a physical device over the Internet, that interaction is subject to the same latency and reliability issues as any other Internet communication. So there’s the potential for delays in response and for our requests and commands to go missing altogether. This could make the real world start to feel very broken. Imagine if you turned your lights on and they took two minutes to respond, or failed to come on at all.
In theory, there could be other unexpected consequences of things adopting Internet-like behaviors. In the Warren Ellis story “The Lich House” a woman is unable to shoot an intruder in her home: her gun cannot contact the Internet for the authentication that would allow her to fire it. This might seem far-fetched, but we already have objects that require authentication, such as Zipcars.
When you book a Zipcar online, the service sends details of the reservation to the car; swiping a smart card authenticates you as the person who made the booking (image: Zipcar)
Iot Is Largely Asynchronous
When we design for desktops, mobiles, and tablets, we tend to assume that they will have constant connectivity. Well-designed mobile apps handle network outages gracefully, but tend to treat them as exceptions to normal functioning. We assume that the flow of interactions will be reasonably smooth, even across devices. If we make a change on one device (such as deleting an email), it will quickly propagate across any other devices we use with the same service.
That will not always happen in IoT systems. Many connected devices run on batteries, and need to conserve electricity. Maintaining network connections uses a lot of power, so they only connect intermittently. This means that parts of the system can be out of sync with one another, creating discontinuities in the user experience. For example, if your heating is set to 19° C, and you use the heating app on your phone to turn it up to 21° C, it will take a couple of minutes for your battery-powered heating controller to go online to check for new instructions. During this time, the phone says 21° C, and the controller says 19° C.
Schematic of heating system with app and controller giving different status information
These discontinuities won’t always be noticed: sometimes the delays will be very short, and sometimes users won’t be around to notice them—for example, when they are turning on a remote device. So the UX may feel synchronous, even if the service technically isn’t. But when we do notice, the UX may feel quite disjointed.
Code Can Run In Many More Places
The configuration of devices and code that makes a system work is called the system model. In an ideal world, users should not have to care about this. We don’t need to understand the technical architecture of a conventional Internet service, like Amazon, in order to use it successfully. Instead, we form a conceptual model of what Amazon does and how it works that’s good enough to help us understand what to do. We know we can search or browse products, that we need to add them to a basket, set up or log into a user account, pay, and then we’ll get a delivery. We don’t need to understand the different machines involved in making this system function. But as a consumer of an IoT service right now, you can’t always get away from some of this technical detail.
A typical IoT service is composed of:
- One or more embedded devices (the things in IoT)
- An Internet service
- Perhaps a gateway device (a separate device needed to connect some embedded devices to the Internet)
- One or more mobile or web apps for the user to interact with the service via a mobile, tablet, or PC
Compared to a conventional web service, there are more places where code can run. There are more parts of the system that can, at any point, be offline. Depending on what code is running on which device, some functionality may at any point be unavailable.
For example, imagine you have a connected lighting system in your home. It has controllable bulbs or fittings, perhaps a gateway that these connect to, an Internet service, and a smartphone app to control them all. You have an automated rule set up to turn on some of your lights at dusk if there’s no one home. If your home Internet connection goes down, does that rule still work? If the rule runs on the Internet service or your smartphone, it won’t. If it runs on the gateway, it will. As a user, you want to know whether your security lights are running or not. You need to understand a little about the system model to understand which devices are responsible for which functionality, and how the system may fail.
It would be nice if we could guarantee no devices would ever lose connectivity, but that’s not realistic. And IoT is not yet a mature set of technologies in the way that ecommerce is, so failures are likely to be more frequent. System designers have to ensure that important functions (such as home security alarms) continue to work as well as possible when parts go offline and make it as easy as possible for users to understand what’s happening, and recover from any problems.
The Philips Hue system consists of connected bulbs, a gateway, an Internet service, and a smartphone app (image: Philips)
Devices Are Distributed In The Real World
The shift from desktop to mobile computing means that we now use computers in a wide variety of situations. Hence, mobile design requires a far greater emphasis on understanding the user’s needs in a particular context of use. IoT pushes this even further: computing power and networking is embedded in more and more of the objects and environments around us. For example, a connected security system can track not just whether the home is occupied, but who is inside, and potentially video-record them. Hence, the social and physical contexts in which connected devices and services can be used are even more complex and varied.
To get your copy of Designing Connected Products click here and to download a free ebook which has more Design IoT related titles click here. You can also join the authors at Solid Conference in San Francisco June 23rd-25th.
Image of Phillips Hue system courtesy Phillips.
