I grew up in a family of builders. My father is still in the construction business, and my mother is a practicing designer. We tore old walls down, built new walls, and shaped our environment. I started applying those skills after buying my first home — a 105-year-old federal style row house in Washington, D.C. that was barely standing. After 543 days, 212 tools, three trips to the doctor, thousands of feet of lumber and drywall, and miles of wire and pipe, we had a new home.

I had begun to fall in love with my tools. The power they wielded, the simple sophistication of a lathe, the force of a circular saw — what could I build next? What could I make?

Experiences don’t happen in digital, they happen because of it

Therein lies the problem. Once I had renovated the house, I no longer had a problem to solve, so I looked to my tools for the inspiration. The relentless drive I had the previous two years had dwindled, and I didn’t complete my new projects consistently. I learned my lesson the hard way: Never love the tool more than what it can build.

The combination of machine learning, data sciences, machine vision, and robotics has created a profoundly powerful set of tools that is quickly changing the world around us. We’ve called this emerging field “augmented experience” (AX). It’s the synthesis of creative vision, operational capability, and technology. It’s the key to creating the humanizing experiences that will drive global infrastructure and progress.

Source: MU/DAI

Naturally, data is at the heart of AX and understanding how and when we use it will determine the success of any design challenge. Fundamentally, data is no different than any other tool designers use. However, whereas hammers don’t change, web technologies do. Therefore, we have to evaluate when and how we use data as part of the design solution.

The Times, They Are a Changin’

Above all, we need to create intelligent systems that manage data and surface relevant information. However, a data-centric model that passes unnecessary complexity to human users will fail as a product or service.

Therefore, the role of the designer requires a critical lens toward decision-making and the appropriate use of technology to solve problems. We have to examine now, more than ever, the needs of the human user. We have to create digital threads between here and there, connecting people to each other and the experiences they want to have.

Experiences don’t happen in digital; they happen because of it. Digital is the invitation, the lens, the augmentation that enables our seemingly magical powers of seeing, hearing, and knowing more. The experience is what happens when we are catalyzed and inspired, when we connect with friends and family, and when we see something we’ve never seen before.

These concepts belong in a magical context. If the audience can see the sleight of hand, the illusion fails. However, when the hand movements are so fast and practiced that they become invisible, magic happens.

The key for designers is to create magic, not interfaces. And making digital invisible requires the right data at the right time.

The Future Is Now

A handful of companies are driving augmented experiences through data and new tools — and they are relying on designers to propel the movement forward.

Source: Tesla

Tesla’s autopilot is an example of the balance of data and design in practice. Tesla designers removed all but the absolute necessities for drivers to make critical decisions and safely operate their vehicles. Tesla isn’t sacrificing data — it’s applying it with the same careful consideration it engineers into its motors and autonomous systems.

Another example of data-driven design is Salesforce’s use of mobile apps to combat illegal fishing and marine piracy. Marc Benioff, CEO of Salesforce, leads the endeavor. His goal is simple: to protect the oceans from illegal fishing and piracy. He turned to MU/DAI to help create a powerful mobile tool, Mazu. It’s built on machine-learning capabilities to identify the silhouette of a ship through the camera of any mobile device. Once the ship is identified, it aggregates all possible data points into maps, courses, and commentary on the vessel.

Source: MU/DAI

Mazu uses every possible technology to remove human cognitive and physical load. We realized the power of the application was in a crowdsourced approach to positive identifications. Users review notifications when unidentified vessels are nearby and alert local authorities to any intruding vessels. We fundamentally understood the range of capabilities the technology affords without ever losing sight of the challenge: getting the bad guys.

Becoming the AX Designer

The introduction of AX design and data-driven experiences has quietly evolved over the past 10 years, bubbling to the surface in pops and splashes. The floodgates opened once an ecosystem of low-cost devices and powerful open-source machine learning platforms appeared.

You can feel the rush of this movement with the rapid development of sensor-driven (literally) experiences like Tesla’s autopilot and Salesforce’s Mazu. These intelligent systems are results of aggregation and new capabilities driven by designers and design-led companies.

Here are some essential ideas to understand to become part of the data-driven design movement:

• The goal includes fewer interfaces, less effort, and less time. Keep in mind that data, just like a device, powers experience. It’s the thread that translates behaviors and preferences into reactive interfaces that feel smarter and more human. But it isn’t an inherent attribute of experience; there’s no need to show it.
• Data is humanizing. It enables a relationship through hyper-customized experiences that are designed for real people, not personas. Previously, it would have been impossible to create an interface for what every person needs. These days, it’s not enough to create for what most people will need. However, it’s the role of the designer to bridge the power of the emerging technology and the needs of the user.
• It’s critical to understand technology because using data requires it. Machine learning, contextual computing, machine vision, and sensors are categories we consistently encounter in the context of data-driven design. Understanding how they capture, process, and use data will give designers the ability to quickly adopt a new set of skills and a new language for communicating with deep-tech partners.

With so much powerful technology and data at our fingertips, this is an exciting time to be a designer; we’re standing at the very intersection of them. But with our power and excitement comes responsibility to do right by the users who trust us. Keep them in mind, and don’t let tools take over.

Image of set of old tools courtesy of Shutterstock.