Designing Big Data for Humans (with a Little Help from Hollywood)

We humans are inherently emotional, imperfect creatures. As a designer, I’ve made the mistake of assuming that users of whatever product I’m designing will follow the task flows, organizational structures, and nomenclature that I’ve imposed on them perfectly and without hesitation. I’d been forcing users to adapt to my way of thinking.

I’m not the only one guilty of this behavior. Numerous applications and digital experiences put this burden of attention and effort on the user. As the user, you end up doing much of the work.

You end up having to:

• Puzzle through that long list of functions in the drop down menu to figure out which one applies to you
• Swipe through 10 screens of content to find the app you really want
• Do numerous searches and phone calls to find another way to get home when the airline cancels your flight without any alternatives

In all these cases, while technology and intelligent algorithms are implicitly present, they still aren’t helping the user.

What Hollywood can Teach Us

Even in futuristic Hollywood the user experience is often flashy, but not very usable. Minority Report is often cited for its introduction of a lush, gesture-based interface. As the film’s protagonist, Tom Cruise lives in a world where all the machines know everything about him. He gets targeted ads served up as he is simply walking down the street. He is able to control and stream masses of data with gestures made wearing some seemingly magic gloves—presented as points for controlling the overwhelming amounts of information flying through the air in front of him.

While much of the technology featured in Minority Report—from retina scanners to crime prediction software—was prescient and is only now starting to be realized, the user interface itself leaves much to be desired. It looks good, but the burden of attention and effort is on the user to learn a new type of multi-touch language to control the computer. The user must learn custom gestures and finger movements! The user must wear a specialized new input device in the form of three-fingered gloves!

Data is as plentiful as Minority Report—I aspire to an interface as approachable as the OS in Her

Contrast that to Spike Jonze’s movie Her. The interface between human and machine is so seamless and effortless that the character played by Joaquin Phoenix can fall in love with his operating system. His OS is aware of the context of his calendar, contacts, and—through his smartphone’s camera—every experience he sees. With that context, every suggestion becomes more relevant as the OS learns more about him over time. It’s a futuristic world with intelligent machines and predictive algorithms, but the technology does not overwhelm the human. Instead machines compliment humans.

The lesson I’m driving at here is why not let the computational power of a machine that understands context start to remove some of the burden on the user? Let the human focus on what a human does best.

Designing for Humans

My work in the real world focuses on designing enterprise applications for analyzing big data. In many ways, the data available to us in the enterprise space today is just as rich and plentiful as the data in Minority Report, but I aspire to the vision of creating an interface as approachable as the OS in Her. We’re trying to harness the computational power of algorithms to reduce the burden of attention and thought on the user. If big data is going to see its potential in the enterprise, we as builders of technology need to ensure that business users can eventually get to a spot where they can focus on asking the right questions and analysis—not manually manipulating the data so that it can be analyzed. Humans have a tremendous capacity to detect patterns and insights in data and can be aided by machine intelligence.

To meet this goal there are a few key design principles that we’ve focused on here at Trifacta, but that I believe apply readily to any design team in the data space:

Visually Simple and Functionally Strong

Analysts should be able to start to exploring and transforming data immediately. Interacting with real data values is much more natural than abstracted metadata in the form of column and row names. When data is too large to scan through in its native form, simple visual summaries such as histograms can assist by providing visibility of the forest as well as the trees.

Elegance at Scale

Today’s data sets span the kilobyte to the petabyte. This means allowing analysts to perform data transformation on the most detailed data values while also magnifying their capabilities to work at scale. And all of this activity needs to happen at near real-time so that the user can get an immediate understanding of the potential output of the transformations that they are implementing.

User Empowerment

Each interaction should enhance the experience with data and make the user feel smarter. Because users don’t necessarily know in advance exactly what they’re looking for, design for exploration—let users iterate through the process without a penalty for failure or hypothesis testing.

Conclusion

Following these design principles, we can focus on making sure that our algorithms and computational power are leveraged in service of human ease and workability. We can keep the machines doing what they do best—chugging through many different computational algorithms to offer up the best ones to us—and keep humans doing what they do best: recognizing patterns in data and making choices based on the full context and awareness that comes with being a human.