Product Sonification

We spoke with Karel Barnoski, an award-winning audio designer who has integrated sound into products and experiences ranging from stoves to cameras. Karel shares his keen insights about how the language of sound relates to interaction design and UX, and about the power of audio to connect users to brands and products.

We spoke with Karel Barnoski, an award-winning audio designer who has integrated sound into products and experiences ranging from stoves to cameras. Karel shares his keen insights about how the language of sound relates to interaction design and UX, and about the power of audio to connect users to brands and products.

Q: Karel, your company, 2octave, is very unique. Can you give our readers an overview of your work and how it relates to user experience and interaction design?

A: 2octave specializes in product sonification, audio branding, and audio strategy. We design audio as an integral part of the user experience. Our work is tied to how people interact with the world around them, whether it’s an environment, an interface, or a product like an appliance or camera.

Sound is a critical aspect of the user experience that’s been sorely overlooked. Strategic use of sound can improve a product’s usability and enhance the impact of a brand. Sound appeals to our emotions, and can be leveraged to connect users to a brand even when they aren’t looking at a product.

Q: Can you tell us about your background and how you got involved with this field?

A: The seed was planted when I took my first piano lesson at the age of 8. I’m 31 now, so that’s about 23 years of playing piano—classical, jazz, rock, blues, etc.

In college, I majored in visual design and minored in music theory. I combined my visual and musical ideas for many of my class assignments, which is when my two interests became one. After college, I started working at Kodak as a visual interaction designer, but there were many initiatives at Kodak that gave me the opportunity to design custom music, sound effects, branded pieces for kiosks, interactive Web applications, and product prototypes.

Q: From your perspective and experience, how much can sound impact a user’s perception of a product, software application, or service?

A: Without getting too deep into human perceptual psychology, sound is perceived much in the same way as other stimuli. By manipulating sound you can effect the user’s perception of an experience. Through sound, you can make a product be perceived as elegant, simple, complex, etc. Therefore, it’s essential to design the audio experience parallel with the other aspects of the product, otherwise there might be a disconnect between the auditory and visual experience for the user.

If a product sounds elegant, users are going to perceive that product as elegant. If it sounds cheap and crude, like the sound was thrown in at the last minute, that’s the way users are going to perceive the rest of the product.

Q: How and when do you get involved in the client relationship and how do you get customers to understand the business value of sound?

A: Establishing a relationship with a project’s stakeholders generally starts with marketing and human factors. There are proven methodologies for how a sound is designed that leads to better usability of a product, and the same is true for return on investment. Most marketing folks I am engaged with understand the value of a well-designed product. Therefore, it’s easy for them to see the importance of sound as piece of the product design puzzle.

Q: Can you give us some examples of your sound design successes both in terms of execution and from a business perspective?

A: The first example is an “Audio Brand Language” initiative I did for a major appliance manufacturer that was successful because it was validated through consumer testing. A set of branded audio assets was designed for a group of brands. Consumer testing results showed that the sounds were aligned with the attributes of each respective brand image.

Another example is Kodak’s V-Series camera, which is considered the forerunner and leader of the Kodak Easy Share camera product line. When the series was first released, it was dubbed the “James Bond” camera. I worked with Kodak‘s creative team to design a sound experience that echoed the camera’s sleek James Bond style. When you turn the camera on, the power-on sound reminds you of a James Bond movie soundtrack. All the functional sounds that communicate user actions are derived from that central theme, so the entire user experience is unified.

Q: Can you give us some examples of great sound experiences in different areas of our lives today?

A: The first experience that comes to mind is Guitar Hero because it’s a perfect combination of sound and user interaction. Overall, video games have the best sound experiences found in any product today.

As far as devices go, the simplest yet most effective sound I’ve heard is the iPod click. Without looking at the interface, the click tells me if I’m going up or down in a menu. It also increases and decreases in volume as you change the volume of the current song. It’s easy to take the click for granted until you turn it off and try to navigate.

Skype is a good example of effective sound in a software application. It has a playful feel without being overly cartoonish, cliched, or annoying. When I use Skype, the sound of an incoming call never seems to bother me. Remember this: when you use a product, any product, if you turn off the sound, it wasn’t well designed.

In regards to environmental sound experiences, I think of the tunnel between Terminals B at C at O’Hare Airport. There, at one of the busiest airports in the world where people are under all kinds of stress, I always feel better after I ride the walkway through that tunnel. It’s almost spa-like with ambient sounds cross fading in and out as the user travels down the tunnel.

Finally, there’s the classic audio signature of the Harley Davidson exhaust system. When you hear that rumbling sound, you know it’s a Harley.

Q: Are organizations looking at product sonification as a competitive advantage in terms of user experience?

A: Absolutely. It’s been a slow process, but in the global marketplace, organizations continually seek a competitive edge, and they are starting to see sound as a medium that has been undervalued. Marketers and designers are starting to see sound as an important point of differentiation.

Look at mobile devices, appliances and cars—any product that a potential buyer can interact with at the point of sale. I’ve seen a refrigerator in a showroom that was not working, so there was no audio or visual feedback, not even a hum. Next to it was a similar model at the same price point that was plugged in and working. Naturally, more shoppers were opening and closing that product’s door. When more people interact with products at the point of sale, the potential for selling goes up.

Q: Beyond some of the business drivers, what are the significant technological factors behind this revolution, and how do they affect your work?

A: In recent years, audio technology has improved similar to computer processing. As a result, the price per unit continues to decrease. In mass production, even minimal savings per unit equates to millions of dollars saved.

I categorize audio playback technology into three main groups: pure tones, MIDI, digital audio. The cheapest and simplest solution is pure tones, like a monophonic chime created by a piezo. The second group is MIDI. MIDI technology is flexible and efficient. MIDI files take up very little memory on a device.

Digital audio is the third audio playback technology group. Unlike pure tones and MIDI, digital audio files, such as MP3, allow for endless sonic possibilities. Think of the Intel sound, which has 300 layers of sound, and how dynamic that sounds compared to a pure tone.

Sound technology is on a bell curve and is ramping up so fast it seems like every two or three years there’s a new compression scheme that’s half the size and twice the quality, and it’s cheaper.

Q: Do your techniques for conceiving and developing sound concepts parallel the visual design and creative development processes that would be familiar to our readers?

A: 75% of my process is a one-to-one correlation to that of a visual designer. Just like working in the visual design process, I study the requirements and digest the creative brief and overall direction. Then I sit down and make audio sketches, just like a designer might make on paper.

Sound sketching for me means working quickly to let ideas flow in a stream of consciousness. I come up with a hundred different quick melodic ideas and then start to refine them into sets of themes to recognize patterns—almost like an image board to refine colors and shapes, but with music, it’s about timbre, tempo, and other attributes that make sound unique. Next, I choose sketches that align with the overall creative direction of the project, make iterations, and take “comp” to the design team. Once we agree on a final theme, I build the audio assets that go into the product or interface.

Q: How does your process differ from the visual design process?

A: As far as the creative process goes, the differences are slim. But, when you get into the production end of it, there are huge differences in the technical aspect of the process. Obviously, the software and the techniques I use to produce the final assets are quite different. But the idea that you optimize your final assets for integration is the same. It’s just that the tools are different to get to the final deliverable.

Q: Much of what you do is on the bleeding edge. What industries or verticals use product sonification most effectively?

A: There’s an evolving need in appliances as they become more and more interactive. In general, the experience in one’s home, especially in the kitchen, is an area that is growing fast. Another area I haven’t worked in, but I think is going to be huge, is the medical field. There is room for improvement in patient care both from a usability and experience standpoint. This includes the sounds devices make, the way technicians retrieve information, and the like. I’ve got some good ideas of how this experience can be improved through sound.

Mobile devices, especially cell phones, are on the bleeding edge because they’re based on communication through sound. Phones are small devices that demand high quality sound output. Therefore, more effort has been put into these products than, say, appliances. Automotive is also a growing market. As the interfaces in vehicles improve in audio playback and the types of interactions people are having increase in complexity, there is going to be a greater need for a cohesive experience with those interfaces that align with the brand image of the vehicle. Lastly, environments where there is a lot of secondary noise like airports, subways, and other mass transit systems are ripe for sound design. It’s going to be a real challenge to improve those environmental situations from an audio perspective.

Q: What does the five-year horizon look like to you?

A: The big picture is really exciting. It’s actually hard to imagine how far this field could go because, like computing, 10 years ago it was a totally different landscape. But, from a business perspective, as technology becomes cheaper and generates higher audio quality, I think more businesses will embrace audio as an integral part of their product design roadmap.

You’re going to see more and more people get into the field of sonification and you’re going to see the awareness and the quality improve. It’s a revolution because a lot of this technology is so accessible. You don’t necessarily need a music studio. You just need a laptop, software, and a good understanding of design to produce good results.

Karel, thank you very much for a fascinating conversation.

It’s been really interesting for me, too.

This article was originally published on the User Interface Resource Center (UIRC). For more info, please see https://uxmag.com/uirc