Audio and the User Experience

Expanding the Role of Audio

As applications move online, digital devices proliferate, and user interfaces spring up like dandelions, it’s worth asking whether user interfaces must remain mute or there are useful ways in which we can expand the role of audio in UX design. Specifically, are there ways to provide effective audio cues to better help direct or assist users?

We can examine other areas of design—industrial design and game design, in particular—for inspiration.

The video game industry has a long history of successfully using audio to enhance the user experience. Musical themes help establish key characters. Music and ambient sound build moods—whether cheery or spooky—based on game play. And games use sounds to steer gamers toward certain goals or notify them of the rewards they’ve earned.

The Philips HeartStart Home Defibrillator provides a compelling example of audio in industrial design. The device uses audio instructions along with information graphics to guide a lay person through the process of administering a shock to a victim in sudden cardiac arrest. Audio also saves lives in the complex user environment of the commercial airline cockpit, where proximity alarms that warn pilots of impending mid-air collisions give explicit spoken commands to ascend or descend in order to avoid disaster.

In emergency situations, where there is an urgent need for decision making, audio can be a compelling way of ensuring users take a certain action. We’re hardwired to obey audio commands, which is why these systems are effective. No clutter. No ambiguity. It’s loud and clear.

Audio Usage in UX Design

There are three areas where we can incorporate audio into our user experiences.

Ambient Background Sound

Our day-to-day lives are filled with background sound, some of which we enjoy—say, ocean waves at the beach—and other sounds that we strongly dislike—like construction noise.

While ambient sound works well in video game design to establish mood and location, it doesn’t have a clear role in other user experiences like software applications. However, as computers become quieter, will we miss the sound of the fan that indicated our device was running smoothly? Will people—especially in cubicle-filled offices—need the equivalent of a white-noise generator in their computers to raise the background sound level high enough to drown out distractions? (Anyone who has worked in an office in which the HVAC system has suddenly shut down knows just how noticeable their neighbors’ paper shuffling, coughs, and phone conversations can become.)

Task- and Event-Based Signals

When it comes to sound design for digital user experiences, this is the use of sound that is the most developed: Blips and bleeps indicate you’ve sent an item to the trash or an email message has just arrived.

But we can expand task-based audio cues to include actions that are specific to expert users. For example, in Photoshop, visually aligning an object to a pixel-specific guideline can be a challenge, especially when viewing an image at less than 100%. An audio signal—say, a simple click—to let you know the object is aligned to the guide—could increase the productivity of digital artists and visual designers.

Additionally, we often take for granted the offline audio cues that assist input to digital devices. The click of the mouse and the tap-tap of the keyboard reinforce the fact that we’ve completed a particular action—for instance, entering a line of text or selecting an object. This is fine when a user has a physical keyboard, but in cases where the user inputs data via a touch screen displaying a virtual keyboard, these sounds are noticeably absent.

Warnings and Notifications

In a 2003 column examining voice recognition and audio interfaces, usability guru Jakob Neilsen—after pointing out the numerous detrimental aspects of such systems—makes this semi-positive comment:

“…voice could be used to direct the user’s attention to important events or elements on the screen in a richer way than the obnoxious beep that currently constitutes most computers’ audio vocabulary.”

We could use voice audio to help walk users through critical situations or even to avoid such situations altogether. Laptop users might benefit from a voice reminder that their battery needs recharging or to back up their files before shutting down or the holy grail—a warning about an imminent hard drive failure.

Similarly, in less critical situations, we could use a voice reminder to indicate that a user has missed a field when filling out a form. Instead of forcing a user to scan an entire page to find the red text indicating which of a dozen fields he left blank, a voice could simply say, “Please type your name.”

The disadvantage to such notifications is that people quickly tune out false alarms. So, extensive testing would be necessary to keep inaccuracies to a minimum. And because audio is intrusive, it’s best to ask for a user’s permission before using audio notifications—and always provide a clear way of disabling the audio.

Audio, when used judiciously, enhances the user experience through its engagement of another human sense and by providing a richer atmosphere for interaction. We shouldn’t allow the abuses of audio in the past—which have resulted in the banishment of most audio from our daily online interactions—to prevent us from trying again to use audio more effectively. While incorporating audio cues and other sounds in UX projects may be foreign territory for most visually oriented UX professionals, it is territory well worth exploring.