For nearly 25 years, the keyboard and the mouse have provided the physical connection that mediates most of the interactions we have with computer technology. But human communication is nuanced and complex, filled with physical elements, subtle signals, facial expressions, and gestures. With a touch, a glance, or a motion, we can convey a host of information. Till now, such subtleties have been largely absent from our day-to-day interactions with computers. However, gestures are beginning to make their way into our interactive vocabulary through a variety of innovative input devices that—although not new inventions—have reached a tipping point when it comes to affordability, availability, and consumer acceptance. From video games to mobile products to notebook and desktop computers, we are seeing an explosion of alternative input technologies and methods—touch screens, touch pads, tablets, wearable interfaces, and other devices.
For UX designers, these input devices open up opportunities for creating richer experiences that are more immersive and natural—expanding the ways in which people interact with computers. In the near future, we’ll be designing interactions that are not only usable and useful, but also deepen people’s connection with their technology. And as our dual lives continue to evolve, the most compelling products and services will help us bridge the gap between the virtual and the physical worlds.
The Human Interface Device Revolution
Human Interface Device (HID) is the name for a USB input specification for keyboards, mouse devices, joysticks, and other input devices, but it can also be a useful term for describing an expanding category of digital input devices. As UX professionals, we think about a person’s interaction with a graphic user interface (GUI), but rarely address the physical connection between the user and the computer. However, with the host of new human interface devices already available—and even more coming to market—we’ll soon have many more input methods to consider as we design digital experiences. To understand these devices, let’s begin by examining those for specialty purposes such as digital gaming, music, and art. The human interface devices already in use for niche products will no doubt influence the general adoption of new input technologies.
The video game industry has always pushed boundaries when it comes to creating unique input devices—and has the market reach to gain their widespread acceptance. I can remember my initial confusion when, in the late 1970’s, I first used the Atari 2600 paddles for games like Pong and Breakout, which soon turned to delight once I understood the method of interaction—turning the wheel slowly or quickly to position the bar that represented me, the player, onscreen. Today, of course, the gaming industry is producing far more sophisticated devices for interaction.
Sports, exercise, and other active games lend themselves to the incorporation of physical motion. Game systems can capture physical motion through dance pads that sense a user’s steps, steering wheels that let a user control the movement of a vehicle onscreen, or musical instrument-like devices that take in a user’s drumming or guitar strumming. The innovative Nintendo Wii controllers, which utilize accelerometers and optical sensors, let users control a game through movements that simulate live action—you can practice your golf swing or bowling technique in a digital world. Gaming platforms like the Wii are particularly intriguing, because similar controllers for the Web could let users browse sites in nontraditional ways.
For musicians, a host of input devices are available and have evolved from the days of MIDI-synthesizers to today’s PC-connected controllers. Devices range from piano keyboards to guitars to wheel controllers that mimic turntables, all outfitted with USB connections for a desktop or notebook computer. The Korg Kaoss pad, shown in Figure 1, is a touch pad that enables performers to manipulate samples and digital audio effects with hand gestures and motions. An advanced version of the device, the KPE1 Kaoss Entrancer, can also manipulate video images and visual effects in real time.
For years, drawing tablets like the current Wacom Intuos model have been important tools for digital artists. The tablets, ranging in size from 4 x 6 inches to 12 x 19 inches, allow artists to use a stylus to draw on a flat, smooth drawing surface—similar to a large track pad—and see their strokes appear onscreen in Photoshop or another drawing application. The drawing tablet, while not as versatile as paper, mimics the physical experience of sketching, which enables artists to apply the real-world techniques they’ve learned to their virtual art. Gradual advances—like stylus tips with different shapes and levels of hardness, thicker stylus grips, and the ability to sense the angle of the stylus’s tilt—have made it possible to replicate tools as varied as paint brushes, pencils, felt-tip markers, and airbrushes. Flipping the stylus over automatically switches to the application’s eraser tool, making the experience of using the input device almost exactly like that of using a pencil.
An even more compelling input device, the Wacom Cintiq pen display, shown in Figure 2, lets users work directly on a large LCD screen, reducing even further the distance between their hand motions and the resulting visual display.