Closing the Communication Loop

By Steve Baty

Published: March 12, 2008

“What usability testing won’t necessarily tell us are the ways in which an application fails to meet users’ expectations.”

When our online service channels fail to meet the needs of our customers, if we’re lucky, customers will resort to an alternative channel to get the assistance they need. In doing so, our customers offer us the potential of gaining rich insights into their needs and mental models. Feedback forms, complaints, call center logs—all of these tell us valuable information about customers’ failed interactions.

It’s in the nature of user experience work that we really begin to understand the success of our designs only after a project goes live. We minimize the risk of a complete failure by using iterative design methods and carrying out usability testing at various stages of the implementation. Whether we follow user-centered design or activity-centered design or even agile development methods, there is a certain element of uncertainty about the quality of the finished result until it hits the production servers.

What usability testing won’t necessarily tell us are the ways in which an application fails to meet users’ expectations—whether by way of missing information, missing features or functions, or things that simply fail that most pragmatic of tests: could users be bothered?

Lessons Learned for a Site Redesign

A few years back, I was involved in the redesign of a Web site for a consumer electronics company. The existing site was considered to be a success, but had passed its use-by date. Internal stakeholders felt the site was visually tired, but otherwise working fairly well to support the company’s offline activities.

During the initial research work I undertook on the redesign project, I had the opportunity to interview the customer service manager in charge of the company’s call center. She told me how busy the call center had become over the previous two years, as the sales of the company’s product lines increased. They were handling just over 300,000 calls a year at the time of my research.

I was interested in learning what sorts of problems had triggered so many phone calls and set about reviewing a month’s worth of call issue logs to look for patterns. What turned up was surprising. Approximately 40% of all calls derived from one of two questions:

  • Where can I purchase X product?
  • I have X product. Where can I get it repaired?

A further 15% of calls related to requests for copies of operating manuals, lost during a house move or thrown out by mistake.

When I brought these issues to the company management, they were surprised. They maintained a database of retail outlets and authorized service centers that the call center used in answering customer enquiries. Similarly, the company’s technical officers maintained a repository of manuals and other documentation. What was more surprising to me was that all of this information was already available on the company’s Web site. It was just very, very hard to locate. But instead of improving the site’s information architecture, the company was increasing the call center’s staff.

As a short-term measure, the company decided to add a message to the on-hold music on the call center phone lines, informing callers that information about where to buy its products and the location of its service centers was available through the Web site. However, rather than reducing the number of phone calls, the incidence of complaints increased as frustrated customers took the opportunity to vent.

Taking a step back, it was painfully clear that the audience for the site was looking for the information on the site, failing to find it, then picking up the phone as their fall-back. So promoting the visibility of the company’s retailers and authorized service centers became priority number one for the site redesign project.

The redesign resulted in a reduction of around 25% in the total volume of calls to the call center—despite increases in product sales. A review of page view data and database query logs highlighted the success of this project, showing a proportional increase in traffic to the areas of the site containing these high-volume information targets.

Incorporating Customer Feedback into Product and Service Designs

“Call centers provide us with a real-time snapshot of our customer’s needs.”

Call centers, in particular, provide us with a real-time snapshot of our customer’s needs. Incorporating issue logs into our project research allows us to react to those needs on a regular basis, minimizing or eliminating potential pain points before they get out of hand. Online channels can be similarly useful when undertaking a review of a Web site.

Some companies treat complaints as a painful reminder that things aren’t ideal. Sometimes a complaint can generate a flurry of activity aimed at placating an angry customer—the bigger the customer, the bigger the flurry. Smart companies, however, actively review all customer complaints and feed them into their improvement programs to turn frustration into satisfaction and delight.

“Smart companies actively review all customer complaints and feed them into their improvement programs to turn frustration into satisfaction and delight.”

One of the first steps I take when redesigning a corporate site is to ask for copies of any complaints the company has received about their Web site in the past twelve months. In addition, I like to ask the company what the response was to each complaint.

I recently worked with a manufacturing company that had taken the unusual step of sending all customer complaints on to their product design and engineering teams. The design and engineering folks used these complaints to generate ideas for new products, then involved the customers in product testing ahead of a product’s full-scale release. This approach to product design has resulted in high customer loyalty in the face of stiff competition from large international firms.

Another recent project highlighted the value of online Help systems in answering a range of questions about customer expectations that would otherwise be difficult to understand. A large transportation company was operating a highly-successful ebusiness site, with high transaction volumes, high customer satisfaction ratings, and low customer turnover. Customers were completing upward of 90% of all interactions online, with rapid transition to online services in new markets around the world.

However, a review of the company’s ebusiness strategy highlighted a significant gap in the site’s operation: the company acquired very few new customers through the site. Customers could have signed up through the site, but very few were actually doing it.

Our initial review of the site showed what we thought to be the cause of the low volume of signups online: a convoluted registration process that could take 48 hours to complete due to the need for the company to generate a personalized digital certificate for each new customer. So our first recommendation was to streamline the registration process to make it easier for potential customers to get started.

Then we turned our attention to the site’s online support—live chat with customer support personnel. After reviewing about 2,600 logs from these chat sessions, we concluded that the underlying problem was something else entirely: potential customers were looking for some information the company didn’t want publicly available and would provide only to registered customers. In the absence of that information, potential customers were reluctant to go through the registration process—no matter how quick and easy we made it.

Responding to the needs of this audience segment has resulted in a shift in the company’s organizational priorities, increasing the importance of new customer acquisition through the ebusiness site and facilitating a change in the way the company manages and publishes the information customers need when signing up.

Looking Beyond the Organizational Boundaries

“Users are talking to us constantly, through a wide variety of channels.”

These cycles of design, implementation, and feedback are familiar to most user experience professionals. What we sometimes fail to consider is that users are talking to us constantly, through a wide variety of channels.

But we can take this concept a step further, especially if our business models involve a chain of relationships from product design, manufacturing, distribution, sales, and after-sales support. No matter which link we provide in the chain, our customers are engaging in conversations with other organizations up and down the chain, too.

These other links in our webs of business relationships can also provide us with insights into the needs of our own audiences. At the very least, we might ask these business partners for information about common customer complaints or requests that ultimately arise out of failures on our part. For example, a product retailer may field a large number of questions about a particular feature, reflecting a gap in the information available through our Web site.


Mining online Help and offline service channels for recurring complaints, requests, and enquiries can inform the design of our online service channels, making them more valuable to the site’s audience and introducing new efficiencies into an organization through the migration of customer enquiries from expensive offline support centers to relatively inexpensive online channels. Closing this communication loop enriches our understanding of a site’s audience and provides a low-cost user research tool that contributes continuously to the design of online services.


A very interesting and useful article, Steve. Most of this could probably be successfully implemented in a lot of companies.

Great post.

It’s funny how much a Web designer can be like an unsung super-consultant, reorganizing entire business strategies by gently pointing at (gasp) the needs of the customer. You’d think it would be standard practice to perpetually implement changes to improve feedback, but I can’t tell you how many times that I, as a customer, have dug my way through a Web site to find a way to make a simple suggestion—that would have saved customers loads of time and energy—only to have it fall on deaf ears.

Great article to serve as a reminder… Two, in fact.

  1. Help files and internal call center resources—call logs, an so on—are great sources for finding points of pain and items needing to be fixed.
  2. No matter whether you have it available on your site or not, if the customer cannot find it, it does not exist.

Great thoughts and examples. I couldn’t agree more that smart companies view any communication from their customers, no matter the channel, as an opportunity to learn and improve.

Great article and great “super-consultant” comment by mc. I’ve said, at work, that we need to talk to receptionists and other front-line employees to learn more about what our site’s visitors actually need from the site, but now I have an excellent article to learn from and refer to.

I just saw this Boxes and Arrows article, which oddly has the same story as this UX article: Extreme User Research

Weird. I have no doubt that this story plays out repeatedly around the world, but…what a coincidence?

Join the Discussion

Asterisks (*) indicate required information.