Aligning UX Issues’ Levels of Severity with Business Objectives

Envision the Future

The role UX professionals play

A column by Paul J. Sherman
October 4, 2010

Many of us in the field people now generally refer to as user experience have long used levels of severity as a means of indicating the criticality of a product’s or service’s usability issues to clients.

Over the past several years, I’ve grown increasingly dissatisfied with the vague and somewhat solipsistic nature of the gradations UX professionals typically use to describe the severity of usability issues. High, medium, and low don’t begin to sufficiently explain the potential brand and business impacts usability issues can have.

After incrementally iterating on several existing classifications of severity, I finally decided in late 2008 to simply create some new ones, which I’ll present in this column. For lack of a better term, I call them business-aligned usability ratings.

Champion Advertisement
Continue Reading…

What’s the advantage of my ratings? In short, business-aligned usability ratings help me communicate the actual business impacts of the issues we identify during usability testing. They include information about how a user experience issue can affect an organization’s brand equity and revenue.

Before I present these ratings, let’s first review some definitions of usability and user experience, as well as some common classifications of usability or user experience issues.

Defining Usability—Again

Let’s go to the more authoritative sources first. The ISO standard for usability (ISO 9241-11) defines usability as “the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use.”

Another popular definition is Jakob Nielsen’s conceptualization of usability as a “quality metric” with five components:

  • learnability—How easy is it for users to accomplish basic tasks the first time they encounter a product or service?
  • efficiency—Once users have learned a product or service, how quickly can they perform tasks using it?
  • memorability—When users return to a product or service after a period of not using it, how easily can they re-establish proficiency?
  • errors—How many errors do users make, how severe are those errors, and how easily can they recover from the errors?
  • satisfaction—How pleasant is it to use a product or service?

In my own practice, I’ve typically defined usability in a broad context. Here’s my working definition:

Your intended users can accomplish what they’re trying to do with your product or service.

However, as you’ll soon see, I’ve oriented my ratings of usability issues with a tighter focus on business problems and their solutions.

Defining User Experience

ISO 9241-210 defines user experience as “a person’s perceptions and responses that result from the use or anticipated use of a product, system, or service.” Other definitions follow this perception/emotion-centric view, as we can see in Wikipedia’s definition: “User experience is about how a person feels about using a system.” Others adhere to a component-based definition of user experience—for example, Peter Morville’s user experience honeycomb, which comprises usefulness, usability, desirability, value, accessibility, credibility, and findability.

I’ve always liked Morville’s definition of user experience best, but found even it somewhat lacking because—like other definitions I’ve run across—it didn’t seem to put enough focus on the business outcomes. So rather than simply creating and advocating for yet another definition of user experience, I began evaluating usability or user experience issue ratings with a focus on an organization’s business objectives.

Benefits of Business-Aligned Usability Ratings

Business-aligned usability ratings offer UX professionals several advantages, as follows:

  • They connect the dots between a given usability or user experience issue and the business ramifications of that issue, helping both you—as a UX professional—and your stakeholders understand what is at stake if you leave an issue unaddressed.
  • They help UX professionals focus on real issues—that is, issues whose resolution would have an actual impact on the user experience rather than issues that may be more academic.
  • They encompass both brand equity and revenue impacts, which other ratings don’t seem to cover adequately.
  • They are behaviorally anchored—that is, they are based on observable, well-defined, or inferable criteria.

Rating Usability or UX Issues

Here are my working definitions for four levels of severity for usability or user experience issues.

  • critical severity—An issue of critical severity would definitely result in users’ being unable to complete their intended task, as well as an immediate, noticeable, and significant negative impact on an organization’s brand equity, revenue, or profitability.
  • high severity—A high-severity usability issue is one that would likely result in users’ not being able to complete their intended task and, from a business perspective, negatively affect an organization’s brand, revenue, or profitability.
  • medium severity—A medium-severity usability issue would be likely to significantly impede or frustrate users, but probably would not prevent users from eventually accomplishing a task. It might also negatively affect an organization’s brand, revenue, or profitability.
  • low severity—A low-severity usability issue would likely present some type of momentary or transient difficulty or result in confusion for users, but would not prevent users from accomplishing their task. It would have no discernible effect on an organization’s brand, revenue, or profitability.

Why Consider Both User Experience and Business Outcomes?

In a word: alignment. The competitive environment for digital products and services has never been more tenuous for market incumbents or more full of opportunity for market entrants. To remain competitive, organizations must provide excellent, not merely passable user experiences. Loosely paraphrasing Jakob Nielsen, it is wise to remember two maxims:

  • All things being equal, most users spend most of their time on other sites—with the exception sites like Facebook, I’m guessing.
  • If a user isn’t happy with how your site supports their goals, they will leave for greener pastures.

All of this is just a fancy way of saying that user experience is an important competitive differentiator. To adequately leverage user experience design and evaluation, we need tools that enable us to better measure user experience. The rating scheme for usability or user experience issues I’ve presented here is one such tool that can help you to communicate the real-world, financial consequences of such issues to business stakeholders more effectively—as well as the potential costs of not addressing them. 

Founder and Principal Consultant at ShermanUX

Assistant Professor and Coordinator for the Masters of Science in User Experience Design Program at Kent State University

Cleveland, Ohio, USA

Paul J. ShermanShermanUX provides a range of services, including research, design, evaluation, UX strategy, training, and rapid contextual innovation. Paul has worked in the field of usability and user-centered design for the past 13 years. He was most recently Senior Director of User-Centered Design at Sage Software in Atlanta, Georgia, where he led efforts to redesign the user interface and improve the overall customer experience of Peachtree Accounting and several other business management applications. While at Sage, Paul designed and implemented a customer-centric contextual innovation program that sought to identify new product and service opportunities by observing small businesses in the wild. Paul also led his team’s effort to modernize and bring consistency to Sage North America product user interfaces on both the desktop and the Web. In the 1990s, Paul was a Member of Technical Staff at Lucent Technologies in New Jersey, where he led the development of cross-product user interface standards for telecommunications management applications. As a consultant, Paul has conducted usability testing and user interface design for banking, accounting, and tax preparation applications, Web applications for financial planning and portfolio management, and ecommerce Web sites. In 1997, Paul received his PhD from the University of Texas at Austin. His research focused on how pilots’ use of computers and automated systems on the flight deck affects their individual and team performance. Paul is Past President of the Usability Professionals’ Association, was the founding President of the UPA Dallas/Fort Worth chapter, and currently serves on the UPA Board of Directors and Executive Committee. Paul was Editor and contributed several chapters for the book Usability Success Stories: How Organizations Improve by Making Easier-to-Use Software and Web Sites, which Gower published in October 2006. He has presented at conferences in North America, Asia, Europe, and South America.  Read More

Other Columns by Paul J. Sherman

Other Articles on Usability Testing

New on UXmatters