Client Reactions to User Research Findings

Practical Usability

Moving toward a more usable world

A column by Jim Ross
August 8, 2011

“This is great, but if I presented this to our executives, it would be like setting off a bomb in the room.” That was the highest compliment I’ve ever received from a client after presenting my user research findings. The findings had revealed the failure of the application and uncovered sensitive political dysfunction within the company. The participant quotations were especially devastating.

Sometimes, as a user researcher, I feel like an investigative reporter, unearthing sensational findings and hidden truths that will amaze and intrigue my audience. Sadly, user research findings are rarely this sensational, and it is seldom that they completely amaze and stun our clients. By client, I mean anyone for whom we’re doing research—whether a consulting client, an internal client team, or a project team.

Champion Advertisement
Continue Reading…

Ideal Reactions to Research Findings

Ideally, our clients are as interested in our user research findings and recommendations as we are and find them valuable. User research usually reveals new information about users and provides helpful new insights and perspectives on something our clients already know about. It can also confirm and validate their assumptions. Regardless of whether our clients already know about something, our research should provide useful information that has its basis in an understanding of user and business needs.

Negative Reactions to Research Findings

Unfortunately, we don’t always get these ideal reactions to our user research findings. Sometimes misunderstandings about our research activities and their purpose can lead to the following reactions:

  • “Ho hum. Where are the designs?”
  • “We already knew that.”
  • “You’re wrong!”
  • “You talked with only 12 people.”
  • “Why didn’t you mention this problem?”
  • “The recommendations aren’t specific enough.”
  • “We could have done that ourselves.”

Ho Hum. Where Are the Designs?

I find it amazing whenever I’m the person in the room who is most interested in my user research findings, but that’s sometimes the case. Often, when I present my research findings, it’s the first chance clients have had to hear about users and their needs, so you’d think they would be fascinated by this information. But some people are impatient and just want to see the designs.

Sometimes there is a long wait before clients hear our research findings that fuels this impatience. Since user research is usually the first stage in a project and often takes a long time, clients must sometimes wait several weeks before seeing any results. Without frequent communication about what is happening during this time, clients can become impatient and anxious about not having seen any designs yet.

How to Prevent This Reaction

  • At the beginning of a project, clearly explain the research and design activities you have planned, show examples of what such research provides, and make it clear when clients can expect to see designs.
  • Provide frequent updates during the research, so clients realize that activity is taking place and have clear expectations of the information the research deliverables will provide.
  • Judge your clients’ appetite for user research, and if they are resistant, take small steps at first, providing tangible results. As your clients begin to buy in to doing user research, you can gradually increase the amount of research you do.
  • Make your research deliverables more impactful. Create empathy for real users and generate interest in them by including participant quotations—in text, audio, or video; showing photos of users’ environments, documents, and other artifacts; and creating personas and scenarios that bring users and their tasks to life.

We Already Knew That

When a client says, “We already knew that,” I always want to say, “Well, then why didn’t you do something about it?” Unfortunately, sometimes clients react to user research findings by wondering why they spent so much time and money for us to tell them what they already knew. Usually, clients exaggerate how much they “already knew” about their users. Plus, they may not really understand the purpose of the research.

Sometimes user research gets oversold as an activity that will reveal amazing insights. While user research does provide important information that we can use during the design process, it’s rare that our findings consist solely of completely new information that no one has ever thought of before. Unless our clients know absolutely nothing about their users—which is rare—our findings are going to include things they already know. However, what some clients don’t recognize or appreciate is the value in confirming or disproving their assumptions.

Although our clients may implicitly know about users and their needs, this information is not usually documented explicitly anywhere. There is great value in gathering, organizing, and formally presenting this information in research deliverables—both for the current project and for future endeavors.

Another reason for this reaction is that user research findings can sound like obvious common sense once we’ve formally stated them. So, after hearing the findings, it’s hard for people to remember that they didn’t realize those things before. They may have known about the problems on some level, but taken them for granted. It often takes someone from the outside to come in and point out problems to which others have become accustomed.

But most important, the main purpose of user research is not for our clients to merely understand the findings, but for the designers’ understanding of the users and their tasks to inform their designs. Regardless of whether our clients already know something, the designers need to know it to design effective solutions.

How to Prevent This Reaction

  • Make it clear that your research is likely to provide new information, as well as confirm or disprove existing assumptions.
  • Communicate the value of gathering, organizing, and publishing user research findings in official documentation that your client can use in future projects, as well as the current one.
  • Clarify that the main purpose of the research is for the design team to understand both the user and business needs that should inform their designs.

You’re Wrong!

On a recent project, we observed and interviewed a group of interns who were using an SAP Human Resources application. Our line of questioning gave them the correct impression that we didn’t know much about SAP. Later, they tattled on us to their manager. “These people aren’t SAP experts!” Alarmed, the manager went to our client, who calmed her down by assuring her that we weren’t supposed to be SAP experts.

We were fortunate to have a well-informed client, but some people assume that we’re supposed to be a subject-matter expert or will become one as a result of our research. Usually, we aren’t experts in the subject matter of our clients’ products. Our experience and expertise is in user research or design. We rely on our clients for knowledge about business needs, and we rely on users to gain our understanding of their needs.

User research findings report on what users say; their actions, behaviors, and environments; and our own observations and interpretations. Sometimes, what users say and do is not correct. So, because we rely on information from users, some of our findings may be incorrect. Nevertheless, incorrect findings can often reveal important insights. For example, a participant may tell you that there’s no way to perform a certain function in an application. That function may exist, but they just don’t know about it, which indicates a problem in itself.

How to Prevent This Reaction

  • Clarify that you are not supposed to be an expert in the subject matter or technology that is involved in a project. You are a user experience expert. You will learn about the subject matter from the clients and the users.
  • Communicate that, while research participants may give you incorrect information, it is important to understand their misconceptions.
  • Explain that analyzing research findings is an iterative process. You welcome the client’s feedback and clarifications, and if necessary, you can make revisions where your findings are truly incorrect.

You Talked with Only 12 People

“What about the other 10,000 users,” some clients ask. I’d like to say, “Okay, do you have hundreds of millions of dollars and a few years to spend doing research with all of those people?” What do they propose as an alternative then? Do research with all 10,000, or do nothing?

Clients who wonder how research findings from a small number of participants can be applicable to the entire user population are often under the misconception that user research involves asking people what they want. “How can you know what everyone wants by asking only a small number of users for their opinions?”

After a recent presentation, a client said, “It’s funny that so many of the problems you brought up were things that we created specifically because people had told us that’s what they wanted.” They had held meetings with employees to find out what they wanted in the application. The problem is that, in the abstract, people often don’t know what they want and can’t predict how their requests will affect an application’s usability.

How to Prevent This Reaction

  • Clearly explain that user research is about observing and understanding behavior, not asking opinions or gathering lists of wants.
  • Explain the value and validity of conducting user research with small numbers of participants to understand their behavior and find problems in a design. Explain the difference between surveys and user research

Why Didn’t You Mention This Problem?

I once had a client who wondered why I hadn’t mentioned the annoying horizontal scrolling in an application. Yes, it was an annoyance we would take care of in our redesign, but none of the participants had mentioned it or experienced problems with it. Does that mean it wasn’t a problem? No. But it wasn’t necessary to get into that level of detail in the findings. It was an obvious design problem that we would address in our redesign.

User research usually focuses on higher-level issues such as user characteristics, needs, and tasks, and the problems users face. Unless we observe people having problems with a particular part of a user interface or they mention it, user research often remains at a higher level than the details of specific user interface problems. It’s not the time to get into the details of usability problems, as we would when conducting a heuristic evaluation or usability test.

How to Prevent This Reaction

Explain the types of findings user research activities will provide. For example, contextual inquiry and user interviews provide higher-level findings about users and their needs rather than describing every usability or design problem.

The Recommendations Are Not Specific Enough

Some clients expect to receive specific, actionable recommendations that will solve every problem. For simpler problems, we can often provide specific recommendations. But more complex problems often require additional research or design exploration to solve. There isn’t always an easy solution that we can describe in the text of a report or presentation. For complex problems, it’s better to provide general recommendations in the form of design requirements rather than boxing ourselves or the designers into a corner with a restrictive and too-specific design solution. Later, during the design process, we can create design solutions for the problems we’ve identified.

How to Prevent This Reaction

Make it clear that you are presenting findings, overall recommendations, and design requirements. Explain why you’re not getting into the details of specific solutions yet. You need to keep your recommendations general to allow design exploration during the design phase.

We Could Have Done That Ourselves

“We can do that ourselves. We’re not potted plants!” was the reaction I received from an executive after presenting my user research results and recommending that we do similar research on a related project. Sometimes we make user research look so easy and the results seem like such common sense that clients believe they could do it themselves. And some of them probably could—with training, reading the right books, and some practice. But people often mistake traditional requirements gathering for user research and end up just asking people what features they want in the new application.

How to Prevent This Reaction

Explain how you conducted the research, distinguishing it from typical requirements-gathering activities. This will show that user research work is more complex than it might appear. Ideally, through their quality and depth, your research findings will differentiate themselves from traditional requirements gathering.

Set the Right Expectations

The overall lesson: negative reactions arise from misunderstandings about the purpose of user research, what it involves, and the results it provides. Clearly explaining the user research you plan to do at the start of a project, during the research, and before presenting the results of your research is the best way to prevent such misunderstandings.

In this column, I’ve discussed just some of the common reactions clients have to user research findings. I’m sure there are others. If you have others you’d like to share, I welcome your comments. 

Principal UX Researcher at AnswerLab

Philadelphia, Pennsylvania, USA

Jim RossJim has spent most of the 21st Century researching and designing intuitive and satisfying user experiences. As a UX consultant, he has worked on Web sites, mobile apps, intranets, Web applications, software, and business applications for financial, pharmaceutical, medical, entertainment, retail, technology, and government clients. He has a Masters of Science degree in Human-Computer Interaction from DePaul University.  Read More

Other Columns by Jim Ross

Other Articles on User Research

New on UXmatters