Stakeholder Research Precedes UX Research

August 20, 2012

One of the worst things that can happen to a UX researcher is when the people who have asked for a study regret, reject, or refute key components of the study—either when it is currently in progress or after you’ve completed the study. To avoid this frustrating situation, people who practice UX research must set the right expectations with stakeholders who request a study or—if it was you who suggested doing a study—say they want one. I cannot stress enough the importance of starting off a study on the right foot. The quality with which a study begins has a huge impact on its outcome. In this article, I’ll provide guidance about what questions to ask your stakeholders prior to any UX research activity. The answers to these questions can help you create a study proposal and set stakeholder expectations.

Champion Advertisement
Continue Reading…

Why Interview Stakeholders?

As soon as your immediate stakeholders express interest in conducting a study, schedule a 30-minute meeting with them. These stakeholders may be, for example, a product manager and lead software developer or a designer and marketer. You should interview your stakeholders for the following reasons:

  • They are aware of the business goals for the product.
  • They have a clear understanding of priorities.
  • They can help you identify the right audience for the product and characteristics of study participants.
  • They’ll establish the development timeline and lead the implementation of any changes that might result from your study.
  • They’ll—especially software engineers and designers—actually make the changes to the product based on your study findings.
  • They may provide you with the wireframes, mockups, or prototypes you’ll be using during the study.
  • They’ll be great at helping you capture critical observations.
  • They might also play a critical role in recruiting study participants.

Question 1: What Is the Product?

In some cases, you may be so involved with the product team that you do not need any introduction to the product. If so, you could skip this question—but beware of skipping it entirely. Even if you are involved with the product team, people may forget to update you about a new feature they’re working on, so you should always ask some form of this question. For example, We’re talking about product X, right? Anything new I should know about it?

If you’re not familiar with the product, ask stakeholders to give you a short product pitch: Why should it exist? What is it for? What does it do? Have you launched the product yet? Why did you develop it? What user needs does it meet?

Try to get stakeholders to give you answers that are less about the technology and more about the business of the product. Don’t get into too many details. At this point, you need a high-level pitch only. You’ll get all the necessary details later when you prepare your study proposal. Once you’ve got the gist of the product, it is time to find out who its users are.

Question 2: Who Are the Product’s Users?

It’s a good sign if your stakeholders’ answers to this question generate a discussion. This means that someone, at some point, has actually thought about the people who are or will be using the product. If there’s a debate about their characteristics, that’s okay. The discussion should lead to a decision about what participants you should invite to the study.

So ask the stakeholders these questions: Who are the users? Are there different groups of users? Why? Try to lead the discussion toward identifying one group of people who share certain well-defined characteristics. If you notice that several groups of people, having different characteristics, come up during your discussions, that’s natural and probably okay. Don’t push too hard to come up with a single group at this point in time. You might return to this issue later on during your study’s planning process. For now, you just need to make a quick decision.

As you hear your stakeholders’ answers, consider whether it would be hard to recruit the group of people that they’re describing. If it seems difficult, ask your stakeholders to come up with alternative characteristics or a different user group—just in case it turns out to be extremely challenging to find the people they want for the study.

One special case of the Who are the users? question is when that question is itself the focus of your research. In such a case, their answer is usually something like: “This is what we want to find out.”

Now that you know the what and the who, it’s time to talk business and discuss the core reasons for the study.

Question 3: What Do You Want to Know? Why?

Ask about things that seem to be related to the study request: What do you want to learn by conducting this study? Why do you think you need this study? What decisions do you need to make based on the results from this study? Try to find out why the stakeholders have made their request for a study. However, if you suggested the study, you’ll probably be the one answering these questions.

At this point, once you have some answers, you might be tempted to start thinking about what research method would help you to close the knowledge gap you’ve identified. But try not to think about that just yet. Instead, focus on listening to your stakeholders and gathering information. You can think about an appropriate method later. Take good notes on the discussion. This step is critical to your coming up with appropriate research questions.

For example, if you get an answer like “We want a usability test because we want to learn whether users like our product,” you might be tempted to observe that there are three challenges regarding this answer:

  1. The stakeholder has mentioned a specific research method—a usability test. According to best practices, you should first define your research questions, then choose a research method that is effective in answering those particular questions.
  2. The stakeholder has chosen the wrong method to answer his question. A usability test is not the best method for determining how much people like a product or service.
  3. The stakeholder’s statement reflects a built-in bias—the assumption that users like the product—and the desire to prove this hypothesis right.

Some researchers might respond to such a stakeholder request by explaining why it is wrong. But I would strongly recommend that you don’t say a thing about it. The reason is that, at this point, you are trying to gather enough information to enable you to come up with a coherent study plan with crystal-clear research questions. Your research questions will make this argument for you later.

Timing is everything. The next critical topics to discuss with your stakeholders are time, schedule, and deadlines.

Question 4: When Do You Need the Study Results?

To set better expectations, I always ask my stakeholders this question: When is the latest possible time that you’ll need the study results? This is to help ensure that the study will be effective and its results relevant. The answers I often get might surprise you:

  • Tomorrow—Bad surprise. In some cases, you can be flexible, but in many cases, you just can’t. When you get this answer, it’s an indication that stakeholders don’t know much about the UX research process, and you’ll need to schedule some time on both of your calendars to close this knowledge gap. Now, don’t get me wrong. I’m not saying research can’t be done quickly. There have been occasions when I’ve received requests for studies in the morning and provided answers in the evening of the same day—after running to the nearest mall to recruit participants for a one-task, ten-minute café study. (A café study is a low-cost, fast-turnaround usability study that is usually conducted at cafés, malls, or conferences.) Usually, when you must provide such a fast turnaround, it means you’ll have to cut some corners. Think about it before you say yes to a study that must generate results tomorrow. Sometimes cutting corners is perfectly fine. In other cases, it is inappropriate, unacceptable, and just plain wrong.
  • Next quarter—Good surprise. You’ll have lots of time to plan the study, gather great feedback, recruit the right participants, and analyze and communicate the results. Just be careful not to commit to too many studies in the same quarter. And please don’t take me too literally. While next quarter probably means that there is time for you to conduct the study without cutting corners, this may mean that you’ll need to provide results within a month or in a couple of weeks.
  • No idea—An unknown surprise. This is a trick answer. Stakeholders might need results next week or next month or next quarter. They just don’t know yet. They’ll know when X happens. Hear them out, but be sure to let them know that it’s possible you might not be available when they want to conduct the study. When you give them this kind of reply, it might make them realize that they need to give you a deadline after all.

The timing issue can bring a whole study project down or change it completely. Depending on the deadline that stakeholders set, an A/B test might change to an eyetracking study or a traditional usability test or a quick café study. In the same way, a field study could turn into a series of in-person, in-depth interviews or phone interviews or a quick survey. Now, let’s look at a question that I do not always feel comfortable asking, but that I have learned is extremely important: What’s going to happen once the study is complete?

Question 5: What Do You Intend to Do with the Research Results?

In many cases, as a researcher, you must make tough decisions and set hard priorities, deciding which client, team, or product should get some research love. Sometimes you’re not the one making those decisions, but when you are, you want to be sure that you’re making the right decisions.

One of the most important things to ask stakeholders who request a study is: What are you going to do with the results? This may initially seem like a rhetorical question. Of course, they’ll do something with the results. They’re asking for the study, aren’t they? Well, it’s not that simple, and it’s definitely not that clear cut.

All stakeholders who initiate a UX research project have a certain set of expectations. For example, they may expect participants to be able to do something, but not something else; expect certain concepts to be less well understood than others; or expect people to have one need rather than another. Your research will, hopefully, provide clear answers to their questions: Users can do this, but they don’t understand that; and most of them need a feature like this. However, in many cases, your study’s results will conflict with what stakeholders expected. Some of them may realize that their expectations were wrong. But others may choose to ignore your findings. This is why you need to ask this question.

When you ask What are you going to do with the results? you are actually saying, I might choose not to work with you if you are not sufficiently eager to act upon the study’s findings.

There are two types of answers you might get in response to this question:

  1. “Obviously, we will act upon the findings.” This is not necessarily a green light, but it’s definitely not a red light. The stakeholders might still ignore your findings, but at least they are open to acting on what you discover.
  2. “Well, it’s complicated; it depends on what you find; we’ll see.” This is not a red light either, but it’s a warning sign. Beware. Sometimes, if I think it will help, I immediately reply, “In this case, I must consider whether I should invest time in the study.” The stakeholders usually change their answer at that point. If either they don’t or you choose not to reply aggressively, there’s another approach I sometimes like to take: I offer not to provide recommendations after the study, only findings. Or I may suggest that we agree together on what to do about what we discover during the study. This usually helps. Green light.

Bonus Question: What Do You Already Know?

Sometimes stakeholders internalize study findings so completely that they think the research you’ve done was redundant. They may say that they already knew what you’ve discovered. Or that they don’t understand why the research was necessary at all. This is an extreme and, fortunately, rare situation that happens most often with studies whose goal was to discover user needs and wants. For someone who conducts studies, this is a dangerous situation, because it’s a signal that they won’t be asking for your services again the next time they want to learn about the needs of users. They’ll rely on what they find out themselves or feel is right or think is working just fine or what they learn from salespeople. Intuition sometimes works, but not always. When stakeholders say they already knew what users needed before your study, it’s a sign that you need to do a better job of interviewing stakeholders right at the start of a project.

To prevent this situation from happening, you might want to ask your stakeholders one simple question: What do you already know about user needs? Since, by now, you already know what stakeholders want to learn during your research project, all you need to ask them is: What do you already know about this? Ask your stakeholders to answer the research questions they want UX research to answer. Document their knowledge prior to the study. It’s as simple as listing the questions they want your research to answer and getting your stakeholders to answer them.

When you ask some questions, they’ll just reply that they don’t know the answers, and that’s why they’re asking for this study. In response to other questions, they’ll try to guess what answers the study might provide. But to a few questions, they’ll give detailed answers based on their current knowledge. In some cases, they might give you two answers, saying something like, “Big Boss 1 thinks users need X, and Big Boss 2 claims users want Y. We expect your study results to determine what we should do.” That’s perfectly fine. You’ll probably discover that neither of the bosses thought of Z.

Once you’ve documented what stakeholders already know and don’t know, save this information and keep it handy in case you need it. If, after you’ve presented your findings or even after a field visit, a stakeholder claims she’s learned nothing from your research because she already knew everything you’ve discovered, find this information and make smart use of it.

You have brought immensely useful information to the organization, and stakeholders should recognize that. This is a smart technique that sets you up for success by helping to ensure that people remember your contribution. Consider doing this business development for future engagements. However, you don’t want stakeholders to perceive you as being confrontational. When you have proof that your stakeholders’ perceptions were completely—or even slightly—off prior to your study, make your case in a positive manner. You don’t have to be aggressive to make your point. You just want to gently remind them why they asked for the study, what they said they knew before it began, and how different their answers were from what you’ve found.

Stakeholder Research Tips

To get the most out of your stakeholder interviews:

  • Shut up and listen first, before you start talking.
  • Take notes.
  • Do not interrupt.
  • Encourage a conversation among stakeholders.
  • Ask open-ended questions.
  • Look for body-language signals.
  • Don’t ask leading questions.
  • Don’t think too much. You are now in information-gathering mode. If you think too much now, you’re not listening.

Do these tips seem familiar? Of course they do. Our stakeholders are, in fact, study participants, too. So we interview them in the same way we interview other participants. Happy research! 

VP, Head of User Experience, at WeWork

New York, New York, USA

Tomer SharonA user experience researcher at Google New York since 2008, Tomer is currently doing user research for Google Search. Previously, he led the UX research effort for Google’s online advertising management platform, DFP (Doubleclick for Publishers). Prior to working for Google, he was a user researcher at Check Point Software Technologies in Israel, where he led the research effort for dozens of networking and Internet security products on various platforms. As founder and president of UPA Israel, Tomer led the chapter to many achievements, including raising awareness of the need for easy-to-use, efficient, and fun technology products, as well as growing and nurturing a professional community of 1,000 practitioners. He speaks at conferences and professional events, is a published author of articles and papers, and a past editorial board member for UX Magazine. Tomer is the author of the forthcoming Morgan Kaufmann book, It’s Our Research: Getting Stakeholder Buy-in for User Experience Research Projects, which will be out in 2012. He holds a master’s degree in human factors in information design from Bentley University.  Read More

Other Articles on UX Strategy

New on UXmatters