What Is the Client’s Role in User Research?

Practical Usability

Moving toward a more usable world

A column by Jim Ross

March 4, 2013

You’ve just signed a contract with a consulting firm, engaging them to provide user research services. You’ll write checks to pay for those services. You’ll attend meetings. But what else does your role as a client on a user research project entail?

In my column, Practical Usability, I usually discuss user research from the researcher’s point of view, but clients play an important role as well. Successful user research involves close collaboration between clients and researchers to ensure that the research focuses on the right issues and provides acceptable recommendations. So, in this edition of my column, I’ll speak directly to the clients of researchers about the steps they should take to stay involved throughout a user research project and ensure its success.

Champion Advertisement

Define the Problem

As a client, the first thing you should do is define the problems you want the research to solve. Before the project officially starts, gather the main stakeholders—the people who have a stake in the project’s outcome—to answer these questions:

What are the problems that you want to solve?
What do you already know about the problems?
What do you want to learn through doing user research?
When do you need to have the findings?
What type of deliverable would be most effective for your organization—a detailed report, a formal presentation, or a quick and informal discussion of the findings?
What are you going to do with the results?
What obstacles might stand in the way of implementing the findings?
What would you consider to be a success for this project?

To properly focus the research plan, you’ll need to convey this initial information to the researchers at the start of the project.

Kick Off the Project

At the kickoff meeting, present the information that you’ve gathered about the current situation and the problem you need to solve. Invite stakeholders who can help provide insights into the following:

reasons for the project—that is, the problems you need to solve
any prior research and the information you’ve gathered on the issues
user groups, their characteristics, and the tasks they perform
business or organizational goals for the research
what you want to learn from the research

If there is an existing application, provide a demonstration of the main tasks that users perform with it.

Ease into the Subject Matter

Especially for researchers coming in from the outside, one of the most difficult parts of beginning a new research project is learning the subject matter. As the client, you and the other stakeholders are the subject-matter experts. Educate the researchers and designers by providing a high-level overview of your business, as well as any background documentation that would help the project team to get up to speed quickly.

Provide Enough Time for Research

Unless there’s an important deadline that you must meet, be flexible about scheduling the research—and don’t create artificial deadlines. Unnecessary time pressure often results in poor-quality findings. If you’re going to spend a lot of time and money on user research, allow enough time to both prepare for and conduct the research properly and thoroughly analyze the results.

Select Stakeholders for Interviews

After the kickoff meeting, researchers often gather additional information and perspectives through stakeholder interviews. Select the stakeholders who should participate in these interviews. Remember, stakeholders are not users. As the term implies, they’re people who have a stake in the project, and their role is to represent business needs.

Recruit Participants

At the beginning of a research project, no one knows more about your product’s users than you do—especially when the researchers aren’t part of your organization. So your helping to recruit the right participants is crucial to ensuring the success of the research. Describe the characteristics that define each of the user groups to the researchers. They will use that information to create a screener document comprising a list of questions that potential participants will answer to determine whether they’re the types of people you’re looking for.

On some projects, the researchers or a recruiting company will recruit the participants. However, when the participants are your employees, customers, or members, with whom you have existing relationships, it usually makes sense for you to do the recruiting. People are more likely to participate in research when they get a request from someone in their own company, a company with whom they do business, or an organization of which they’re a member. Ideally, you already have access to lists of employees, customers, and members.

Delegate Recruiting and Scheduling

However, even with the advantages of an existing relationship, it’s not always easy for a client to recruit participants. Consider whether you have the time and the necessary skills to contact, screen, and schedule participants. If not, delegate this task to someone who does. If you don’t have time, but want someone in your organization to handle recruiting and scheduling, an administrative assistant might be able to handle this task.

Get a Description of the Study from the Researchers

Ask the researchers to provide a description of the study and what participating in the research requires. You can use this description in your email messages or phone calls to prospective participants. When sending email messages to participants, include your name and contact information to make them seem more legitimate. Very few participants will actually contact you, but it makes them feel better if they know there’s someone they can contact.

Ensure That You Get the Right Participants

Be careful to recruit the right types of participants. I’ve had clients make the mistake of recruiting experts—the people who know the most about the system—rather than typical users. For example, they might recruit system administrators or department managers because they know so much about the system and can provide the most information. But these experts are far from being typical users, so involving them skews the results of the research. Be sure your researchers provide you with information about how to recruit participants, as well as details about the types of people to recruit, then follow their instructions.

Schedule Logically

Schedule the research sessions logically. Allow time for the researchers to get from one place to another between sessions. Include time for meals and breaks—plus some extra time, in case sessions go longer than expected. Scheduliing sessions within a tight time frame generally works better than spreading too few sessions over too many days. Schedule sessions logically by their location. Needlessly sending a research team on multiple trips would waste a lot of time, and you’d pay a lot in travel expenses. For example, I once drove from Philadelphia to Reston, Virginia—a seven-hour, round-trip drive—for one 90-minute contextual inquiry.

Review the Discussion Guide

The researchers will prepare a discussion guide for the sessions. This is a document that lists the tasks that the participants will perform and the questions that a researcher will ask the participants. Carefully review the discussion guide in advance of the sessions to make sure the research will obtain the information you need. Discuss any changes that you’d like to make with the researchers.

Nevertheless, it’s important to realize that a discussion guide really is just a guide. Research sessions are unpredictable and never go exactly as planned. Researchers need to be flexible and make changes on the fly to capitalize on unexpected opportunities that arise. You have to trust the researchers to use their judgment regarding when to stick to the guide and when to make changes to their approach or ask additional questions.

Observe the Research

There’s simply no better way for you to understand your users and the problems they face than listening to them and observing them perform tasks. So, if you can observe the research sessions without affecting the comfort of the participants or the quality of the research, you should. Sure, you’ll get a final report or presentation, but that doesn’t offer nearly the same value as observing the research firsthand. Even watching video clips isn’t the same as being there in person. If you really can’t attend in person, at least try to observe and listen to the sessions using Web conferencing tools.

Encourage Others to Attend

Encourage others in your company to attend the research sessions as well. Include people who might not know much about usability and user research. There’s no better way for them to gain an understanding of and empathy for your users and the problems they face.

Pay Attention

This may sound obvious, but it’s more difficult than you might think to focus your attention on observing the sessions and make sure that everyone who is observing pays attention throughout the sessions. After watching several repetitive sessions, it’s all too easy to lose focus and get distracted by checking email, looking at Twitter, reading blogs, taking calls, or trying to get some work done. To keep everyone focused, take notes and discuss what you’re seeing with the others in the room. List major issues on a whiteboard.

Keep an Open Mind

You’ll probably come into the research with some existing hypotheses and preconceived notions, but don’t let those close your mind to information that differs from them. People tend to dismiss information that doesn’t fit with their beliefs. At worst, observers might blame a participant for not fitting the user profile that they expected, allowing them to dismiss everything the participant says and does. Try to remain objective about what you’re seeing.

Don’t Jump to Conclusions

During the research sessions, the researchers see patterns developing, but they wait to gather all of the data before analyzing it and making any definite conclusions. In contrast, observers sometimes jump to conclusions after watching only a few participants, then view the remaining participants’ sessions as either reinforcing their conclusions or being exceptions that they can dismiss. Although it’s a good idea to list and discuss the themes and issues that you see emerging throughout the sessions, keep an open mind, and don’t come to your final conclusions until the research is over.

Resist the Urge to Make Changes During Testing

The best way to drive researchers crazy is to make a lot of changes to the discussion guide and the prototype during usability testing. The more changes you make during testing, the more opportunities you create for error and inconsistency.

Changing the discussion guide is likely to confuse and throw off the researchers, who already have a great many things that they must do and remember during the sessions. If, on top of all this, you make changes to the guide, you’d also be making them either type up the revisions to the guide, then reprint it; manually write the changes on every printed copy of the existing guide; or try to remember all of the changes.

Making changes to the prototype or the discussion guide also risks affecting the integrity of your test results. If the researchers give task A to participants one through three, then give Task B to the remaining participants, you’ll have seen only three participants perform Task A. You’ll never know what the remaining participants would have done with that task. Similarly, if you change Page X to Page Y after seeing three participants’ reactions, you’ll have made a design decision based on the results from only three participants. Remember, a good researcher gathers information objectively and starts noticing trends and issues, but waits to make any final conclusions until he’s gathered all of the data. You should apply the same philosophy to making changes during testing.

The exception to this general rule is when there are problems—such as a poorly worded question in the discussion guide that is confusing participants or an unexpected problem with the prototype that affects how it works. In such cases, it makes sense to make the changes that are necessary to correct the problems. But resist the urge to make other changes while you’re at it.

Arrange a Good Time to Talk with the Researchers

Some observers expect to be able to contact the researchers during the research sessions—perhaps to give them instructions or suggest additional questions to ask participants. But doing so would be extremely distracting to the researchers and disruptive to the sessions.

During each session, researchers must observe, listen, understand, think about the next question to ask, maintain a neutral demeanor and body language, make good eye contact, and establish a good rapport with the participant. Asking the researchers to take questions from the observers as well would require them to handle the additional tasks of noticing questions that come in, understanding what those questions mean, thinking about how and when to ask the questions, and rephrasing the questions in an unbiased manner.

A better way to handle any additional questions that you might want to ask a participant is to save them till the end of each session. At that appointed time, the researchers can come to the observation room, note your additional questions, then return to the participant and ask those questions.

Never Interrupt a Research Session

Unless it’s a true emergency, never interrupt a research session. The integrity of the research is far more important than your ability to observe a session or communicate with the researchers.

Don’t Observe Field Studies

While there are great benefits to observing user research in the lab, it’s best if clients don’t observe field studies. Because field studies involve observing participants in their natural environment at home or at work, there is no observation room. So there is no way for clients to observe the sessions without their presence affecting the research.

Because participants perceive the clients as representing the product or system that is the subject of the research, they often feel uncomfortable when they’re present. As a consequence, they may censor what they say and do during a session. In addition, the limited physical space in a participant’s environment makes it difficult to include extra observers in a field study.

The quality of the research itself is far more important than your ability to observe. If you absolutely must observe field studies, schedule a few extra sessions and attend only those sessions to get a feel for the research. For the remaining sessions, trust your researchers and rely on their findings.

Feed the Researchers

During the research sessions, the most important people to keep mentally and physically sharp are the researchers. Doing user research is very mentally demanding and can be exhausting. Researchers who are fatigued or hungry won’t be in the top mental condition that is necessary to effectively facilitate the sessions and notice everything that occurs during them. So don’t schedule too many sessions in a day, provide enough time for breaks between sessions, let the researchers relax during the breaks, and make sure the researchers get fed.

Allow Enough Time for Analysis

A common lament of researchers is that there’s never enough time for analysis. Few people other than researchers understand what the analysis of research data involves. For clients, it often seems like there’s a long wait between the end of the research sessions and the presentation or report on the findings.

A thorough analysis is very valuable, but also very time consuming. It typically involves listening to or watching the recordings of all the sessions, while taking notes; combining related findings, organizing the findings, looking for patterns, coming up with recommendations, and creating the deliverable.

How much time is enough for analysis? This depends on several factors:

number of participants
length of the sessions
type of research—a well-defined activity like usability testing takes less time to analyze than something more free form, such as field research

Don’t set arbitrary deadlines. Yes, there will be times when you do have tight deadlines, but otherwise, be sure to allow enough time for analysis. The more time you allow, the more detailed and higher quality the results of the analysis will be. If you’ve spent a great deal of time and money recruiting participants and conducting the research, rushing through the analysis to get the deliverable out fast is a tremendous waste of time and resources.

Wait for the Findings Before Making Changes

Some people make the mistake of implementing changes immediately following the research sessions, without waiting for the researchers to present their official findings and recommendations. Unless there is an urgent problem that needs to be fixed immediately, it’s best to wait. While you’re waiting for the researchers’ presentation, it is okay to discuss what you’ve observed and talk about some possible solutions. But don’t take any actions until you’ve reviewed the final deliverable.

Ensure That Everyone Attends the Presentation

Schedule the presentation of the research’s findings far enough in advance to get it on everyone’s calendar, and make it clear that attendance is mandatory. Those who miss the presentation won’t be able to catch up simply by reading it—and usually won’t read a report—so make sure that everyone attends. Otherwise, they’ll miss out on having a deep understanding the research and its recommendations.

Implement the Recommendations

An unfortunate and surprisingly common outcome of user research projects is that the research goes really well and everyone agrees with the findings and recommendations; but after the final presentation, the results just sit on a shelf and nobody ever actually implements them. To avoid this fate, you need to have a plan in place for implementing the recommendations.

After the presentation of the research findings, schedule an additional meeting to review the recommendations, assign priorities to them, estimate the level of effort that implementing them would require, determine which items to focus on implementing first, and assign people to be responsible for implementing those changes. Then, follow up periodically to see which recommendations the development team has implemented.

Conduct Iterative Research

At the end of a research project, don’t assume that you’re finished. You should do user research iteratively, evaluating and validating design changes and improving your understanding of your users. Perform field studies every so often to add to your knowledge of users. Create personas that embody the knowledge that you’ve gained. As you do additional research, add any new information that you discover about users to the personas.

Conclusion

Clients should take an active role in a user research project. User researchers are experts in understanding users, their tasks, and their needs. Clients are experts in their business needs and goals. A successful user research project needs to balance both business and user needs. Clients are instrumental in achieving that balance. When clients adopt the role of an active partner in research, a successful research project is the result.

In User Research

ina rosen

March 15, 2013 12:27 PM

I have done user research for quite a number of years now, and I agree with most of the points.

I think we, as user researchers, need to blur the line a bit concerning the Lab, though. User studies are not, in fact, comparable to lab studies; there are way too many independent variables at play.

I believe that one of the benefits of qualitative research is, in fact, that we can choose to work iteratively within the same project. For example, we can adjust the interview guide or the material and discuss the alterations with users. In many instances, we should learn from previous interviews and bring these learnings into play in subsequent interviews.

I often find that clients can get involved in lab environments, as well as in the field. When the client interacts with her own users, we become facilitators of insight. We help the client to interact and learn from users.

But where we should not blur the line is data analysis. We are the experts when it comes to understanding and interpreting the data; whereas—as you say—the client is an expert in their own line of business. When both areas of expertise meet, we develop the best conclusions and recommendations. But we can’t lose our integrity as researchers. We do the analysis, we discuss the results with the client, then we draw up our conclusions and recommendations, so they are actionable for the client.

Sunday John Ali

June 1, 2015 1:39 PM

The presentation is okay, but is this presentation the same as researcher client interface?

usman hamza

October 23, 2015 2:57 AM

I have a question and will be very glad if anyone of you could be of help. When a researcher becomes a client, what happens? Link this to ethics in designing an experiment.