Table 2.1—Step-by-Step Guide for Handling Any Situation
- Take a moment to evaluate the situation before jumping to action.
- Resolve any threats to physical safety.
- Verify that you’re not causing or magnifying the situation.
- Check the participant’s comfort level.
- Use careful language and tone to probe on the situation and begin to resolve it.
- Regain control to bring the session back on track.
2.1 Take a moment to evaluate the situation before jumping to action
You’re running an interview with a remote participant over the phone. The session seems to be going well until the participant doesn’t respond as quickly to your questions. You hear him whisper to someone in the background, but can’t understand what he says. What, if anything, should you do?
When you encounter anything outside of expected session circumstances, take a moment to step back and evaluate the situation. As the moderator, you keep track of so many things that it can be difficult to take that time. But doing so is vitally important. Why?
- You need to make sure that you understand the cause of the situation so you can address it appropriately. Rushing to do or say something may lead you to inadvertently make the situation worse or further compromise the results of the session.
- The situation may resolve itself. In this remote session example, a few moments may be enough for the participant’s whispering to stop and for you to be able to move on with your questions. Also, consider the example of a usability study participant who becomes confused or frustrated. A delay of a few extra seconds is often enough time for a participant to find his bearings and continue without any intervention from you. This delay may also be all a distracted participant needs to reengage with a task or question on his own.
- If the situation persists, those additional seconds will keep the participant from seeing you sweat—or, in a remote session, hearing you fumble your words as you try to figure out what to do. Remember that, as the moderator, you establish the pace for the session. Take this moment to think through your options and prepare what you’re going to do and say if the situation continues.
This moment may feel like a lifetime to you, but it doesn’t necessarily feel that way to the participant! Even in a potentially life-threatening situation—for example, a fire alarm goes off—take a moment to gather as much information as possible before proceeding. Your need to respond appropriately should outweigh your instinct to react immediately. Remember that you’re responsible for the participant’s well-being as well as your own.
If the situation is especially tense or you’re not quite sure what’s going on, you may want to take your moment in another room, away from the participant. Taking a break is an incredibly powerful tool that we’ll discuss more in section 3.7. A break gives you a chance to think about what’s happening in the session without the pressure of monitoring everything else. If you have observers, you can use this break to check with them and get a “gut-check” on what you think is happening and why. The participant may also benefit from a few minutes to cool off, or deal with his distractions.
Table 2.2 shows examples of what you can say if you need to take a break. Note that you can use these phrases even if they aren’t entirely true—consider them mild deceptions for the greater good. Pretexts are discussed further in the sidebar “Using a Pretext.”
Table 2.2—What to Say: Taking a Break
For a usability study with a multiroom setup:
- “We’re at a good stopping point right now. Let’s take a short break and resume the session in five minutes.” (If appropriate given your setup, you can add: “Feel free to leave the room to get a drink or go to the restroom while I’m gone.”)
For on-site and remote sessions:
- “I’m so sorry, but would you mind if I took a quick five-minute break to get a drink?” or “This seems like a good stopping point. Why don’t we take a quick five-minute break?”
USING A PRETEXT
While you should avoid misleading the participant, there are times when the truth is going to be harmful. For example, if you notice that the participant does not have the characteristics you need and you want to consult with your stakeholders—who are in another room—about what to do, you should not express your concern about his qualifications directly to the participant. Being honest may hurt his feelings or make him nervous about participating. Instead, make an excuse to leave the room.
Technology can be blamed for many things and provides a convenient excuse if you need to leave the room or end the session early. Just make sure that you have something realistic to blame so that your reason for leaving isn’t obvious. For example, if you say, “The computer isn’t working correctly,” the participant may realize that you’re lying because he can see that it actually is working. But if you say, “Our prototype isn’t working the way it’s supposed to,” he does’t have any reason to know that it is just an excuse.
Examples of what you might say include:
- “It seems like we may be having technical issues with our recording equipment. Let’s take a short five-minute break while I run next door to look into it.”
- “I apologize, but I just noticed that there is something wrong with our prototype. Please excuse me for a few minutes while I go to the other room and see if anyone from our team knows what’s going on.”
- “I’m sorry, but it looks like the product I wanted to get your feedback on isn’t working. I have a few more questions for you, but after that, we’ll end the session early. You’ll still get your full compensation though!”
If you don’t feel comfortable blaming the technology or it isn’t applicable—for example, in an interview where you earn’t using any equipment—you can use the study itself as a pretext:
- “I meant to give you your compensation at the start of the session. Let me actually go grab that now. Let’s take a break for a few minutes.”
- “I just noticed that I’m missing a page of my questions. Let’s take a quick break so I can run next door and print that out.”
- “The rest of these tasks/questions earn’t applicable to you/your role, so we’re going to skip them. This means we may be done a little bit earlier than expected, but don’t worry, you’ll still receive your full compensation!”
2.2 Resolve any threats to physical safety
In the middle of a contextual inquiry, you hear a low rumbling sound. You take a moment to evaluate the situation, and quickly realize that you’re experiencing an earthquake. What do you do next? (For more about how to handle earthquakes and other natural disasters, see section 14.2.)
When a situation involves a physical threat, you need to respond to and address the threat as soon as you’ve taken a couple of seconds to understand what’s going on. Respond quickly and appropriately, even if that means that you’re unable to continue the session. The safety of you and the participant takes priority over your data collection needs!
Keep in mind that the participant may not know what to do, especially if he is in your facility. You must take responsibility for his safety as well as your own. For example, if a fire alarm goes off or your building is evacuated, bring the participant with you as you follow standard evacuation procedures. While the participant may end up seeing your observers, your top priority is to keep him from physical harm instead of maintaining the integrity of the session.
In the example of an earthquake, you should instruct the participant about where to go—for example, under a desk or in a doorway. If you’re moderating a usability study from an adjacent control room, and you think it’s safe to spend another couple of seconds using the microphone, tell the participant that you’re experiencing an earthquake and instruct him to go under the desk or in the doorway immediately.
If you’re in the participant’s space, follow his lead to exit the building safely or follow the appropriate safety protocol for the event that you’re experiencing.
Depending on the severity of the situation and safety of the environment, you may want to cancel or end the session rather than asking the participant to wait around until the situation is resolved. If you do this, be sure to give him his full compensation and thank him for the feedback provided. If the environment is safe, and the participant is not visibly shaken and willing to continue, offer to take a short break before resuming so you and the participant can both regain focus.
2.3 Verify that you’re not causing or magnifying the situation
During an interview, the participant starts making snide remarks after each question you ask. You’re not quite sure why he is responding that way. The issue does’t resolve itself, and taking a break does’t help. There is no physical danger to resolve but you’re worried about the participant’s attitude and its effect on the feedback he is providing. What, if anything, should you do?
Let’s step into some awkward territory for a minute, but we promise, it’s for your own good! Sometimes a situation is magnified, if not created, by something that you’re doing. Part of being a good moderator is being humble enough to constantly evaluate and reflect on how you’re interacting with participants. Before you jump to a conclusion about why the participant is behaving in a certain way, try to detect anything you may have done that led to this behavior.
Table 2.3 shows some examples of problematic moderator behavior that we’ve observed, the moderating styles (as identified by Table 1.1) that may encourage this behavior, and the effect it can have on the participant’s comfort level.
|Moderator Behavior (Moderating Style)||How the Participant May React|
Over-the-top fake (Friendly Face)
You’re trying to be very kind and respectful toward the participant while maintaining a friendly tone. However, you don’t realize that you come across as patronizing or disingenuous.
The participant becomes irritated and curt with his feedback. He may feel like you’re not taking his responses seriously.
Buddy-buddy (Friendly Face)
You want the participant to be comfortable, so you start acting like the two of you are old pals. You laugh together and find yourself making jokes. He tries to make you laugh and looks to you for affirmation, as with any natural friendly exchange. For an example of what this may look like, see Figure 2.1.
The participant begins to try to please you, unknowingly reflecting your attitudes and biases. His efforts to please you throw him off his natural course of interaction and keep him from answering questions fully and honestly.
Obliviously abrasive (Down to Business)
You maintain a serious, expectant tone as you ask questions. However, your tone comes across as overly stern. You also stare intently at the participant without looking away or blinking while he talks. For an example of what this may look like, see Figure 2.1.
If the participant is naturally talkative and high profile, he may feel insulted or put off by your serious tone. If he started out nervous, your tone may only increase his discomfort. The intent staring may unnerve him to the point of asking to leave.
Death by silence (Down to Business; By the Book)
You’re trying to be as neutral as possible so you take a very minimalistic approach to moderating. You’re completely silent most of the time, using only brusque probes—for example, “And…?”—even though the participant is obviously nervous and uncomfortable with the silence.
The participant starts to blame himself and apologizes for “doing a bad job.” Because you don’t respond, he becomes more uncomfortable.
Clock watcher (By the Book)
You’re trying to stay on schedule, and look at the clock or your watch at the most inopportune times. You ask a question from your study plan and, while the participant answers, look at the clock, look ahead in the study plan, or obviously pay no attention to the participant’s answer.
The participant gets frustrated and makes sarcastic or snide remarks because he does’t feel like you’re listening to him.
As discussed in Chapter 1, there is a delicate balance between the goals and method of your study, the type of participant you have, his current emotional state, and the appropriate moderating style to use. These last two elements in particular can change multiple times throughout the session. You should always carefully monitor how the participant responds to avoid making him uncomfortable. Again, part of the art of moderating is in evaluating your technique throughout the session and adapting it as necessary.
LEARNING TO ENJOY THE SILENCE
Most moderators are uncomfortable with silence during a study, especially new moderators! You may feel especially uncomfortable if you’re using a think-aloud protocol, since it may seem like the participant forgot your instruction to share what he is looking for or expects to see.
However, silence can be a valuable tool. Sometimes allowing a participant to quietly process and then answer the question or resume thinking aloud is more valuable than probing immediately for his thoughts whenever he becomes silent. For example, most people have difficulty thinking aloud while reading thick blocks of web content, so hold off on your questions while the participant is obviously reading. Similarly, not answering the participant’s question right away can lead the participant to answer it on his own. However, if he shifts toward you, or repeats the question while raising his voice, it’s time to say something.
It takes practice to become comfortable with this approach, but it pays off when the participant does’t feel rushed or harassed by your frequent reminders to talk about what he is thinking.
To find out if you’re creating or contributing to a problem:
- Check your body language:
- Are you too close to or too far from the participant?
- Are you maintaining eye contact, or are you making too much eye contact?
- Have you been yawning or glancing at your watch?
- Are you sitting upright, or are you slouched down in your chair?
- Have you been frowning or grimacing instead of maintaining a friendly, neutral expression?
- Replay what you’ve said during the session:
- Have you neglected to reassure the participant that you’re listening to him? Have you been chiming in with the occasional “mm-hm” to encourage him to continue talking?
- Have you been talking too much, and not letting the participant talk?
- If running a usability study with a think-aloud protocol, did you give the participant enough time to respond before reminding him to think aloud? (See the sidebar “Learning to Enjoy the Silence” for more on this.)
- Did you stop using the participant’s language and start using language from the product’s user interface, domain, or any other jargon he may not understand?
- Are you using close-ended or leading questions that “give away” the answer you want or expect to hear?
- Have you been laughing with or making jokes with the participant?
- Replay your paralanguage—how you said what you said:
- Did you sound judgmental about an opinion or detail that the participant shared?
- Did you insinuate that the participant did something incorrectly?
- Could your tone be heard as patronizing or disingenuous?
- Evaluate your note-taking behavior:
- Are you shuffling papers while the participant attempts tasks or answers your questions?
- Have you been taking notes only when the participant says or does something “wrong” or unexpected?
- Check the physical setup of the room:
- Is the temperature too hot or too cold?
- Is there anything in your equipment setup that the participant seems to be struggling with—for example, a monitor that is pushed too far back on the table, or a chair that is set too high or too low for the participant?
We’ll talk more about how to build awareness of your own behaviors while moderating in Chapter 16.
2.4 Check the participant’s comfort level
Once you’ve verified that the situation you’re seeing hasn’t been caused or exacerbated by your own behavior, check to see if the participant’s degree of comfort has declined to an unacceptable level.
Most user research starts out being somewhat uncomfortable for the participant. He may arrive without knowing exactly what he’ll be asked to do, or he may worry that he is being tested in some way. If you are coming in to his space, he’ll be unable to get away from the typical hassles of his environment, which may become magnified by the uncertainty of what he’ll be asked to do during the session.
Unless you’re doing very specific research that deliberately puts users in uncomfortable situations, you want to minimize, if not altogether eliminate, anything that compromises the comfort or safety of the participant. Part of your job as a moderator is helping the participant feel comfortable participating and providing his feedback, and ensuring that he remains comfortable—physically and psychologically—during the session.