When Interviews Go Wrong

April 18, 2011

Despite our best efforts to prepare for and run an interview smoothly, there are often challenges that crop up in the heat of the moment. Ideally, our interviewees are cooperative, well motivated, eloquent, knowledgeable, truthful, consistent, concise, precise, and coherent, states Steinar Kvale in his book InterViews: An Introduction to Qualitative Research Interviewing. However, like the typical user, the ideal interviewee does not exist. Some people are harder to interview than others, and sometimes, interviews drift off into unproductive territory due to factors beyond our control.

In this article, I’ll introduce you to six types of people who can potentially jeopardize the quality of the data you’re able to collect during stakeholder and behavioral interviews or usability tests.

Champion Advertisement
Continue Reading…
  • The Chatterbox
  • The Clam
  • The Pollyanna
  • The Unraveler
  • The Distracted & Evasive
  • The Attitude Problem

The Chatterbox: Identifying Characteristics

People who fit the profile of The Chatterbox

  • give lengthy responses to every question?
  • won’t stop talking—either because they’re uncomfortable with silence or unaware they’re rambling
  • go off on irrelevant tangents
  • answer every question with a story, because they enjoy chatting and revealing their whole life story

Problems you might encounter when interviewing The Chatterbox include the following:

  • being unable to cover the entire scope of your questions within the allotted interview time
  • producing irrelevant material, thus making analysis more difficult

What to Do

While it’s initially a relief to find yourself with a talker, it can soon become clear that The Chatterbox won’t necessarily stay on topic. You need to take control of the situation quickly to avoid losing precious minutes. When you’re shifting to a new topic, clearly signal your transitions between questions and tasks. If The Chatterbox continues to wander, smoothly interrupt using his or her name and redirect The Chatterbox back to the question or the point of the interview. However, if The Chatterbox really needs to get something off his or her chest, you should draw out the discussion on that point until The Chatterbox has exhausted it, then close off the discussion explicitly and direct him or her back to the question at hand.

Example Scripts

Here are some tactics you can try to get an interview with The Chatterbox back on track:

  • “Let’s move on to the next question…”
  • “Paul, I’d love to hear more, but I also want to know about…”
  • “Sue, that’s an interesting story, but let’s get back to the question…”
  • “Earlier you mentioned…”
  • “I’d like to go back to…”
  • “Park that thought and tell me more about…”
  • “Okay, I understand your thoughts on that now. What can you tell me about…?”

The Clam: Identifying Characteristics

Interviewees who have the characteristics of The Clam

  • give monosyllabic answers
  • are shy and quiet
  • speak only when you ask them a direct question
  • are nervous, which may make them inarticulate or cagey about what they reveal
  • make you feel like you’re trying to get blood out of a stone

Interviewing The Clam can be problematic, because of the following risks:

  • the danger of collecting insufficient material
  • being unable to cover the entire scope of your questions within the allotted interview time

What to Do

It can be uncomfortable to have to put someone on the spot, but that’s exactly what an interview requires. Try to read whether you or the subject matter is triggering their hesitation, or they just need a little time to formulate their thoughts. If it’s the latter, be patient and endure the silence. Otherwise, check your body language and ensure you’re smiling, nodding, and have a welcoming facial expression. Try asking general, friendly questions to loosen them up. Avoid asking closed questions, and rephrase questions if they don’t seem to be resonating. Follow up on responses and ask The Clam to expand on them.

You may need to reinforce that The Clam’s specific contribution is important to your study and reiterate that the interview sessions are confidential. It can also help to remind interviewees that you didn’t design the material they’re evaluating or reassure them that are there are no right or wrong answers. Sometimes it might be the recording equipment that is the issue, so shifting it out of sight can help them relax.

Ultimately, if The Clam persists in giving short, superficial responses, explain what you need to get out of the interview and ask for what you want.

Example Scripts

Here are some things you might say to draw out The Clam:

  • “Let me put that another way. What if…?”
  • “Don’t hold back. I didn’t design this, so you won’t hurt my feelings if you have negative feedback.”
  • “This is not a test. There really are no right or wrong answers here. Tell me how this design could perform better for you.”
  • “I’ll be talking to a cross-section of our user base about this design, and it’s very important to get lots of different perspectives. Lee, your contribution is going to make this product easier for people to use in the future.”
  • “Help me understand more about X….”
  • “I really need a quotation, summing up your feelings on this issue.”

The Pollyanna: Identifying Characteristics

People who fit the profile of The Pollyanna

  • seem overly optimistic or positive about a product or design
  • are excessively sympathetic and gentle, so not completely candid
  • refuse to acknowledge or comment on design or process flaws
  • seem hypocritical or irrational, because their opinions don’t match the results of their tasks

Interviewing The Pollyanna presents these problems:

  • design flaws get overlooked and don’t get reported
  • negative comments are understated and positive comments are overstated
  • outlier data can skew your overall results

What to Do

Many people want to please others, so they’re not considered negative. Thus, it’s understandable why The Pollyanna might insist on presenting a positive outlook. There are several paths you can take in this situation.

If you’re dealing with people who are pathologically glass half full, you need to convince them that, by commenting frankly, they are doing the right thing by others. By exposing your product’s flaws, they can help make it better in the future. It can also help to remind The Pollyanna that your feelings won’t be hurt by negative comments. The Handbook of Usability Testing, by Jeffrey Rubin, suggests that taking an assertive Devil’s Advocate role, losing your objectivity, and calling them on hypocritical statements, can be appropriate if there is a discrepancy between interviewees’ performance and their remarks. However, this approach requires some sensitivity, so you do not make interviewees feel foolish or judged.

Cultural norms can also dictate that criticism is impolite. If a cultural norm prevents someone from being critical, try activities that ask the interviewee to imagine what someone else might think about the product. Distancing themselves by using a proxy lets Pollyannas be freer in their feedback.

Example Scripts

To get The Pollyanna to give frank and balanced opinions, you can try saying:

  • “So, you don’t have any issues with this design, but what about your mother? What problems might she encounter with it?”
  • “There are no right or wrong answers to these questions. I’m looking for your honest thoughts, so we’ll know what would be good and bad for other users in the future.”
  • “I didn’t design this, so please don’t hold back. I need to report how we could improve this design. We’ll keep your comments confidential.”

From The Handbook of Usability Testing:

  • “It’s interesting that you say you like it, because it seemed to give you a lot of difficulty.”
  • “I’m surprised by your answer. Are you sure you don’t consider this task unusually difficult to perform?”
  • “Don’t you think the firm is doing you a disservice with this design? You’re the first person who has felt this way.”

The Unraveler: Identifying Characteristics

Interviewees who exhibit the traits of The Unraveler

  • seem very anxious to please and to be correct
  • feel like they’re failing when they struggle with a task or can’t easily answer a question
  • get flustered, anxious, and worked up
  • start apologizing and berating themselves

Interviews with The Unraveler are problematic for the following reasons:

  • your need to avoid exposing participants to emotional stress
  • the danger of collecting insufficient material

What to Do

People who fit the description of The Unraveler don’t appear very often, but when they do, it can be quite alarming, and the interview can be hard to rescue. Interviewees should leave a research session feeling good about their contribution and positive about having participated. It’s in both of your best interests to achieve this.

To prevent The Unraveler from manifesting, decide at what point during tasks you should intervene if participants are unsuccessful. This might be after a certain period of time or number of attempts, or you might gauge this by their level of distress. If you already have sufficient data about a certain feature’s performance from other sessions, don’t let them struggle with it unnecessarily; instead, get them to move on to the next topic or task immediately.

If a meltdown is imminent, use The Unraveler’s name to get his or her attention. Offer a glass of water, and give them time to pull themselves together. Have an informal chat. You may need to put your notebook and recording equipment away to ease their apprehension and remove the perception of surveillance.

Example Scripts

To ease The Unraveler’s discomfort, try saying:

  • “Okay, let’s move along. I’ve noted that this definitely needs improvement.”
  • “We’re not testing you, we’re testing the design. We do this kind of research because we believe technology needs to adapt to suit people, not the other way around.”
  • “Let’s take a break. You’re helping us a lot by finding all the weaknesses in this design.”
  • “This is a good moment to pause. Would you like some water?”?

The Distracted & Evasive: Identifying Characteristics

Interviewees whose characteristics match those of The Distracted & Evasive

  • get easily distracted by mobile phones and any computer, TV, or radio
  • respond to interruptions by co-workers, children, pets, and spouse
  • give vague, broad, or superficial answers with few specifics
  • deviate from the topic under discussion

Problems that can result from interviewees’ being distracted and evasive include the following:

  • recording superficial observations rather than revealing consequences for users or core user values
  • the danger of collecting insufficient material
  • being unable to cover the entire scope of your questions within the allotted interview time

What to Do

The Distracted & Evasive come in three guises:

  • people whose mind is simply elsewhere
  • people who can’t answer your questions because they lack subject-matter expertise
  • people who have an agenda and don’t want to answer your questions

Use your instinct to gauge which of these applies to a given interviewee.

For those who just aren’t concentrating, ask them to turn off any distracting technology, close the door, restrain their kids and pets, and do whatever else they can do to eliminate the diversions. Reinforce the fact that you have X questions to cover or X time left, so you need their full concentration to get through the material.

If you suspect interviewees lack the experience or expertise to answer some questions and are trying to appear more knowledgeable than they really are, gently get them to reveal this to you. Then, depending on their level of familiarity with the research topic, as well as the degree of their fit with the recruitment criteria, it might be wise to suspend the session, adjust your lines of questioning toward a novice user, or tap into whatever elements of their perspective might still be relevant to the overall research questions.

With others, it’s a matter of assessing their reliability and determining whether they’re telling the truth, giving sanitized answers, or escaping accountability through doublespeak or passing the buck. By responding with silence or adopting an entirely naïve front, you can make the interviewee’s true motives more transparent and force them to get real.

Example Scripts

  • “Sam, we’re offering $100 for these sessions because your contribution is really important, so we really need your full concentration for the remaining 30 minutes. Can the kids play elsewhere?”
  • “A BlackBerry is hard to ignore, isn’t it? Would you mind switching it to silent for the next while? Then we can zip through the rest of the session and get you back to your day.”
  • “It sounds like your experience with this product is developing. It’s useful for us to get opinions from people who don’t consider themselves experts. Can you help us with that kind of perspective?”
  • “I have to admit that I didn’t follow that. Would you tell me again what this team is responsible for and how this system fits into your process?”

The Attitude Problem: Identifying Characteristics

People who have The Attribute Problem

  • can be arrogant, rude, abrasive, bitter, depressed, ambivalent, or unengaged
  • often don’t want to share information
  • may have a superiority complex

Problems you might encounter when interviewing people with The Attribute Problem include the following:

  • the danger of collecting insufficient material
  • design flaws get overlooked and don’t get reported
  • negative comments are understated and positive comments are overstated
  • they may offend you or get aggressive

What to Do

Rare in consumer sessions, but more common in a business environment, people with The Attitude Problem have political or personal baggage that makes them uncooperative with your research. The first thing to remember is not to take it personally. This interviewee is responding to attitudes or internal conflicts that are beyond your responsibility. Take a breath and remind yourself that you’re unlikely ever to encounter this person again.

If interviewees have a superiority complex, ignore that. The Usability Handbook suggests that rather than being assertive or competing with The Attitude Problem as an expert, you can establish your credibility through your thoughtful lines of questioning, your well-organized and confident approach, and your respectful demeanor.

To get useful information from people with The Attitude Problem, you may need to clarify the goal of the research project, explain the research process, and communicate why they have been selected to participate. Remind them that this is an opportunity to be heard on a topic of mutual interest. If they’re unengaged, identify what they care about, what’s in it for them if the problem gets solved, or how it will impact them if it isn’t solved. Try to detect whether interviewees have The Attitude Problem because they

  • don’t recognize a problem that needs to be solved
  • are not concerned about the problem
  • don’t wish to change or have things change
  • don’t believe change is possible
  • are skeptical about your role’s contribution to the project

If they’re offensive, don’t make effective research participants, or cross the line and become aggressive, keep calm, stay polite, and end the interview. Report the problem to whoever recruited the participant and be specific about the unreasonable behavior that forced you to terminate the session.

Example Scripts

  • “Let’s step away from focusing on X then, Julie. What would you say the main problems are?”
  • “What would it be like for you if that happened?”
  • “How do you feel about this changing? What will this do for you?”
  • “How does this problem affect your work?”
  • “What else would have to happen for this problem to be solved?”
  • “What do you think of the firm’s commitment to effect this change?”
  • “Okay, Pat, let’s finish the interview there. I have all I need from this session. Thanks for your time.”

In Conclusion

Apart from imperiling the quality of your research, these six types of participants can be an enormous drain on your energy. It takes emotional intelligence and on-the-job experience to recognize them and respond appropriately.

For some user research studies, it may become clear that the participant pool is predisposed to include some of these interviewee types. In that case, perform role-plays during your planning stage to prepare you more thoroughly for The Attitude Problems, The Pollyannas, or The Clams that you expect to encounter.

By building your awareness of these six interviewee types, you’ll be better able to recognize their characteristic behaviors when you encounter them and respond accordingly. Now that you’ve armed yourself with appropriate steps to take and scripts to use, you’ll be better equipped to deal with these types of people swiftly and gently. 

Freelance UX Design Consultant at MM Communications

Melbourne, Victoria, Australia

Mia NorthropMia has over 10 years of experience designing digital user experiences. She applies her passion for finding the sweet spot between user needs and business objectives, working for clients as diverse as Medibank, NAB, RMIT, eBay, Telstra, Ford, Merrill Lynch, Coles Group, EMC, and ANZ. Formerly, Mia worked for a who’s who of Australian Internet companies, including Fairfax Digital, Sensis, and SEEK, as well as the interactive agency Razorfish and the consultancy Symplicit. Her favorite part of UX projects is conducting UX research that illuminates the ideas that make the biggest impact on design.  Read More

Other Articles on User Research

New on UXmatters