By the way, this is just a prototype. It doesn’t have the images or text in it yet. No, that’s not Spanish, that’s just placeholder text. Since it’s a prototype, not all the links and functionality work, so at times, I’ll just describe what would normally happen, and you’ll just have to imagine what the site would really do.
Just try to do these tasks as you normally would if you were doing this at home. Except, try to think aloud, telling me, step by step, what you’re trying to do and what you’re thinking. That might be kind of awkward, but if you stop talking, I’ll just keep pestering you to think aloud, or I’ll start asking questions. At some point, it will probably become difficult to perform these tasks, think aloud, and answer my questions all at the same time, but try your best anyway.
Meanwhile, I—and all of those unseen observers that I mentioned—will be watching and listening to you and taking notes. I’ll ask you a lot of questions, but I won’t be able to answer many of your questions. You may not know whether you’ve correctly accomplished the tasks, but I’ll know, and I’ll make note of that. You’ll probably know when you’ve done or said something really interesting, because you’ll notice me furiously jotting down notes.
But remember, this isn’t a test of you. You’re helping us test out the Web site. So just act naturally and show me how you would really do these things, if this were a real Web site, these were tasks you would normally want to do, and you were at home in your normal environment. Any questions?
From the perspective of a participant, user research is not very natural. We ask participants to try to act naturally in the artificial environment of a lab, or we impose ourselves on their environment and hope our presence doesn’t affect their behavior. We often forget how unnatural user research can be and what effect it can have on participants.
Why Does It Matter That User Research Isn’t Natural?
It matters because we want to observe natural behavior, and we want to get participants’ honest feedback. If they feel uncomfortable or as if they are doing something artificial, it affects their behavior and responses.
Experiencing user research from the participant’s perspective can be eye opening. I had my first bad experience as a participant about 20 years ago, long before I became involved in usability. I was traveling to France for the first time and had a layover of a few hours at Heathrow Airport in London. I was young, jetlagged, confused, alone, and very self-conscious about being in Europe for the first time. After I had cleared customs, a woman asked me whether I would participate in a study of what people do while waiting in the terminal. It sounded interesting, so I agreed—not realizing they would give me a very large number to hang around my neck, so they could observe where I went. Of course, I was completely embarrassed and sat, as inconspicuously as possible, reading a book for three hours. The poor design of this observational study completely affected my behavior. I still wonder whether any of the other numbered participants were brave enough to act naturally that day.
What’s Unnatural About User Research?
User research studies aren’t completely natural, but that’s okay. As long as we understand and acknowledge the ways in which user research is unnatural, we can minimize the impact this has on participants’ behavior. In this first part of a two-part Practical Usability series, I’ll examine ways in which user research can be unnatural. Part II will recommend ways of minimizing or eliminating these unnatural aspects of user research, allowing you to observe more natural user behavior.
Recruiting
The most natural user research would take place without participants’ even knowing we were studying them. However, in most cases, this would be highly illegal and unethical unless you were working for an espionage or law enforcement agency. The principles of informed consent require that participants understand your study and consent to participate.
The People We Recruit
The first place unnaturalness creeps into user research is in how well the people we recruit for a study represent the typical user. Unfortunately, typical people don’t usually volunteer for user research. Those who do volunteer are sometimes in it only for the money, or they just enjoy being a participant. Such professional research participants may lie to get through a screener and may be either uninterested or overly eager to please, in the hope that they’ll get invited back for future studies.
The Incentives We Provide
In addition to attracting professional research participants who want to supplement their income by participating in research, incentives influence behavior. Research that has evaluated the effects of providing incentives on user research has shown that incentives cause participants to feel obligated to put more effort into their participation.
Rewarding Good Participants
Well, what’s wrong with getting people to put more effort into a study? People who are natural when thinking aloud and can eloquently express themselves are considered great participants. They provide great sound bites, feature heavily in highlight videos, and get invited back for future research. Observers are more likely to remember them and place more weight on their comments, while dismissing quieter participants as bad or unrepresentative participants.
By screening for outgoing, articulate people, we reward people who we think would be good participants and punish those who would make bad participants. Professional research participants learn this and adapt their behavior, so they’ll get invited back. Because of this, introverted and quiet people are often underrepresented in research—even though they may represent a significant part of the user population.
Where the Research Takes Place
User research can take place either in the lab or in the field.
Lab Studies
We often ask participants to leave the place where they would normally do something we’re studying and come to our artificial usability lab or focus group facility. Why? Usually, we do this for the convenience of the researchers and observers. That’s where we’ve got the recording and observation equipment set up; we can save time and money on travel; everyone can come to one place to observe the research; and we can get the study done more quickly. Meanwhile, the participants go through the inconvenience of taking time off from work or their personal lives to come to us and participate in the study.
Field Studies
Most people feel much more natural and comfortable when being observed at work or at home. They are on their own turf, using their own technology, with their usual tools and artifacts available. Because the purpose of field research is to observe natural behavior, this eliminates most of the artificiality of lab-based research.
However, even field research is not completely natural. It’s not an everyday occurrence to have several people come into your home or workplace who want to observe you, interview you, record what you do and say, and take photos. Your home is a much more personal space than your workplace, and you have a high expectation of privacy there.?With several strangers in their home, we can’t expect people to act the same they would when they’re alone.
