But some questions are easier for people to answer than others. In this column, I’ll discuss some types of questions people have a hard time answering accurately.
Asking People to Remember the Past in Detail
When you ask someone about their experiences with a product or their opinions about it, they can provide their overall impression and tell you about the types of activities they performed using it, but it’s usually difficult for them to remember specific details. For example, when conducting a usability test for an auto-manufacturer’s Web site, one might ask participants about online research they’ve done recently, as part of their car-buying process. They would probably remember some of the Web sites they visited, the general tasks they performed, and their overall opinions of those sites, but it’s usually much more difficult for them to remember details about the sites. Human memory is fallible and inaccurate. If some participants can remember a few details about a particularly good or bad experience, that’s great. But you shouldn’t push them to try to remember more or you might actually cause them to make up details they can’t recall.
How to Ask Questions About the Past
It’s okay to ask questions about past experiences, as long as you recognize the limitations of people’s memory. Here are some tips for asking questions about the past:
- Ask questions to understand participants’ general experiences and opinions. Don’t expect or press participants for details they can’t provide.
- Ask participants to talk about memorable incidents, but expect that they may not remember much detail.
- Display the Web site, application, or product a participant is talking about. Letting participants walk through a user interface makes it much easier for them to describe what they’re experiencing. They can demonstrate a task or point to something concrete rather than relying completely on memory.
Asking People to Predict the Future
People aren’t very good at predicting what they would or wouldn’t do in the future—or in a hypothetical situation. If you ask them to do this, they may answer your question, but their comments might not accurately reflect what they would actually do. For example, if you ask people whether they would use a new feature, some might say Yes because they wouldn’t want to rule out the possibility that they might actually use it. Others might say No because they can’t yet imagine why they might use it. It’s especially difficult for people to provide an opinion on something that doesn’t yet exist.
How to Learn About the Future
Wait to ask participants for their opinions about specific features or design elements until they’ve tried using them in a prototype. It’s much easier for people to provide opinions about something tangible rather than to comment on an abstract concept. This still won’t predict whether people will use a particular feature in the future, but it can give you a good sense of whether a design is successful.
Discussing Details Out of Context
In traditional user interviews, people can tell you about their tasks in general, but it’s difficult for them to accurately remember each step. People perform many tasks automatically, without thinking about each individual step, which makes it especially difficult for them to talk about their tasks out of context. For example, think about teaching a child how to tie his shoelaces. Could you tell him what to do without having a shoe with which you could demonstrate how to do it?
How to Learn About Users’ Tasks
It’s much easier for people to demonstrate their tasks than to talk about them in the abstract. So visit participants in their natural environment and observe them performing their actual tasks. You can either observe them without interrupting their work, then ask them questions afterward, or you can conduct a contextual inquiry, during which you ask participants to talk about each step as they perform their tasks. The key is to observe people’s natural behavior and get them to talk about their tasks in the context of performing them.