Taming the Elephant in the Room: Brand Perception and Bias

By Demetrius Madrigal

Published: February 8, 2010

“People’s preconceived notions can be another elephant in the room—a barrier to achieving accurate and actionable feedback on a concept or design.”

I’ve been a bit of a pizza snob ever since my first job making pizza at the age of 15. I’ve always had a pretty low opinion of Domino’s Pizza®, so I was very impressed when I saw Domino’s recent efforts to reinvent both their pizza and their brand identity by tackling their problems head on. When I first saw their ads and online video, which you can see in Figure 1, I was amazed that they had chosen to use actual user feedback that denigrated their product. It’s extremely rare to see companies admit their own faults so bluntly and publicly, but the best way to deal with the elephant in the room is to acknowledge it, confront it, and ultimately overcome it. We have yet to see whether Domino’s can successfully overcome their negative brand perception, but the message they’ve sent has been enough to motivate me—someone who had previously sworn never to eat Domino’s Pizza again—to give them another try.

Figure 1—Domino’s brutally honest rebranding effort

Their rebranding effort has brought up quite a few memories for me, because I’ve seen the pain on the faces of designers and engineers when users eviscerate the products they’ve spent months or years developing. I also know convincing stakeholders they need to rethink a product can be a painful and costly endeavor. However, one of my most salient memories involves interacting with research participants who already had their minds made up about the quality of a product before they ever laid eyes on it. People’s preconceived notions can be another elephant in the room—a barrier to achieving accurate and actionable feedback on a concept or design.

The Angry Elephant

“To show these participants we were truly ready to make serious changes, we had to … be brutally honest about the failings of the existing design.”

A short time ago, my business partner, Bryan McClain, and I were giving a talk about communication and research at an ecommerce company, and someone asked a very interesting question: What was our most difficult interaction with a research participant? The question instantly brought to mind this memory: Some time ago, we were asked to conduct a focus-group session with a specialized group of participants. In this particular case, the focus-group participants had previously tested builds of the product, and their feedback had been overwhelmingly negative. Our client, to their credit, had already decided to redesign the product from the ground up and realized they needed direction. The company had chosen to bring in a fresh research team, so they hired our company to engage users, find out exactly what was wrong with the current design, and provide guidance on how they could redesign the product.

I’ll never forget the experience of walking into the room where the focus group was to take place to greet this group of participants and being met with glares bordering on open hostility. We explained that we were performing research the company intended to use to improve the design of the product, but that wasn’t anything they hadn’t heard before, and they were understandably skeptical. Our initial attempts to extract meaningful information fell flat as the participants just spat out short comments, then quickly clammed up. It was obvious we would have to win them over and show them we were on their side. Vague phrases such as we recognize that there are problems with the design were not enough to assuage their doubts. To show these participants we were truly ready to make serious changes, we had to do what Domino’s has done in their recent advertisements: be brutally honest about the failings of the existing design. So, after going through an extemporaneous list of the product’s design shortcomings, we were finally able to show the participants that we understood their pain, cared about their feedback, and were ready to make changes.

This kind of experience interacting with user research participants emphasizes two essential parts of successfully communicating with participants: establishing objectivity and building alliances.

Establishing Objectivity

“When engaging with user research participants, it’s essential that you put some distance between yourself and the product concept or design.”

When engaging with user research participants, it’s essential that you put some distance between yourself and the product concept or design. If you have any biases or a personal connection with the product design, participants may feel uncomfortable giving you negative feedback, because they unconsciously sense your disappointment. For this reason, we commonly advise that designers not lead user research for products they’ve designed. If a dedicated user researcher is not an option, we typically advise companies to have a different designer do the user research—one who has no personal attachment to the project. When we are interacting with participants, very early on in our conversations with them, we make a point of letting them know that we did not design the product, so anything they might say will not hurt our feelings. This gives participants the freedom to offer negative feedback without the social stigma of being mean or hurtful.

Building Alliances

“A user researcher is a user advocate. It is his or her job to prioritize the needs of users, even if it creates additional difficulties for the product team.”

When taking on the role of user researcher—whether you are actually a user researcher or usability professional or are a designer, engineer, product manager, or in an another profession—it is important to take to heart that, for that period of time, you are on the side of the users. A user researcher is a user advocate. It is his or her job to prioritize the needs of users, even if it creates additional difficulties for the product team. In doing this, a researcher builds alliances with participants, so participants feel that the researcher—and thus, the company—values their feedback and will use it to influence the design of the product. While establishing objectivity can help participants feel free to speak openly, building alliances can really motivate them to participate actively in a user research session. It’s another strategy you can use to maximize the amount of actionable user feedback you’ll get from a research session.

The Happy Elephant

“A company that enjoys a strongly positive brand perception may have difficulty getting objective feedback about a product….”

Of course, preconceived notions about the quality of a product can also swing in the other direction. A company that enjoys a strongly positive brand perception may have difficulty getting objective feedback about a product, because participants have already made up their minds to love the product before they even lay eyes on it. We’ve tested products with participants who were brand loyalists and overlooked numerous faults in a product’s design. In one particular instance, a participant had difficulty performing a majority of the tasks during a usability test, but reported that he loved the design, so we shouldn’t change a thing. It was only through our pointing out and inquiring about specific areas of difficulty he had encountered that he finally admitted that there were aspects of the user interface that should change. There are two methods that come to mind for dealing with this type of issue: anonymous testing and prioritizing objective data.

Anonymous Testing

“If it is possible to conceal the identity of the company developing the product during user research, it can be useful to do so.”

The easiest way to overcome such brand bias—whether user research participants have positive or negative perceptions of a company’s brands—is to simply avoid it entirely. If it is possible to conceal the identity of the company developing the product during user research, it can be useful to do so. Often, by not associating a brand with a product or concept, we can acquire a more accurate understanding of users’ reactions to the product itself.

As consultants, we can accomplish this by performing user research under our own company name. We can bring participants to our offices and use prototypes and builds that show no company logos. It can be much more difficult to establish anonymity if we are testing at our clients’ offices, but, of course, it isn’t always necessary to do so. We usually do suggest some type of anonymous testing when we’re working with companies that are trying to overcome a strong negative or positive brand perception. However, anonymous testing isn’t necessary for companies that are less well-known or have a more neutral brand perception.

Prioritizing Objective Data

“In cases where you do encounter real brand advocates, it is important to incorporate objective data into your research.”

Objective data refers to the information user researchers collect through direct observation of participants, while subjective data refers to data researchers collect through participants’ verbal reports. In cases where you do encounter real brand advocates, it is important to incorporate objective data into your research. For example, during a usability test, we note specific instances in which a participant has difficulty with a device or user interface. Then, when later interviewing the participant, we typically make statements like this: It’s great that you loved the Web site, but you seemed to have difficulty doing XYZ. A participant usually responds by acknowledging the difficulty, then making comments about their expectations or suggesting improvements to the user interface.

There are also ways to objectively document emotional responses such as frustration, disgust, and happiness. Learn to recognize such emotional cues, document them, and inquire about them when doing any type of product research, including generative user research such as ethnography. Typically, objective data is more accurate than subjective data, so it is important to prioritize data accordingly when findings are inconsistent.

Conclusion

“Show participants that you are on their side and invested in developing a product that will be both useful and enjoyable for them to use.”

Participant biases, both positive and negative, can get in the way of obtaining useful feedback when performing concept or usability testing. There are a number of ways to work around these biases, as follows:

  • Establish objectivity. Show participants that you are not personally attached to a concept or design, allowing them to provide feedback without hurting your feelings.
  • Build alliances with participants. Show participants that you are on their side and invested in developing a product that will be both useful and enjoyable for them to use.
  • Use anonymous testing. Filter usability testing through a subsidiary or a contracted agency to entirely avoid any brand bias.
  • Prioritize objective data. What users actually do is more important that what they say they do.

If you take these lessons to heart and apply them appropriately, they can be enormously helpful in making some of your more difficult interactions with users much more productive.

4 Comments

Are there ever stages in product development where you would suggest a designer be present during usability testing and feedback? Possibly in the very early stages of product design / development? Might more design involvement be effective if you happen to be working with designers who are reluctant to make product adjustments from the feedback you gathered?

Thanks for the question, Holly. It’s actually great to have a designer present to observe any research that you’re doing. There are a few guidelines for observers though. First, it’s important to make sure that the participant is comfortable. For usability testing, this can mean limiting the number of observers to one or two. Large numbers of observers can add pressure to the session, making the participant feel nervous and affecting his performance while attempting to complete tasks. It could also affect the amount and accuracy of the feedback the participant provides. For example, a nervous participant may tend to clam up rather than providing good feedback.

Also, it can be important to limit the role of designers during the usability tests to passive observers. I’ve seen designers cringe in front of participants while the participants were having difficulty with a task, which would deal a significant blow to objectivity during the study. I’ve also seen designers jump in and help participants perform the task rather than giving them a chance to figure it out on their own.

For discovery research such as contextual interviews or ethnography, there is less need to restrict designers from asking questions or otherwise participating in the research, but it is still important to ensure that the observer is objective during questioning rather than inappropriately attempting to confirm assumptions.

Finally, if you’re doing anonymous testing, it’s important to make sure that any observers are not wearing any clothes or badges with company names or logos.

The suggestion here is that the person conducting the testing should not be directly invested in the product. Yet anyone who is directly invested in the product is likely to benefit from observing usability testing and feedback at all stages of the product design / development process. In a testing facility, both objectives are met with observers behind a one-way glass. The observers can see the testing without influencing the participants behavior or responses. Other options are to use video recorders or a Webcam to capture the session and then designers and others can watch it streamed live or recorded.

Hi Jocelyn,

As you pointed out and as I mentioned earlier, there is great value to having designers and stakeholders observe a session. In my experience, the one-way glass tends to make people more nervous than cameras. People seem to have an easier time getting used to an unmanned camera. I think one-way glass has acquired some negative associations with interrogations and the like on television.

I’d also like to point out that the researcher is not so much not invested in the product, but rather that he or she is not personally invested in a particular iteration of the user interface in the same way that the designer of that interface might be. Ideally, researchers would always be invested in the product and care about how it provides value to users.

Some great points here. I’d like your opinion on how you can actually use brand perception and bias to help the research.

Do you have examples of how preconceptions about a brand can help a brand or product extend further or, in fact, using the extremely negative and extremely positive consumers to aid the innovation / NPD (New Product Development) process?

Thanks,

Andy

Join the Discussion

Asterisks (*) indicate required information.