Top

What Makes a Good Research Insight Great?

June 19, 2017

You’ve just completed a readout of your latest ground-breaking research, presenting an hour-long slideshow, and hopefully, you’ve wowed your audience with what you’ve shown them. But all too often, after you’ve reported your research results, everyone returns to their workspace and develops a serious case of insight amnesia. Stakeholders quickly forget the juicy morsels of information that would make your company’s products better. Your insights remain stuck in your slide deck and may never again see the light of day.

There are two questions that arise from this dilemma: First, how can you make your research insights more readily available to product teams so they don’t have to slog through your deck to find them? There are multiple, well-known solutions to this problem. The second problem, which is the focus of this article, is how can you ensure that your product team uses your research insights?

Sponsor Advertisement
Continue Reading…

When considering the role of user research on a product team, we can sum up what every user researcher strives for in just one word: impact. The question that often looms in a researcher’s mind is: How can I have a real and tangible impact on the direction of the product my team is working on? But it’s not the data from research that ultimately impacts a product. It’s the insights that derive from the data. The ultimate value that user researchers add is their synthesized interpretation of the data they’ve gathered during research sessions. How can a user researcher have a significant impact on a product? What kinds of insights are impactful? What makes a good insight great?

In this article, I’ll describe six characteristics that make a good research insight great:

  1. Grounded in real data
  2. Simple in language and concept
  3. Meaningful and memorable
  4. Speaks to your audience
  5. Inspires clear, direct action
  6. Reinforces ownership and commitment

1. A Great Insight Is Grounded in Real Data, Avoiding Bias and Opinion

Don’t allow bias or opinion to creep into data collection. To better understand the difference between opinion and fact, it is helpful to consider the differences between two key terms that describe part of the data-collection process: evidence and interpretation. According to Jim Kalbach, author of Mapping User Experiences, evidence consists of facts and observations without judgment, while interpretation is an attempt to explain data based on what we’ve observed. Just as lawyers first gather all of the evidence before stating their case, researchers must ensure that their insights are based only on what they saw and heard during the research sessions, not what they felt at the time. So draw your conclusions only from what you actually see and hear. Let your intuition flow from your interpretation of actual data, not something you pull out of thin air. Figure 1 illustrates the relationships between elements that lead to insight generation.

Figure 1—Visualizing elements that generate insights
Visualizing elements that generate insights

Ideally, there will be multiple interpretations of the research data to give a more balanced perspective. In a recent study that I conducted with IT professionals, who implemented a specific type of software for their organizations, participants described having been in their field for twenty or more years. However, when I asked them about their experience in setting up this particular kind of software for their organizations, some indicated they had performed this task for only a couple of years. Because of their lengthy careers in IT, it would have been easy to conclude that these professionals were probably experts at the task they performed during the study. But, in fact, this task was not something most of them had much experience doing.

After hearing and seeing the evidence, I felt it was important to caution stakeholders not to assume that, just because these people have tons of IT experience, they knew how to do the tasks in the study. This insight was grounded in real data: Lots of IT experience did not equate with expertise in setting up the software we were testing. The lesson here is a simple one: Let your senses—your observations—drive your data collection, not your intuition. Save intuition for when you’re synthesizing your interpretations of the data.

2. A Great Insight Is Simple in Language and Concept

A research insight that is devoid of cryptic, academic language and uses a casual tone and straightforward words is more effective. Presenting insights that attempt to boil the ocean by encompassing every single observation is like flogging your stakeholders over the head with a big sledgehammer and leaves them confused, headachy, and looking for a quick escape—a guaranteed prescription for insight amnesia.

Communicating multiple concepts in a single insight will confuse stakeholders and reduce the effectiveness of the insight. Stating individual insights that stand on their own is much more effective. One study that I conducted of how managers receive notifications indicated that they prefer to know what is coming up in their schedules rather than view their past schedules. They receive notifications via various methods, including email, text alerts, and face-to-face interactions with their employees. Each of these concepts has useful design implications. However, putting them together would reduce their impact on the product direction.

An overloaded insight might be: Managers prefer to know what is coming up so they can plan for the future and prefer to receive their notifications about events through email, text alerts, and face-to-face communications with employees. Because this insight communicates more than one key idea, there is a good chance that half of your stakeholders will remember just the first part of the insight, while the other half remembers just the second part. Thus, the insight has only half the impact that two separate insights would have had. Express key concepts as independent insights—especially if each of them has significant design implications.

3. A Great Insight Is Meaningful and Memorable

Using catchy phrases to express your insights can help improve recall among stakeholders when they are trying to validate or refute their ideas.

One study I conducted involved looking at how employees feel about approvals—for example, getting a manager to approve time off work or a company purchase. In this study, the insight I uncovered was that, under certain circumstances—especially those where risk is very low and there is great trust between an employee and a manager—employees never really need to get their manager’s explicit approval. Instead, they might send an Outlook invitation with their name and the subject line Vacation, for example. Even if their manager did not respond, they assumed their request was approved. Instead of stating this insight including all of these details, making it less likely that stakeholders would remember it, I summed up the insight in ten words or less: When it comes to approvals, silence implies consent. This statement was short, catchy, and something stakeholders could remember.

Storytelling is an ancient tool that human beings use to help bring meaning and purpose to their worlds. In user research, insights are really excerpts from a larger story about users. According to Donna Lichaw, author of The User’s Journey: Storymapping Products That People Love, stories go beyond using your sense of vision or hearing, they are about being. An insight that tells a story is the ultimate empathy builder. When stakeholders hear a compelling story, it is as if they are experiencing the pain or joy of the protagonist. Ensuring your insights tell a story will make them more meaningful and memorable to your stakeholders—especially for those who tend to suffer from bouts of insight amnesia. Such stories should help to reinforce the insights product teams need to remember as they think about the user experience of their product.

4. A Great Insight Speaks to Your Audience

A poetry instructor once told me that you’ll know a poem is special when it takes the top of your head off. Such a poem surprises and moves people at the same time. The most effective insights are those that move people in a certain way. They come from answering stakeholders’ burning questions that help shape your core research goals. They motivate stakeholders to take specific actions or change a product team’s mindset about a topic of concern. When an insight is compelling, stakeholders may sometimes even quote your research verbatim.

When stakeholders start quoting your research insights, you’ll know that something has stuck with them. After a recent study to help my team understand how human-resource professionals conduct exit interviews, one of the UX designers came up to me and started talking about one of the key findings from my research. He cited the key finding verbatim, which allowed him to imagine the implications of this insight for his effort to build an optimal experience for exit interviews. The fact that he recalled this information exactly as I wrote it was a testament to the insight's effectiveness. The insight affected him in such a way that he was able to recognize the right implications from the insight right away.

5. A Great Insight Inspires Clear, Direct Action

Insights from research should be actionable for UX designers and product teams. Recently, when iteratively testing a prototype that implemented the setup of a software product for employees’ use, it became very clear that the changes we had made to the prototype during the first iteration were not effective. Using the first iteration of the design, participants had to complete two steps on one screen. One insight from the first round of testing was that participants had failed to notice the second step and, thus, tried to proceed without completing it.

The design recommendation that resulted from this insight was to make the second step more salient, so users would notice it prior to attempting to proceed to the next page. So the designer decided to add a constraint to the page, disabling the Next button until the user had completed the first step. But the insight we gleaned during the second round of testing was that participants were still ignoring the second step and actually tried to click the disabled Next button. While this insight warranted a similar recommendation to make the second step more salient, it also communicated a clear, direct action that allowed the designer to change his approach to fulfilling this recommendation: he moved the second step its own separate page, so it no longer competed for attention with the first step. This insight yielded a clear, direct action that the designer could take, allowing him to iterate until the design achieved its objective.

6. A Great Insight Reinforces Ownership and Commitment

How can user researchers communicate their insights to UX designers and other stakeholders in such a way that they are willing to commit to and follow through on the agreed-on recommendations? The key ingredients of great insights are ownership and commitment. Stakeholders can own insights that generate actionable recommendations, increasing the likelihood that they’ll follow through with solutions. Two psychological principles underlie these two ingredients: the endowment effect and commitment and consistency.

The endowment effect describes the way people tend to place greater value on the things they own than those they don’t own. This effect applies to research insights. The ownership of insights can begin during the research-planning stage. When stakeholders create hypotheses in preparation for a research study, associate each hypothesis with a particular stakeholder. This ownership information should be part of the study guide as long as it remains relevant. When stakeholders feel that they own an idea—or, in this case, a hypothesis—they are more likely to honor their commitment to follow through on any actions their idea generated.

Furthermore, stakeholders will be consistent in their desire to commit to and own an idea, regardless of whether their hypothesis is correct! This is the principle of commitment and consistency. When great research insights either validate or refute a hypothesis, recognize the stakeholder who originated the hypothesis. This recognition gives the stakeholder points for participation, as well as acknowledgment that their idea worked or could use improvement —or perhaps that it inspired additional ideas for new design directions.

Conclusion

Of course, not all insights from user research will meet all six of these criteria. Although research insights won’t necessarily take the top of your head off or be expressed as catchy phrases or meaningful stories, they should always inspire action on the part of your product team or other stakeholders. The goal for research insights may be to make a better product, improve its user experience, or simply change team members’ mindsets by quashing biases and false assumptions about users. 

Senior User Experience Researcher at ADP Innovation Labs

New York, New York, USA

Michael MorganMichael has worked in IT within the financial and business-service industries for almost 20 years. A generalist, Michael worked as a software engineer, product manager, and business analyst prior to beginning his career as a user researcher nine years ago. He conducts research during all phases of the product-development lifecycle and is currently supporting Scrum teams that are building security-management software for ADP’s HR enterprise software products. His true passion is understanding and telling the story of the user to influence product roadmaps. Michael has written blog posts on UX research best practices for his company’s internal blog, as well as for publication on LinkedIn. When he is not working, he enjoys running, music, and reading lots of UX books.  Read More

Other Articles on User Research

New on UXmatters