Finding Gold in Your User Research Results
Published: July 6, 2009
As my company planned for a recent study, we noticed we were asking more questions about the product to help us determine how our research could drive key business results. For example, we explored product descriptions, market positioning, selling points, key differentiators, and other critical areas that might require deeper exploration with users. So, I started asking myself:
- Was I undergoing some sort of transformation?
- Was I caring less about the users and more about the business?
- Was I trying to think like a Product Manager?
Perhaps a little of all three of these is true.
Therefore, I’ve started writing a due diligence document that we can use to help drive toward better user research results. Some of the goals we want to embrace include the following:
- To think not solely of our own role on a project, but how, over time, we can learn to achieve and sustain a holistic product perspective, both for ourselves and the teams we work with.
- To obtain better user research results that will let us improve the user experience.
- To conduct our business and user research in a way that gets our product teams to trust and respect user research.
- To potentially get results that open, create, and confirm opportunities for improvement or innovation.
This article describes a funnel that starts with the product, progresses to the users, and finally, plumbs the depths of the user research itself. I’ll attempt to show how each of these stages can inform the next stage and move us toward finding the gold.
Stage 1: The Product
When devising a research plan, what is your research goal? Much of the answer depends on exactly what you want to discover—and some of this comes back to your perspective and what you think your job is. Two goals we all share are to improve the product and understand how the product fits into an overall product strategy. This means we need to ask the right questions up front—even before we start to think about what we want to discover from users through our research.
We need to frame the questions in both the language of the business and language of the product. This is important, because doing this puts us on the same footing and lets us use the same language as the people who are funding, leading, marketing, and otherwise driving the product forward. Thus, our first goal is to find out more about the product and the process of the product team with which we are working. Some questions we might ask include the following:
- What are the business goals?
- What does the product do?
- Is this a new product or does it replace an existing product?
- What are the product goals?
- What are the critical tasks or key user journeys for the product?
- What will make or break the product in terms of market success?
- What is the timing for product delivery?
- What are you looking to achieve in this version of the product?
- What does the product do well?
- What aspects of the product give users the most pain right now?
- Where do you think the product needs improvement?
- What are the internal and external product competitors? Does the product have competitors inside the company or just outside the company?
- How does the product fit within an overall set of products or broader product strategy? Is there a product strategy?
- Who makes up the product team? How many people are on the team?
- If it’s an existing product, what makes the product successful?
- What are the standout features of the product? What makes it easy for salespeople to sell it?
- What data is available that can tell you more about the product—for example, analytics, sales, or customer call center data?
- Are there any marketing materials available? How does the company present the product on the Web, through brochures, and in retail stores, if relevant.
- Can you take the product home with you and try it out or play with a demo?
Can You Get All the Answers?
You may be surprised to learn that the product team does not have all the answers to your questions. Usually, there are only pieces of a product strategy in place. So, asking these questions provides a good opportunity to revisit current product thinking and get everyone on the same page.
You’ve probably noticed, in reading these questions, that we have not yet spoken about users or used terms like usability, interaction design, information architecture, user experience, customer experience, or other UX jargon. This is deliberate. We want to give the people we’re speaking with an opportunity to stand outside their own silos, think about the product from a strategic perspective, and understand what the whole team must do to achieve its strategic goals—the effort it will take on the part of all the disciplines to deliver a great product experience.
You may also have noticed that we’ve asked no hardcore technology questions at this stage. Of course, technology is important, because it usually underpins the products we’re designing. However, you want to be careful not to get trapped in the rabbit warren of discussing technology opportunities and constraints. Often, the starter questions I’ve outlined will reveal answers about the technology anyway and also give you a hint about whether the engineering team is onboard to help you realize the product changes that are necessary to achieving a great user experience.
Have You Done Your Homework?
Before entering Stage 1, it is important to do your homework. For example, prior to meeting with your product team, for a product that’s already in the marketplace, have you looked at the product? Have you taken the time to visit a store that carries it or to watch someone using it? Have you given yourself an understanding of the necessary consumer context to take into your product discovery?
Establishing a Common Language
The list of questions I outlined earlier is by no means exhaustive, but these questions start a conversation with your product team that is based on a common language—the language of what it will take to make the product successful. As you plan your user research, they also provide you with a product history, so when you run the research, you can put on a Product Manager hat and think about how to improve the product from that point of view. Doing this shows your research is not just about finding usability issues, solving design issues, or finishing your research on time. Instead, you are helping guide your product team toward a vision. You can think about the product in holistic terms, through the eyes of your various team members’ professional disciplines and roles, and in a way that helps everyone have the product’s success and customers’ best interests at heart.
Stage 2: The Users
Now that you have a better understanding of the product and how it fits into the overall business strategy, it’s time to find out more about how you can develop a research action plan to discover insights that will help you to improve the product going forward. As you plan, it helps to think about chunks of research—that is, breaking up your research into pieces that move in sync with the product rollout. It’s time to take what you’ve learned from your first team meeting and plug it into your research plan. Your second goal is to find out what to focus on first:
- What tasks or user journeys do you need to address?
- Who are the primary user groups? Can you provide descriptions of the people using the product?
- Who can you talk to who can tell you more about the users and their product usage?
- Has your company conducted any previous market or user research to help them better understand both where users are facing problems and what they like about the product?
- What are the user tasks? It’s time to delve a little deeper into what users want to do with the product to help you craft a discussion guide.
- What are the success points for specific tasks?
- How does each task map to product or business goals?
- How many users do you need to speak to?
- How much time do you have?
- What do you need to prove during this research phase?
Remember, this phase is about scope and timing. What do you need to do at this stage to show that your research insights are providing value and improving the product going forward? What do you need to do to start planning for the next two or three phases of research?
Have you ever heard someone say that they want to make sure the “results are statistically significant”—that is, make sure you involve enough users to ensure they can have confidence in the results? Hearing this always worries me, because it suggests more education is required. The team may be relying too heavily on one round of research to provide all of life’s answers and more. Perhaps the team does not have a continuous research plan in place that lets you discover ongoing insights to improve your product. Again, this is an opportunity that affects scope.
Stage 3: The User Research
At this point, in addition to your having a better understanding of the product and how it fits into the overall business strategy, as well as a better understanding of your users, their tasks, and how those tasks map to product goals, you now need to find out more about how you can iterate your research plan to help you find insights when you actually conduct the research. You’ve completed your discussion guide, you know your research goals, and you are immersed in the research. So, your third goal is to look for the gold:
- What are the product goals?
- What are the tasks?
Now ask yourself these questions:
- What are users trying to do?
- Have users repeated any behaviors?
- Are you seeing anything new?
- Do your observations provide any opportunities for quick design recommendations?
- What attitudes are you repeatedly seeing in regard to the product?
- Does any function, screen, or design you are presenting fail to meet users’ specific expectations? Why?
- Is there a design roadblock you need to fix?
- Are there critical moments or user quotations worth noting?
- Are there observations that relate to the product framework or family of products?
- Are there opportunities to update a design pattern based on your observations?
- Do you need to repeat some element of your current research in future research?
- Is there something specific to a particular user that makes him or her use the product differently from what you’ve seen previously?
These are not questions you should keep beside you while you run your research. (Although you could print them out if you like.) These are questions to keep in the back of your mind as you run your research—like a background process—to help guide you toward finding insights. Also, be flexible and open to opportunities to explore something new that you did not include in the discussion guide, but may have important design consequences. The key here is to identify patterns that may lead to key recommendations to help improve the product design. I repeat, it’s about discovering key recommendations that improve the product design. It’s not about the number of issues logged.
Are You Thinking Like a Product Visionary?
All UX professionals want to do a good job. Part of doing a good job involves
- choosing the right research method at the right time
- understanding what it takes to make a product design successful
- understanding how your product fits into a larger product and business strategy
- building on research success and showing your value
- making everyone on the product team own the research outcomes
- bridging research results back into improving the design
- ensuring your research results get implemented and improve products
But doing a great job is not just about delivering on your research goals—although doing that is important, too. Doing a great job, playing a significant role in your company’s success, and providing ongoing value is about delivering great user experiences. It’s about how your work can add real value for both the business and the people who use your products. It’s about thinking ahead—planning how to present your research results even while you are immersed in the research. It’s about how you can keep design in focus. All of this and more can help you find gold in your research results.
Baty, Steve. “Patterns in UX Research.” UXmatters, February 23, 2009. Retrieved July 4, 2009.
Ellerby, Lindsay. “Analysis, Plus Synthesis: Turning Data into Insights.” UXmatters, April 27, 2009. Retrieved July 4, 2009.
Goel, Vindu. “How Google Decides to Pull the Plug.” New York Times, February 14, 2009. Retrieved July 4, 2009.
Lu, Cindy. “Value.” Apogee, July 28, 2009. Retrieved July 4, 2009.
Merholtz, Peter. “Is Your Company Designed for Humans?” Harvard Business Publishing, March 25, 2009. Retrieved July 4, 2009.
Quesenbery, Whitney. “Choosing the Right Usability Technique: Getting the Answers You Need.” User Friendly 2008, October 2008. Retrieved July 4, 2009.
Spool, Jared. “The 3 Steps for Creating an Experience Vision.” UIE, May 14, 2007. Retrieved July 4, 2009.
Szuc, Daniel, Paul Sherman, and John Rhodes. “Selling UX.” UXmatters, October 20, 2009. Retrieved July 4, 2009.
Szuc, Daniel. “Selling What We Do.” Johnny Holland, May 21, 2009. Retrieved July 4, 2009.
—— “Walking Through Your Product Design with Stakeholders.” UXmatters, June 4, 2009. Retrieved July 4, 2009.
Zheng, Liya, and Jeanine Harriman. “Designing for Positive Impact.” Apogee, March 18, 2009. Retrieved July 4, 2009.