Top

Analyzing and Responding to Initial User Feedback from Focus Groups

July 23, 2018

Many first-time product owners have a hard time responding to and learning from focus groups. They may understandably become frustrated by any negative feedback that their product receives. Or they might overreact to positive feedback, which is not always indicative of a product’s overall success. In this article, I’ll explore how product owners can better learn from and respond to initial user feedback from focus groups. I’ll also touch on how to get the most out of your often limited time with focus groups.

Don’t Bias Your Focus Group Participants

It is critical that you avoid biasing or priming your participants. For example, if your product improves the way small business owners do their accounting and you want to see whether there is a market for your product, do not begin by saying, “Are you often frustrated when doing your business accounting?” Instead, begin by asking broader questions—for example, “What are some of the biggest challenges you face when running your small business?”

Simply observing how often your target customers mention your problem space among their top concerns provides vital information. Once you have acquired preliminary information from your participants, begin asking slightly more specific, but still neutral questions. For example, you could ask, “How are you currently handling your accounting?” Then, follow up by asking, “How is that working out for you? What do you like about the way you are currently doing things and what do you dislike?”

The goal is to get as much unbiased information as possible from your participants before they even know what your product does. In this way, you can avoid priming participants to answer your questions in the way they perceive you’d want them to answer.

Watch and Listen

Once focus-group participants have answered your initial questions and it’s time for them to review the product, always remember to maximize the time you spend listening and observing. If your participants are comfortable talking about what they are seeing and thinking as they go, that is a great way to learn more. Note how participants initially react to your product. When testing a product, see where participants interact with the product and ask them questions such as: “What do you think would happen if you clicked that button?” “Did that do what you expected?” “What are you doing now?” These questions help participants communicate what they are thinking while they are engaging with your product. Once they have become moderately familiar with the product, begin asking them questions such as: “What do you think this product can do?” Follow up by asking, “How do you think you would do [the thing they mentioned the product could do]?”

This approach lets the product speak for itself. It also lets participants provide raw, authentic feedback on what they are seeing and hearing about the product without your having to translate or assist in the process. Once participants have completed their review of the product, ask broader questions about the product, as well as specific questions about their responses to particular features or behaviors that you noticed. These follow-up questions are vital to the next step of the process.

Understand the What and the Why

When the time comes to improve your product using the information you’ve gathered during the focus-group process, you need to understand the why, not just the what of your learnings. For example, it is not enough just to know that someone was confused by a particular workflow or got hung up on a particular screen. You need to know the underlying causes of their difficulty. This is where the questions I mentioned earlier—such as “What did you expect that to do?”—are so important. The answers you get to those questions should guide your solutions to problems.

Without the answers to these questions, you would know only that a problem exists, not why it exists. If you lack this understanding, one potential downfall could be that you make design changes that simply swap one problem for another as you blindly poke at the issue. By asking better informed, more tailored questions, you can take a systematic and iterative approach to solving problems meaningfully.

Don’t Overreact to Every Bit of Feedback

When evaluating the results from your focus groups, remember the adage “the plural of anecdote is not data.” As a product owner, you should respond to focus-group feedback holistically. It is unwise—and often impossible—to address every individual concern, question, or request that participants have proposed during the focus-group process. Plus, you need to remain grounded by your original vision and intent for the product. Do not forget why you’re building this product in the midst of everything else you’ve learned from users. By keeping your focus and vision and discerning the why of user problems, you’ll be ready to begin the process of improving your product.

Solve the Problems

When addressing the problems you need to solve, remember that, while you are the product owner, your participants were potential users. It is their job to present problems and explain them to the best of their ability. During that process, they might suggest changes or solutions. You should not ignore participants’ suggestions, but never blindly accept them as the final solution. Instead, focus on defining the problems clearly and explore all potential solutions. Dream bigger than the immediate solution that came out of the focus group. While it is entirely possible that their initial solution might end up being the best approach, never limit yourself by immediately accepting it.

Another possible approach to solving the problems the focus group identified is using problem-free areas of your product as design guides. See whether you can apply existing solutions or design patterns to the newly identified problems. This approach also offers the added benefit of further unifying your solution’s behaviors, language, and design.

Conclusion

Conducting focus groups can be extremely informative and meaningfully shape what your product becomes. However, if done incorrectly, focus groups can be confusing, frustrating, and potentially damage your final product design. By following the approach and the tips I’ve presented in this article, you’ll be able to facilitate your next focus group more effectively and better evaluate its results. 

Partner & Product Manager at The BHW Group

Austin, Texas, USA

Paul FrancisAt The BHW Group, an Austin-based mobile app development company, Paul has led dozens of projects, including the development of several of BHW’s flagship applications. His primary responsibility is advocating for app users at every step in BHW’s design and development process.  Read More

Other Articles on User Feedback

New on UXmatters