Top

7 Questions to Ask Users Early in the Product Lifecycle

Discovery

Insights from UX research

A column by Michael Morgan
March 20, 2018

Whether you’re a UX designer, product stakeholder, or some other kind of curious-minded product professional, you need to know what makes your users tick. My new column Discovery: Insights from UX research is about unearthing what is already there—just waiting for a UX researcher to discover it.

In Discovery, I’ll explore various approaches to gaining insights about your users by employing UX-research methods early in the product-development process. UX research can help you understand what would make your users’ lives easier.

When you’re asking questions during user-research interviews, the key to getting answers that are not tainted by bias depends on what the question is and how you ask it. Taking the time and thought to pose constructive, reflective questions, ensures that participants can provide the information you need to portray an accurate picture of the customer narrative. Encourage participants to take the time to reflect on your questions and ask you for clarification when necessary.

Supporter Advertisement
Continue Reading…

In this installment of my column, I’ll discuss some of the key questions you really should ask users early on in the product development lifecycle—the opportune time to help drive decision making about whether stakeholders should begin or continue investing resources in a particular product idea. By asking probing questions and getting answers to them early, you can help your organization to focus on developing the ideas that add the most value. Even though some of these questions might seem self-evident, there are times when teams get so wrapped up in a particular aspect of a product, they forget to ask the obvious questions that would help inspire new ideas or iterations to existing products.

Question #1: Do Customers Understand What They Want?

It’s one thing to learn that your customers want something. Whether they really understand what they want is an entirely different question. Understanding why customers want something and whether they really understand what they want are precursors to understanding a product’s adoption. If customers want something, but don’t understand it, they may not actually use it. If they want something and understand it, perhaps they’ll be more likely to use it. According to diffusion researcher Everett Rogers, the greater the complexity of an innovation, the more negatively it impacts that innovation’s rate of adoption. The simpler an innovation and the easier it is to understand, the more likely people are to use it. Therefore, customers’ understanding what your product is and what it enables them do is a critical ingredient of its adoption.

How can you determine whether customers understand your product? You can have them use it. But early on, before there is any product, there really is no way people can use the actual product. This is where user experience research and design come to the rescue!

Build Low-fidelity Prototypes or Storyboards

Low-fidelity prototypes or storyboards can help you assess your potential customers’ understanding of your product or idea. Building prototypes that customers can actually use helps product teams to determine whether customers understand it. Early on in the product development cycle, low fidelity is best, because creating such prototypes requires minimal effort by design teams. Using paper prototypes or simple prototyping tools will get the job done.

If time is of the essence, you can take a step down from prototyping and simply tell a story about a product, describing its use within the context in which product stakeholders believe customers will use it.

For a past study, I constructed a narrative involving a fictional, but true-to-life user that demonstrated the use of the product I was researching. My product stakeholders were especially interested in whether participants truly understood the product. Because we were strapped for time and our designer was not able to put together a series of flows or prototypes in the necessary timeframe, I asked him to portray a sequential narrative, demonstrating the use of the new product through a series of images.

According to UX researcher Whitney Quesenbery, coauthor of Storytelling for User Experience: Crafting Stories for Better Design, visual stories are a powerful way of facilitating an audience’s recognition of an idea, as well as enabling their immediate reaction.

For my study, the designer combined the images and the narrative in a single PDF that provided the stimulus for the study. Rather than my explaining to participants what was going on in the storyboards, we used the storyboards as a means of seeing whether participants understood the product idea. This also allowed participants to react to the details of the story and the use of the product in an unfiltered, unbiased way. We then gathered participants’ impressions and assumptions to measure their comprehension of what they thought the product was doing versus the product teams’ intentions. The storyboard was a quick, cheap way to get feedback without having to go through an entire design cycle that would normally involve constructing a prototype, reviewing it, tweaking it, then reviewing it and tweaking it again, until it was ready for research.

Measuring a user’s comprehension of your idea or product does not have to involve pushing pixels or adjusting colors. Focus on the idea. See if your customers understand it. If they don’t, any additional pixel-pushing might be a waste of time. Creating a low-fidelity prototype or storyboarding a product concept saves time and energy. It gives customers time to reflect on a concept rather than nitpicking over colors and high-fidelity details.

Question #2: Do Customers Want Your Product?

Product managers often assume everyone is going to want to buy their product. They invest countless hours and money in building the product they believe their customers want. But how can product teams really know whether customers want a product? The product might be in a competitive market space, so a product manager might assume that, because customers are buying their competitors’ products, those same customers would also flock to their new product. But what if your competitor has an edge over your product?

Rather than accepting such assumptions, product managers and designers should do UX research that would help them understand whether users would really want what they are building. There are two ways of eliciting customers’ wants: direct and indirect questioning. You can use these approaches separately or in tandem.

Ask Customers Directly

Direct questioning is simply asking: “Do you want this?” But there are drawbacks to asking such questions directly. One is that customers might simply be trying to please you by telling you they want the product, even though they would never really use it.

Participants who receive big incentives for the time they spend participating in your research could be the culprits here. Even though they are receiving their incentive regardless of their responses, they may still feel an obligation to please the moderator. This phenomenon, called the Hawthorne Effect, can trick researchers into thinking customers really want a product when, in fact, they do not. We have been aware of this effect since the 1920s or 1930s, when scientists who were observing employees to analyze their productivity in the workplace realized that worker performance was influenced by more than improved lighting or other environmental factors. The mere fact that employees were being observed made them work harder than if they were not being observed. Eliminating this observation bias is difficult, but not impossible.

At the beginning of every study that I conduct, I remind participants to be candid and honest and tell me what they think, not what they think I want to hear. By telling them this up front, I hope they will feel compelled to tell me what they are feeling when they are feeling it. But, even with this reassurance, there is still the risk that participants will tell you what they think you want to hear. However, at the very least, participants will appreciate your forthrightness and are likely to respect the fact that you are looking for truths—no matter how negative they might be.

Ask Customers Indirectly

If research participants still appear to be telling you what you want to hear even after you’ve asked them directly whether they would want your product, it is important to seek out additional evidence to support their claims through anecdotes. If this evidence does not clearly support participants’ claims or they appear to be floundering when trying to come up with a response, there may be some doubt about whether they would want your idea or product to begin with.

Seek Evidence Through Customer Stories

A discussion about whether a research participant really wants your product does not have to begin with a direct question. It could instead start with a participant’s describing a situation or experience that relates to a product or idea about which you’ve asked an indirect question. Asking participants to reflect upon what they’ve done or how the’ve acted rather than to self-report on whether they want something—without any consideration or hint of the context of use—allows behaviors to surface during the discussion.

In a past exploratory study that I conducted with employees and managers, my product stakeholders wanted to understand the sorts of things for which employees and managers had to seek approval. The research was deliberately broad based, so we could see what employees thought about approvals. We asked questions such as: “Tell me about some of the things you do at work that require the approval of someone else.” The open-ended nature of this question begs an answer that is loaded with actions and situations that support approval-based scenarios. This approach sacrifices directness for a level of detail that a more direct approach could easily miss. For example, “Would you want a product that allows you to ask for approval from someone else?” From the behaviors that participants describe, you can start to glean more insights into how a solution might fit into their lives.

Question #3: How Would Customers Use Your Product?

This question is strikingly similar to the indirect question I previously described that asked participants to describe a situation that pertains to an idea or solution. When asking participants whether they would want a product, always seek out concrete evidence. Asking this question ensures that there is proof of the value—or lack thereof—of an idea or product. Always probe on why and how. Allow participants to tell their own story.

Your ultimate goal is to understand the value of a product and understand what gap it fills for customers. Ignoring how participants would use a product and simply taking their word that they would use it leaves many stakeholder assumptions in place. Don’t stop at: “Sure, I would use it.” Dig deeper into how and why.

Question #4: What Does Your Product or Idea Do for Customers?

This question forces customers to think about how a product affects their life. What work is the product doing for them that would be more difficult if they had to do it some other way or weren’t able to do it at all? For those familiar with the Jobs-to-Be-Done (JTBD) framework, this question will seem familiar. JTBD asks customers what job they’ve hire a product to do. Its goal is also to understand what progress the customer is trying to make by using a product. By examining the jobs a product does, we can better understand the jobs the product might do for the customer.

Question #5: What Struggles Do Customers Experience with the Current Solution?

According to Josh Furnas, Director of Product at Credit Sesame, starting with the customers’ struggles is a very powerful way to unleash a product team’s creativity and find opportunities to innovate. Hearing about customers’ struggles lets teams think more broadly about potential solutions.

There might be ways of using the shortcomings of an existing solution to improve a new product or idea. For example, the existing solution might be lacking in something. The new product can capitalize on that by filling the gap. In the 1980s, the invention of ink-jet cartridges made printing documents and changing cartridges much more convenient for customers. One thing the existing solution had lacked was a reasonable replacement cost. A cheaper alternative to OEM ink cartridges did not yet exist. Third-party ink suppliers got wind of this, saw a gap in OEM ink’s product-pricing strategy, and came out with a solution: refillable ink-jet cartridges. This would significantly reduce the replacement cost of ink cartridges. Rather than spending money on replacement cartridges from OEMs, consumers could buy their cartridges, then when the ink ran out, refill the cartridges themselves. This saved people lots of money—as long as they didn’t mind spending time refilling the cartridges.

Asking customers about their struggles with the current solution gets at the heart of what they are looking for that they’re not getting from their existing solution. This is an invitation to product teams to innovate.

Question #6: What Would the World Look Like If Your Product Didn’t Exist?

This question paints a picture of how a product enhances society and culture. What were people doing before your product came along? Does the product replace a common way of doing something? When email did not yet exist, people used pen and paper to write each other letters or memos. Email made communication faster and cheaper. It changed the way people interact with each other.

Getting a picture of what the world looked like without your product allows your product team to see the product’s immediate value and also take a longer view of its value to its user base. This helps them get out of a solution-centered way of thinking and shifts their mindset to a user-centered one in which goals and needs emerge.

Question #7: What Is Preventing Customers from Using the Product

There are reasons for resisting the adoption of any product on the market. Understanding the barriers to a product’s adoption is one of the first steps to identifying a cure to what ails your product. The product might be inconvenient to use. It might not be easy to understand, making it unusable for its target customers.

On the other hand, captive customers might be forced to use a product as part of their job. Extricating themselves from it might take more time, money, and trouble than it would be worth. The incumbent product might be so ubiquitous that the possibility of adopting any competitive product would be immediately dismissed or quashed. A great example of this is the QWERTY keyboard. Research showed that using it was less efficient than using the Dvorak keyboard. But because the QWERTY keyboard’s adoption had already saturated the market, it was too late for any newcomer to compete. Users would be forced to relearn where the keys were located on any QWERTY competitor. Who wanted to spend the time relearning this?

What would have enabled customers to stop using the QWERTY keyboard and start using the Dvorak keyboard was the time it would take to learn a completely new keyboard layout that would let them perform the rudimentary task of typing. This was a barrier with which most users were unwilling to contend. Had people been able to learn the Dvorak keyboard really quickly, perhaps more people would have been willing to use it. Understanding barriers to adoption ultimately allows product teams to decide whether to fail fast and iterate or pivot. Without this understanding, product teams are driving blind and without purpose.

5 Key Perceived Attributes of Innovations

Pioneer in the research of the diffusion of innovations Everett Rogers in his seminal work, Diffusion of Innovations, highlighted potential barriers to adoption: relative advantage, complexity, compatibility, trial-ability and observability. He says that these five attributes account for most of the variance (49-87%) in the rate of adoption of innovations. He explains that relative advantage and compatibility are particularly important in explaining rates of adoption. A relative advantage refers to what your product does better than another product. Compatibility describes how an innovation fits into the values, past experiences, and needs of potential adopters. Identifying aspects of compatibility and relative advantage can help teams identify their proposed solution’s feature gap and lead to more concrete thinking about feasibility in the long run, so their product can leap past any incumbent solution on the market. These perceived attributes become critical benchmarks that your product needs to achieve to be worth building and selling.

When asking the question about what is preventing your customers from using your product or idea, remember to keep in mind that this question gets to the heart of why your customers would use your product in the first place.

Not All Barriers to Adoption Are Created Equal

When discovering what is preventing customers from using your product, it is important not to assume that all of these barriers have equal weight in terms of their importance to customers. In fact, just one of these could make or break product adoption.

I encountered this issue a few years back, when I was deciding what UX research tools I would use on a new job. One of my all-time favorite tools for usability testing is Techsmith’s Morae. I call Morae “a usability lab in a box.” It not only lets you record video, but also quickly create highlight reels for presentations of your findings. It also lets you add markers anywhere within videos so you can easily find things for later analysis. All of these great features would make most UX researchers who do usability testing long for such a tool. However, I decided to pass on using it. While the relative advantages of features such as quickly creating highlight reels in Morae was huge in comparison to other tools such as Silverback, I was a Mac user and did not want to switch to Windows. Techsmith, the maker of Morae, did not offer a version for Mac OSX. Nor were they ever planning to make a version for Mac OS.

The only way I could simultaneously keep using my Mac and Morae was by installing virtual-machine software with Windows on it. Whenever I needed to use Morae, I would have to boot up Windows on my partitioned disk and use Morae running on the virtual machine. Booting up Windows took large amounts of time and often resulted in a sub-par experience. This was a huge relative disadvantage that I simply could not get past in deciding between moving to a Windows machine or sticking with my Mac. Regardless of all the great relative advantages Morae offered, this one shortcoming mattered the most to me and became a showstopper.

Not only is it important to gain insights from customers about the adoption barriers that prevent them from using your product, it is equally important to understand the relative importance of these adoption barriers.

Conclusion

During research interviews, it is natural for follow-up questions to emerge. This is usually a good sign. Take the chance that something good may come out of a follow-up question. When I hear common themes across the first few interviews, I tend to reflect those themes back to the remaining participants in the form of follow-up questions so I can see whether they agree or disagree with other participants:

“You’re the sixth person I’ve spoken with. One thing I’ve heard from some of the other folks I’ve spoken with is that they felt they would use Product X only when they were by themselves in a private area. Do you agree or disagree with that assertion?”

This practice can substitute for focus-group interactions with all participants in the same room, when you ask them the same question at the same time. What is better about posing a question to one participant at a time is that it eliminates groupthink.

Plus, while a participant may have said something interesting enough to probe further because it could reveal an insight, someone might also take you down a rabbit hole you’ll need to back out of. Perhaps somewhere down the path of questioning lies the seed of a new idea or an improvement. You won’t really know until you go there. Such questions provide good starting points for starting down trails full of fruitful insights. 

References

Furnas, Josh. “Great Innovators Start with Customer Struggles (Not Customer Needs).” Medium, November 30, 2017.

Quesenbery, Whitney, and Kevin Brooks. Storytelling for User Experience: Crafting Stories for Better Design. Brooklyn, New York: Rosenfeld Media, LLC, 2010.

Rogers, Everett. Diffusion of Innovations. New York: Free Press, 2005.

Senior User Experience Researcher at ADP Innovation Labs

New York, New York, USA

Michael MorganMichael has worked in IT within the financial and business-service industries for almost 20 years. A generalist, Michael worked as a software engineer, product manager, and business analyst prior to beginning his career as a user researcher nine years ago. He conducts research during all phases of the product-development lifecycle and is currently supporting Scrum teams that are building security-management software for ADP’s HR enterprise software products. His true passion is understanding and telling the story of the user to influence product roadmaps. Michael has written blog posts on UX research best practices for his company’s internal blog, as well as for publication on LinkedIn. When he is not working, he enjoys running, music, and reading lots of UX books.  Read More

Other Articles on User Research

New on UXmatters