One huge fallacy we sometimes encounter is the belief that some user research is always better than none. Unfortunately, this is completely untrue. Improperly conducted user research can lead to bad decisions about product direction that can result in your inaccurately defining a product’s target market, defining the wrong key functionality for a product, or designing poor user interfaces. Each of these issues is enough to doom a product to failure when you release it to the market.
To compound this problem, often decisions that are based on the findings of user research—regardless of its soundness—receive more trust than they deserve, so they are less likely to be challenged and corrected than if you’d conducted no user research. There are many ways in which user research can go wrong, but we’ll save that for another column. For now, we’ll focus on ways of making good use of unmoderated user research tools.
Beyond Usability Testing
Unmoderated user research tools tend to focus on usability testing, but there’s no reason why you can’t use some of these tools for performing unmoderated concept testing or even miniature ethnography studies. For example, you could construct tasks along the lines of Please demonstrate how you would make a purchase from your favorite online store.
While unmoderated user research does not replace moderated user research, it can be very effective in augmenting moderated user research. For example, generative user research such as ethnography can be extremely costly, but a company can hold down costs by performing ethnographic research with fewer participants, then supplementing their data through unmoderated research sessions.
When performing user research, we look for trends. It’s very important to distinguish between behavioral trends and idiosyncratic behaviors when determining design recommendations. Distinguishing between trends and idiosyncrasies requires many participants—a major factor affecting schedule and budget. Unmoderated user research can be an effective and low-cost method of obtaining the data that lets you make this distinction. You can use moderated sessions to identify and thoroughly understand the behaviors that are of interest. Then, to verify the trends you’ve observed, look for those same behaviors in unmoderated sessions. It’s best to follow this rule: Do not use the unmoderated sessions to identify additional behavioral trends, because the understanding you can glean from an unmoderated session tends to be superficial.
Combining limited ethnographic studies with unmoderated user research isn’t as effective as doing a full ethnographic study, but it is a way for cash-strapped startups to get some invaluable consumer insights. This approach of augmenting your moderated user research by involving larger numbers of participants through unmoderated sessions works with nearly any form of user research.