Testing Low-Fidelity Versus High-Fidelity Mockups

Understanding the User’s Mental Model

“If by low-fidelity sketches you mean wireframes or paper—as opposed to creating mockups using a prototyping tool such as Axure—the differences come down to the logistics of how to prepare and engage with research participants and how to set goals for the product team that maximize the benefits of testing with low-fidelity prototypes,” answers Carol.

“Preparing for test sessions means understanding how the moderator will work with the prototypes. If they are paper, will there be a Wizard of Oz who responds to whatever actions the user takes—for example, pointing to a tab she’d click, causing the Wizard to produce the next screen or paper? If the prototype consists of linked wireframes or PowerPoint slides, does the moderator have a clear understanding of the sequence of frames or slides?

“In either case, the moderator must be very familiar with the prototype and be able to engage with participants in ways that facilitate learning what they want to do and why. Also, the moderator must prepare the participant properly to ensure he feels comfortable with what he is doing. For instance, the moderator might tell the participant, ‘Some elements are interactive, while others are not. Whatever you tap or indicate you would want to tap is helpful to us because it shows us what makes sense to you and how you want the user interface to work.’

“This brings me to the second difference between low-fidelity and high-fidelity prototype testing, which involves setting goals with the development team to maximize the benefits of testing with low-fidelity prototypes,” continues Carol. “That means communicating to the team what each type of testing is good for. If what you’re testing is truly a low-fidelity prototype, it’s good for understanding users’ mental models of where they would expect to find things, as well as assessing the clarity of terminology and iconography by learning from users whether the words and images in the user interface carry the intended meaning. For a truly low-fidelity prototype, matters of graphic or visual design do not come into play. When users are not influenced by the visual design, they can focus on the interactions. Later iterations of the product design can layer in other design elements, and you can then test their influence on user engagement.

“Probably the best thing that results from low-fidelity prototype testing is that the team gets early feedback from users as part of an iterative design process. However, despite these benefits of testing early prototypes, I find that this kind of testing is a hard sell for clients and developers. Because of the availability of sophisticated prototyping tools such as Axure that let designers mock up a user-interface design in a more realistic way, developers typically want to jump right into high-fidelity prototyping. Nevertheless, I try to make the best case I can to convince them of the benefits of taking a smaller first step, by emphasizing what low-fidelity prototype testing provides.”

How Useful Are Low-Fidelity Prototypes?

“Eighty percent of projects won’t benefit at all from early testing of a low-fidelity prototype,” asserts Jordan. “The other twenty percent that would are usually innovative in some way—for example, introducing a never-before-seen user interface or interaction design or perhaps an innovative information architecture. The best UX professionals have enough experience and imagination to avoid low-fidelity testing in most cases. That said, there is a spectrum of high-fidelity prototypes that can benefit from insightful testing.

Static mockups or wireframes are the lowest fidelity of the high-fidelity prototypes. A fully interactive prototype is the highest fidelity. Testing of these two very different types of high-fidelity prototypes can be very different, depending on the specific goals you hope testing will achieve. For instance, many usability tests focus on problem discovery—identifying points where users struggle. Depending on the product you’re testing, it might be more appropriate to establish essential benchmarks or usage baselines. For other products, you might be more concerned about how learnable a product is. Each of these goals would likely require a different testing approach and methodology.”

Consider the Interactivity of the Mockups

“There are two dimensions to this question,” notes Cory. “First is the fidelity, but second is the interactivity of the sketches. In other words, whether you’re testing low-fidelity sketches or high-fidelity mockups, does clicking certain hotspots advance the user from one image to the next?”

“You can test a low-fidelity sketch that is not interactive in the same way you would test a high-fidelity, non-interactive mockup, with a first-click approach, in which you give participants a task and ask them to show you how they would use a site or app to complete that task. You then record the first place each participant clicks. You can do this either manually, in real-time, simply by watching where participants click, then debriefing them either between tasks or after a test session is over, depending on what is necessary to avoid biasing future tasks. Or you can do this asynchronously, using something like Optimal Workshop’s Chalkmark.

“If you’ve specified hotspots in the low-fidelity sketches, so clicking certain areas advances participants to other pages, you can test these as in any usability test—as long as you’ve set participant expectations appropriately. Inform participants that they are looking at low-fidelity sketches and, while some things are clickable, others are dead ends. To prevent your biasing participants to look only for hotspots, tell them that they may hit a dead end at any point and, when they do, they should let the facilitator know what they would expect to see from that point on.”

Test Both Low- and High-Fidelity Designs If You Can

“At Bold Insight, we have tested using a full range of fidelity—from fully working prototypes to user interfaces that are hot off the presses,” replies Gavin. “In the latter case, if a participant said, ‘I would click here,’ the door to the observation room would open and someone would hand a newly printed piece of paper to the moderator, indicating what would happen if the user clicked. Talk about low-fidelity, rapid prototyping! There’s a reason for testing at different fidelities. Most think the development timeline is the reason for low- versus high-fidelity prototyping. But this is wrong. Those who would love to make higher fidelity prototypes earlier are the ones who sheepishly show early designs, expressing that, if they had more time, their designs would have better fit and finish. This is what should not happen.

“I believe in the Bill Buxton philosophy. Bill is the author of Sketching User Experiences: Getting the Design Right and the Right Design. He believes that low fidelity is really important at the early stages when the design is most formative. But low-fidelity prototypes must look like sketches because they’ll elicit more user feedback than beautiful, high-fidelity masquerading as low fidelity. He argues against creating a prototype that is higher fidelity than necessary because people tend not to comment on things that look final. If a design is in a formative stage, make it look formative! More than just time dictates the right fidelity. The fidelity should align with your confidence in a design—so early stage is low fidelity and late stage is high fidelity.”

Conclusion

I agree with Carol and Gavin that ideally it would be great to test both low-fidelity and high-fidelity mockups, if possible. As Greg Nudelman discussed in his book $1 Prototype, a very important benefit of testing low-fidelity prototypes is that users, designers, and developers are less likely to fall in love with a design or become committed to a design too early because of all the effort they have put into creating it and are more likely to iterate and create a better, significantly different design solution. (See my UXmatters article “Book Review: $1 Prototype: A Modern Approach to Mobile UX Design and Rapid Innovation.”) It is human nature to protect something in which we have invested appreciable time and effort, but that natural behavior can work against us when we are trying to create the best UX designs possible and what we really need to do is to throw out a design and start over.

Another benefit of testing low-fidelity designs is that you can iterate new, better designs more quickly. Ideally, you would create and test a series of low-fidelity mockups to work progressively toward a high-quality design, then create high-fidelity mockups when finalizing the design.

The key word here is ideally. If you can test a series of low-fidelity, then high-fidelity mockups, great! This will help you to create strong UX designs. Unfortunately, in practice, many UX designers won’t have the luxury of creating and testing a series of mockups. Some will be lucky to create a single mockup! If you can test only one or two mockups, consider your goals as a UX designer with respect to your company. For example, are you being asked to create a great user experience or build a decent design quickly? Quality or speed? What does your company prefer? If your priority is to create a high-quality design, at the beginning of a project, communicate that you’ll need to create more than one or two mockups to achieve the best design. Of course, the company might not be willing or able to give you the opportunity to create more mockups. If not, building a high-quality prototype that best shows how the final application should appear and behave—one that you can more easily hand off to the development team—might be the best approach. Or, if you are under extreme time pressure, creating a well thought out, low-fidelity prototype might be the better way.