Revisiting Guidelines for Testing Your Own Designs
Guideline 1—When testing your own designs, don’t think of it as a test to pass or fail, think of it as part of your design process.
Whitney Quesenbery aptly recommends viewing usability testing “…not just as a way of seeing if your design works, but as part of the design process. That’s one of the notions behind iterative design. It’s not just that you create design, then test it, then change it, and so on.”
This is excellent advice. Usability testing should not be a stage gate in your design and development process. It should be a tool with which to gather helpful, diagnostic information from your target users. It’s a means of understanding the goodness of a design’s fit to the intended users’ problems. Put another way—if you’re properly incorporating user performance feedback into your ideation, design, and development process, you should never hear the phrase my design failed usability testing. Which leads me to a corollary for Guideline 1:
Guideline 1a—Test early, test as often as possible, and test lo-fi prototypes rather than making usability testing a make-or-break event in your design lifecycle.
Others’ comments on my absolutist stance that designers should always focus on the negative aspects of their own designs drove the emergence of Guideline 2:
Guideline 2—When testing your own designs, you should seek disconfirming evidence, but be alert for joys and delighters, too.
In my last column, I took the position that the invidious effect of confirmatory bias necessitates our always concentrating on what is wrong with our designs. I admit that I oversold the whole confirmatory bias idea—and as a result, I backed myself into a rhetorical corner. Daniel Szuc was kind enough to throw me a rope and pull me out of that corner when he said, “One could be overly negative about just about anything, but the feedback would not necessarily help the design. Just as one could be overly positive, too.”
Susan Hura took a similar position, saying, “I like Paul’s suggestion of orienting yourself toward negative feedback when you test your own designs, but I think it closes you off to some valuable information in usability testing. Focusing on negative feedback doesn’t help you get to a better design solution, especially if the better solution is one you’ve already considered and rejected.”
Dan and Susan are right. A balanced approach is always better, whether it’s you or someone else testing your design. However, this fact remains: People are pretty much hardwired to give preference to evidence that confirms their assertions, attitudes, and belief systems. A design that you’ve created arises from your belief system and represents a value statement that you’ve rendered into a tangible, manipulable form. It’s your baby. So how do you defend against confirmatory bias when it’s your baby that you’re holding up for criticism?