In this column, I’ll answer these two questions:
- Is it possible to do both design and usability testing effectively?
- If so, how can we test our own designs well?
UX literature is rife with cautionary tales about designers testing their own designs. The objections to doing so typically follow this line of reasoning: Designers are emotionally invested in their designs, they believe in their rightness, and are loath to change them or bear criticism of their baby.
I think this is a specious argument. Certainly, some designers—particularly those of the genius design bent—are willing to brook little criticism of their designs. However, most designers I’ve encountered are more interested in actually solving users’ problems than in maintaining the typically illusory artistic integrity of their designs. So I don’t think most designers are naturally resistant to criticism of their designs, particularly when the criticism originates from the people who are the intended users of the products they’re designing.
When I posted this question on Twitter, Facebook, and my own UsabilityBlog, I received some interesting responses. Here’s a sampling of some the best, most pithy opinions. (Keep in mind that folks who replied via Twitter were limited to 140 characters, while those who replied via Facebook or my own site had a bit more room to expound.)
“Designers can test [their] own stuff. Everyone has bias whether [they’re a] designer or not. [You] just need to be aware of your biases.”—an interaction designer/usability analyst, via Twitter
“[Designers] can effectively perform [usability] testing to gather additional insights and new ideas, but to be fully unbiased and seek out real UX issues, and to do it effectively…it’s a struggle. If you are playing the role of both researcher/analyst and designer, you have to be fully aware, at all times, of how you are forming your conclusions. For example, am I just seeking insights that prove my design solutions? It’s best to partner with an unbiased—yet collaborative—researcher.”—a user experience manager, via Facebook
An experienced user researcher who works for a large developer of both desktop and Web applications had this to say:
“Yes [designers can test their own designs], but they have to be actively trying to ‘dis’ [them].”—via Twitter
A hardware designer and usability analyst expanded on this theme:
“I think it is very challenging for UX [professionals] to objectively test designs they’ve created. While [some] designers are accustomed to having a number of ideas and [going through a] critique process, [others] tend to select one option and focus on it. This generally results in significant bias. I know some companies separate the design folks from the test folks. I suspect many organizations see that as too cost prohibitive these days, though.”—via Facebook
You may have noticed that I’ve been holding back my opinion so far. Well, here it is: I generally agree with the consensus these comments have demonstrated. Regardless of whether we like it or think it’s a good idea, designers will increasingly be testing their own designs.