Are You Still Using Earlier-Generation Prototyping Tools?
Published: October 8, 2012
Working as a UX design and user-centered design (UCD) consultant across the world, it continues to surprise me that many software development organizations are still using what I call second- or third-generation prototyping tools and their associated techniques. Despite the fact that fourth-generation prototyping tools are now well established and the organizations that are in the know have gained a significant competitive advantage by using such tools, many organizations have yet to adopt them.
To help you understand the various generations of prototyping tools, I’ll first define a set of criteria for evaluating all prototyping tools, then assess each generation of prototyping tools and their associated prototyping techniques against those criteria. I’ll also describe why each generation of tools arose and why designers continue to use them.
Criteria for Evaluating Prototyping Tools
The criteria that I’ve defined for evaluating prototyping tools comprise three subsets:
- disposability criteria—This first set of criteria concerns the disposability of the prototypes that the tools produce. In other words, how much of a prototype can you use in the actual software implementation.
- management, usability testing, design, and specification criteria—This second set of criteria, which I call MUDS criteria, concerns the degree to which the tools support the key elements of the prototyping activity.
- benefits, limitations, and usages criteria—This third and final set of criteria, which I call BLU criteria, summarizes the key benefits and limitations of the prototyping tools, along with their most appropriate usages.
The MUDS Criteria: Evaluating Tools’ Support for Specific Prototyping Activities
Let’s look at the individual MUDS criteria in more detail. You should assess these criteria as follows:
- management criteria—How well does a tool support collaborative work on prototypes and revision control? Does it provide facilities for distributing prototypes and gaining feedback from stakeholders?
- usability testing criteria—How easy is it to conduct usability tests on prototypes? Does a tool offer any features whose specific purpose is to aid usability testing?
- design criteria—How complex and sophisticated can a prototype be—that is, to what degree can it be interactive and how rich can its functionality be? Can a tool produce high-fidelity—even pixel perfect—designs of screens?
- specification criteria—How well does a tool support the documentation of a prototype’s functionality for development purposes—that is, the software product’s architecture and coding and the functional testing of the final implementation?
Note—Although it may be tempting to think that, as the software development industry generally moves away from waterfall development methods and toward agile methods, specifications are becoming redundant, this is not the case! Agile methods seek to reduce the amount of documentation, not to eliminate it altogether. Developers still need a lot of information—in addition to prototypes that show what screens look like—to define the goals for an agile sprint. For example, they might need the hex values for colors and the contents of drop-down lists.
However, agile methods do qualitatively change the nature of design documentation for software development projects. Rather than thick, beautifully presented requirements and specifications documents, which can be very time-consuming to create and update, development teams need just enough information to get their job done. A UX designer needs to be able to quickly and easily create and iterate a design and update its documentation.
In evaluating prototyping tools against the MUDS criteria, it is useful to think in terms of a simple four-way radar diagram like that in Figure 1, which shows this set of criteria in balance in a perfect prototyping tool.
Figure 1—MUDS criteria in balance
Four Generations of Prototyping Tools
Now, let’s use these criteria to better understand the four generations of prototyping tools and their associated prototyping methods.
The First Generation
Contrary to what some may believe, prototyping software is not a particularly new idea. Almost from the beginning, software product teams wanted to know what the screens of the applications they wanted to develop would look like. They soon learned that if they passed a load of diagrams and text to computer programmers in a requirements document and just let them get on with it, the screens that got developed were often very different from what the stakeholders had envisioned!
As a consequence, some enlightened teams concluded that it would be a good idea to plan what screens would look like before actually building the working software, so they looked for some suitable tools with which to do this. Fortunately, there was a pretty good one that was literally available off the shelf and at a low cost: pen and paper.
Not long after, software designers came up with some formal methods of paper prototyping. In particular, Belcore gave us the Plastic Interface for Collaborative Technology Initiative Through Video Exploration, or PICTIVE for short.  We also got a C.A.R.D. game for participatory task analysis and redesign to go with PICTIVE. 
So how does paper prototyping stack up against our evaluation criteria?
This is easy. Paper prototypes are 100% disposable! Ultimately, you have to communicate everything in the system design documentation in some way.
Let’s evaluate paper prototyping using the MUDS criteria. As Figure 2 and the following show, paper prototyping is very weak in this regard:
- management—Paper prototyping is difficult to manage. For example, if using PICTIVE, facilitating sessions is challenging because it’s all about managing people—who are far more complex than any computer! Likewise, documenting the process and organizing its output is not easy, because you are dealing with lots of quite unstructured paper documents.
- usability testing—An experienced usability test facilitator can get some very useful findings when testing with paper prototypes. However, there is, of course, no interactivity, which presents some significant limitations.
- design—You are basically restricted to conceptual design work.
- specification—The best you can realistically do is to scan your paper prototypes, then copy them into an application for annotation.
Figure 2—Evaluation of paper prototyping using the MUDS criteria
Now, let’s look at the benefits, limitations, and usages of paper prototypes.
The following benefits of paper prototyping are easy to understand:
- Creating paper prototypes is cheap and fast.
- The process of creating them is highly interactive and collaborative—as long as everyone is in the same room, of course.
- Iterating design concepts is quick and easy.
- Making group decisions can happen very quickly.
A fundamental feature of real user-centered design—which, in my experience, is not commonly realized—is that it is multidisciplinary. Indeed, UCD seeks to engage all project stakeholders at an early stage, including users, customers, domain experts, and project owners or sponsors. Within the context of UCD, paper prototyping has the important benefit that it reduces the dominance of software development professionals—including UX designers—over other stakeholders who are involved during the early parts of a design process when key decisions get made.
The MUDS criteria tell us that paper prototyping has the following serious limitations:
- It’s difficult to prototype large or complex systems.
- Specifying designs—even capturing basic designs and the design process that led to them—can be challenging.
- The prototypes have no interactivity.
Paper prototyping has served us well and continues to do so. I always kick off my ideation and conceptual design work with a PICTIVE session, and I encourage everyone to participate. Indeed, I see it as an integral and necessary activity of real user-centered design.
The Second Generation
The serious limitations of paper prototyping soon became a barrier to progress, and it wasn’t long before product teams wanted prototypes that looked more realistic and had greater interactivity. Again, software designers looked for some tools with which to create them, and they were again easy to find. Visual designers—and also some UX designers—used graphic design packages like Photoshop to design screen images. Then, they sliced and diced up the images and added them to HTML pages, using an HTML editor such as Dreamweaver, which provided some interactivity. That’s why I call this the slice-and-dice generation of prototyping tools.
Are slice-and-dice prototypes disposable? This is a complex question. In theory, it is possible to iterate both the screen images and the associated HTML, so the prototype eventually evolves into working front-end code. This is non-disposable prototyping, and examples of this are not uncommon. (I’ve done this several times myself.) However, in reality, many slice-and-dice prototypes get completely junked. The following are reasons for this:
- The target development environment might not be HTML—for example, the system a team is developing may be a desktop or mobile application rather than a Web site or application.
- The UX team that produces the prototype might not write the code in a way that the development team considers correct or even requires. Sadly, I’ve often seen technically competent UX designers spend weeks coding their prototypes to a commercial standard and proudly present them as final front-end code, only to have the development manager immediately ask developers to rewrite all of the code—because it wasn’t compliant with the chosen standards or wouldn’t have worked well with the middleware or back-end code libraries they were using in the final implementation.
- Developers generally see front-end coding as their turf and often rewrite a UX designer’s code just on that principle. This is an organizational reality!
In summary, although the slice-and-dice approach has potential for the creation of non-disposable prototypes, in my experience, the reality is that this rarely happens. Such prototypes—and a lot of the images they contain—often get junked at some point in the software development life cycle.
Now, let’s evaluate slice-and-dice prototyping using the following MUDS criteria:
- management—Generally, a problem with slice-and-dice prototyping is that it relies on synchronizing the output from two different types of software: a graphic design application and an HTML editor. What’s more, there are often different people using these applications—possibly even people in different groups within an organization.
- usability testing—Slicing and dicing can produce very functional prototypes. It can also produce very realistic looking high-fidelity or even pixel-perfect prototypes. While this is sometimes very useful, it can also be a double-edged sword! Doing usability testing with screen designs that look like a finished product can unconsciously lead test participants to perceive a prototype as the actual application. In turn, this can make them less forgiving of any problems and limitations of the prototype during testing, distorting the test findings—particularly for satisfaction metrics.
- design—The situation here is similar to that for the usability testing criteria. The high-fidelity prototypes that designers typically produce when using this approach are sometimes very useful. But when senior stakeholders review such prototypes, they often get bogged down in the details of the visual design and forget to focus on more fundamental and, often, more important issues regarding the prototype’s information architecture and interaction design. I’ve seen stakeholders spend entire review sessions of quite early prototypes discussing the color scheme or criticizing the photos a designer has incorporated in a page.
This is one reason why designers sometimes reduce the fidelity of early prototypes that they’ve produced using a slice-and-dice approach before subjecting them to usability testing and stakeholder review. For example, they might convert them to grayscale or even rough them up a bit—perhaps by using sketchy fonts—to make them look less finished.
- specification—If a designer is really using a slice-and-dice approach to produce a non-disposable prototype, this massively reduces the need for specification data, and, of course, that’s great. However, as I pointed out earlier, this is rarely the case; so a UX design team often must still specify their design. Unfortunately, since neither graphic design packages nor HTML editors include specification features, creating and maintaining specifications can be particularly problematic when taking a slice-and-dice approach to prototyping.
Because second-generation prototyping tools don’t support specification, designers typically capture screen images, then insert them into specification documents that they create using applications like PowerPoint, Microsoft Word, or Visio—either simply adding annotations to the images or writing detailed specifications. This process turns into a nightmare because, not only is it very laborious, the designer must update the specification document every time the design prototype changes—which, with the iterative nature of user-centered design, is a common occurrence.
Indeed, this can be such a nightmare that I have known major corporations who use slice-and-dice prototyping to shy away from making sensible changes to a design—simply because updating the documentation would have been so time consuming! Of course, working in an agile development environment magnifies this problem. For each of an agile development cycle’s many rapid iterations, a designer must quickly iterate the prototype, capture new screen images, and revise the specifications.
Figure 3 shows the degree to which slice-and-dice prototyping tools satisfy the MUDS criteria.
Figure 3—Evaluation of slice-and-dice prototyping tools using the MUDS criteria
Next, let’s look at the benefits, limitations, and usages of slice-and-dice prototypes.
The following benefits of slice-and-dice prototyping are easy to understand:
- This approach has been around for a long time now, and you can generally muddle through with it.
- Most software development organizations already have the necessary software and the skills to work with it.
- You can produce prototypes that look like the finished product—assuming that would be helpful for the particular context or stage of prototyping.
When assessing slice-and-dice prototyping using the MUDS criteria, I highlighted some serious problems with this approach. However, in my experience, there are much bigger problems. Slice-and-dice prototyping generally begins with images a designer has produced in a graphic design package such as Photoshop or Illustrator. As a consequence, visual designers often drive the whole prototyping process. This occurs not least because operating a graphic design application takes considerable expertise—skills that visual designers possess almost exclusively. So, this is where, when, and why many interface design problems occur!
Of course, there are some well-rounded UX designers who also possess these visual design skills, along with an excellent understanding of other aspects of UX design such as interaction design, information architecture, and usability engineering. However, in my experience, this is not generally the case. Many visual designers—despite their claims to the contrary—have little experience, expertise, or even interest in these other aspects of UX design, so they focus mainly on visual design.
As a consultant, I often go into organizations to advise them on prototyping matters, and I tell them that slice-and-dice prototyping is now out dated! The simple fact is that there are now much better ways of doing prototyping. We should return Photoshop to its intended use—manipulating photographic images. The clue is in the name guys! Web developers should use HTML editors to write working HTML—and play only a minor role in modern UX design.
The Third Generation
A key feature of UCD is that it is multidisciplinary. So, as UCD principles started to gain traction in software development, professionals working in a whole bunch of different disciplines, quite rightly, wanted to get in on the prototyping act—including business analysts, usability engineers, and technical architects and developers who had an interest in UX design.
Of course, just as with second-generation prototyping, when these people looked for prototyping tools, they chose the tools they already knew how to use—presentation packages like PowerPoint and diagramming packages like Visio and OmniGraffle. They quickly adapted these tools to help them more easily create prototypes, so we got things like Visio stencils that were specifically for prototyping. This is why I call this third generation of tools the adapted-presentation-and-diagramming generation of prototyping tools—or the APAD generation for short.
APAD tools produce prototypes that are completely disposable.
Let’s evaluate the APAD prototyping tools using the following MUDS criteria:
- management—A big advantage that APAD prototyping tools have over slice-and-dice tools is that the entire prototype resides within a single file—as opposed to two or three different types of files. Another advantage is that the applications people use to create them are so common that it’s possible to distribute prototypes in their native format. For example, just about anyone can open a PowerPoint file, and many software development professionals have access to Visio. Other than this, the story gets worse, because, just like slice-and-dice tools, these tools support only single-user operation, so collaborating on prototypes, managing revisions, and getting feedback on prototypes generally present problems.
- usability testing—With APAD prototypes, the key problem is their limited interactivity. When using APAD prototyping tools, doing anything more than hyperlinking screens is cumbersome at best, and it’s simply not possible to prototype many common types of interactions. This can severely limit the scope of any usability test that uses APAD prototypes.
- design—If you use APAD tools in conjunction with a graphic design application such as Photoshop and custom prototyping stencils, it is possible to produce high-fidelity prototypes. However, a key problem with APAD prototyping tools is that they generally are not pixel based, so pixel-perfect designs typically are not achievable with these tools.
- specification—While it is true that APAD prototyping tools are far better for specifying designs than second-generation prototyping tools, because that is not their primary purpose, they are still relatively weak in this area. Keeping specifications up to date with them is typically problematic. Ironically, people often use APAD prototyping tools to add annotations or specifications to prototypes that designers have created using second-generation tools. For example, it’s common for them to paste Photoshop screens into Visio, then annotate them. When people must use second- or third-generation prototyping tools, they take this approach because it’s the best they can realistically do—even though this leaves the UX team’s prototypes and specifications with a whole bunch of nasty maintenance and integrity problems.
In Figure 4, you can the degree to which APAD prototyping tools satisfy the MUDS criteria.
Figure 4—Evaluation of APAD prototyping tools using the MUDS criteria
Next, let’s look at the benefits, limitations, and usages of APAD prototyping tools.
The following benefits of using APAD prototyping tools are easy to understand:
- This prototyping approach has been around for a long time and, as with second-generation prototyping tools, you can generally muddle through with APAD tools.
- Most software development organizations already have the necessary software—for example, PowerPoint and Visio.
- Distributing your prototypes is very easy because of the pervasiveness of this software.
- The software is easy to use. Most software development professionals know how to produce a basic screen design using these tools.
As with the MUDS criteria, APAD prototyping tools suffer from many of the same limitations as second-generation prototyping tools. While they are a better for specifying designs, they have much greater limitations in terms of simulating interactivity.
Again, my advice here is simple: this prototyping approach is now out dated!
The Fourth Generation
As UCD has increasingly gained traction in the software development industry, prototyping has become an important software development activity in its own right—just like coding, database design, and project management.
Around five years ago, this quite rightly led many to conclude that we now need prototyping tools that are truly fit for the purpose—that is, tools whose specific purpose is to enable software development professionals to prototype software systems. In response to this need, software developers have taken three different approaches to solving this problem:
- platform-specific prototyping tools
- pen-and-paper simulators
- dedicated, generic prototyping tools
Platform-Specific Prototyping Tools
Developers have built platform-specific prototyping tools on top of existing software development environments. These are typically specific to a proprietary technology. A key example of such a prototyping tool today is Microsoft’s SketchFlow, which they built on top of Microsoft Blend.
A key selling point of these platform-specific tools is that it is possible to evolve a prototype into a product’s final front-end code within the same overarching software development environment. Therefore, these tools offer, in theory, the ultimate in non-disposable prototyping. However, the reality is often different in this case—for all the reasons I highlighted in relation to the disposability of second-generation prototypes. There are numerous reasons why, in actuality, the prototyping and development activities might be totally separate. So, even if a product team uses the same software for both activities, the result still might be that a prototype gets scrapped at some point during the software development life cycle.
As you can see in Figure 5, some platform-specific prototyping tools comprehensively address all elements of the MUDS criteria. Indeed, some of these tools address the entire software development life cycle. However, of particular note is their capability for prototyping complex interactions, which can be a major benefit when doing early-stage usability testing.
Figure 5—Evaluation of platform-specific prototyping tools using the MUDS criteria
Next, let’s look at the benefits, limitations, and usages of platform-specific prototyping tools.
The theoretical benefits of prototyping in this way are obvious! These tools have the potential to offer prototyping utopia. They let you evolve early conceptual prototypes into a finished product. You can conduct usability tests using realistic software throughout the software development lifecycle and manage the whole prototyping process.
A key limitation of these tools is that they are platform specific—so non-disposable prototyping is possible only for projects that target the platform. Similarly, because these tools are built on top of sophisticated development environments, they are very complex, esoteric software systems. This, in turn, presents the problem that skills relating to these tools can be extremely rare.
Platform-specific prototyping tools are well suited to development projects that target a particular development environment from the beginning, within well-funded organizations that can afford to find and retain people with the rare skills they must have to use these tools.
In many key ways, the goal of what I call pen-and-paper simulators is to provide a modern-day, digital, equivalent of pen and paper. Current examples of such tools include Balsamiq Mockup and Keynote.
In keeping with the intention for these tools to mimic pen-and-paper prototyping, they also produce disposable prototypes.
The degree to which pen-and-paper simulators meet particular MUDS criteria varies across individual software products. However, the simplicity of these tools generally comes at the expense of making trade-offs with most elements of the MUDS criteria, as you can see in Figure 6.
Figure 6—Evaluation of pen-and-paper simulators using the MUDS criteria
Now, let’s look at the benefits, limitations, and usages of pen-and-paper simulators.
The key benefits of pen-and-paper simulators include
- ease of use—Little or no training is necessary.
- rapid prototyping—It is possible to produce prototypes very quickly using these tools.
- low cost—The prices of these tools are typically very low.
As you might imagine, the following key limitations of pen-and-paper simulators are very similar to those of pen and paper:
- Their use is typically limited to conceptual prototyping, or wireframing.
- They offer only limited interactivity.
- It is difficult to model complex systems using these tools.
- They offer limited or no support for writing specifications.
- They have limited management capabilities.
The ease of use, prototyping speed, and relatively low cost of pen-and-paper simulators understandably makes them very attractive for prototyping. They are particularly useful at the early stages of the software development life cycle. Indeed, many organizations have come to see and use them as a direct replacement for pen-and-paper prototyping. However, while there is considerable merit in this approach, we should remember that these tools do not match the capability of pen-and-paper prototyping in terms of their ability to facilitate rapid iteration of design concepts and stakeholder engagement. In particular, because they are technology-based tools, their potential to unduly minimize or even exclude the input of stakeholders who have more limited technical skills remains a significant danger.
Dedicated, Generic Prototyping Tools
The last class of prototyping tools comprises dedicated tools that were designed to provide a comprehensive and generic prototyping solution. These tools are platform independent, so you can use them to prototype many types of systems—for example, Web, desktop, mobile, and to a certain degree, even devices such as set-top boxes. Key examples of such tools today are Axure RP Pro, Justinmind Prototyper, and iRise.
For this category of prototyping tools, the disposability of prototypes varies across particular software products to some degree, but, in general, they produce disposable prototypes. I do not see this a significant drawback for this type of prototyping tool because, for a variety of reasons that I’ve highlighted earlier in this article, there are, in reality, few opportunities for non-disposable prototyping in organizations—even if prototyping tools have this capability.
In general, dedicated, generic prototyping tools comprehensively address all elements of the MUDS criteria, as Figure 7 shows.
Figure 7—Evaluation of dedicated, generic prototyping tools using the MUDS criteria
Finally, let’s look at the benefits, limitations, and usages of dedicated, generic prototyping tools.
We can best understand the key benefits of dedicated, generic prototyping tools by comparing them to the platform-specific prototyping tools with which they often compete. Dedicated, generic prototyping tools are generally cheaper and easier to use than platform-specific prototyping tools. In fact, just a few days of training is usually adequate. Other benefits include the following:
- Though still relatively rare, skills in using these tools are more readily available than those for platform-specific prototyping tools.
- You can use these tools to prototype any type of system.
- These tools are not tied to any particular—perhaps propriety—technology.
The main limitations of dedicated, generic prototyping tools are that they generally produce disposable prototypes—assuming that this is actually an issue, which it may not be. Plus, the skills that using them requires are still relatively rare.
It is also important to point out: the very fact that these tools are technology based means that they generally cannot compete with paper prototyping at the early stages of the software development life cycle. Of course, this limitation is not specific to this generation, or type, of prototyping tools. It is a limitation of any technology-based approach.
Of course, you can use dedicated, generic prototyping tools on any type of prototyping project and at any stage in the software development life cycle—from early conceptual wireframes to high-fidelity, highly interactive prototypes. This is why they have become the prototyping tools of choice for many designers working at the cutting edge of UX design and UCD.
To return to this article’s original question: given that we can now choose from a variety of fourth-generation prototyping tools, why is it that so many organizations are still creating second- or third-generation prototypes?
In my opinion, this actually has little to do specifically with UX design or UCD. Rather, it is because of more general and very common business problems: organizational inertia and resistance to change.
Many organizations do second- or third-generation prototyping simply because that’s how they’ve always done it! Some may not even know that there are now better tools and approaches available—or, if they do know, they’re unwilling to expend the time, effort, and money that making the transition would require. Likewise, as with any significant organizational change, some may perceive such a transition as being too risky.
Resistance to change is also an issue for individuals who might consider learning to use fourth-generation prototyping tools. It is understandable that UX designers who are primarily visual designers and have considerable expertise using second-generation prototyping tools like Photoshop might well perceive a move to a fourth-generation prototyping tool as a considerable threat—perhaps worrying that such a shift might devalue their skill set or stifle their creative expression.
The key point here is that making any significant organizational change is neither easy nor straightforward—of which there is ample proof. Such change takes careful planning and often requires interventions from professionals with specialist skills in organizational change. So, in this case, simply buying some fourth-generation prototyping software and sending the UX team on a training course is unlikely to result in an organization’s achieving the desired transition!
In spite of the magnitude of the challenge that moving to a fourth-generation prototyping tool presents, the pressure on many organizations to make this change soon is mounting. In the future, as UX professionals embrace this important change, it is likely that those organizations still using second- or third-generation prototyping tools may become significantly less competitive—and ultimately, even uncompetitive.
 Muller, M. “PICTIVE: An Exploration in Participatory Design.” Human Factors in Computing Systems Conference Proceedings, 1991.
 Tudor, Leslie G., Michael J. Muller, and Tom Dayton. “A C.A.R.D. Game for Participatory Task Analysis and Redesign: Macroscopic Complement to PICTIVE.” INTERCHI ’93 Adjunct Proceedings, 1993.