User Experience + Quality Assurance = Usability
Published: July 8, 2013
When writing about quality assurance and user experience, Jakob Nielsen put it perfectly:
“Quality assurance impacts the user experience: when things don’t work, users question their understanding and develop superstitions and inefficient workarounds.”
One key factor unites the disciplines of User Experience and Quality Assurance (QA) at their core: usability. According to Wikipedia:
“Usability is … the ease of use and learnability of a human-made object.”
The principle of usability applies to everything from physical items like hammers to all types of software applications. Often, technology seems to transcend the physical world. As a result, the usability of software is pivotal to its value. In agile software development, the usability of a product is a shared concern of the entire team. However, I’d like to focus on the way Quality Assurance and User Experience address usability throughout four broad phases of agile development: research, development, testing, and release.
Business analysts and designers typically dominate the research phase of agile development, but this phase is also crucial for Quality Assurance. At ThoughtWorks, we’ve used this phase do specification by example, which Quality Assurance usually leads.
The basic idea behind specification by example is the creation of one source of truth on requirements, encompassing all perspectives. Specification by example commonly comprises example-driven development, executable requirements, acceptance test–driven development, and agile acceptance testing. Project team members collaboratively create and maintain these requirements, which lets them run all tests and produce documentation synchronously. In doing so, the team avoids miscommunications and confusion about requirements and design. When the team makes changes, they can apply the specifications and create a refined set of examples to derive an acceptance test. This process happens continuously, supporting users by creating living documentation that evolves much in the same way as product usage.
While the development phase comprises a variety of activities, the process of designing the user interface is of grave importance to both User Experience and Quality Assurance. During this period, User Experience should build and refine a style guide that provides information about standards for the application under development. These may range from colors to design patterns.
This is an opportune time for User Experience and Quality Assurance to work together to develop a shared understanding of how the application should look and behave. As the team continuously develops and tests user stories, they’ll identify certain aspects of the user interface that will not change—as defined by User Experience and the business. The sooner Quality Assurance becomes aware of the areas that won’t change, the sooner they can build regression tests that they should repeat during every testing cycle and note applicable standards.
During the testing phase of the agile development cycle, Quality Assurance may conduct unit tests, regression tests, smoke tests, and exploratory tests. However, the rest of the team focuses on activities relating to other phases. Thus, User Experience researches and designs Component A of the application, then Development codes it, and finally, Quality Assurance runs it through a series of tests. While Quality Assurance is testing, Development is working on developing Component B, because User Experience should already have completed researching and designing Component B and moved onto researching and designing Component C. This is the process that the team typically follows. When Quality Assurance is busy testing Component A and User Experience is researching and designing Component C, there is a high chance that those components of the application may be related. This opens up a world of opportunities for the disciplines to work together.
For example, about a year ago, I was working for a client and took advantage of the opportunity to work with Quality Assurance. I was about to begin my weekly usability testing on a part of the application that focused on users’ ability to progress through a series of steps to completion when I realized that the QA engineer I was working with might be able to help me. I was testing on paper, and the latest build included the steps, but without any progress indicator. Prior testing had indicated that users needed to be aware of their progress, so my test included our latest progress bar designs.
Meanwhile, Quality Assurance was busy doing exploratory testing on the series of steps that had been developed since the last check-in. At this point, I noticed that their exploratory testing was following paths that our users could travel down during usability testing. As a result, I asked for their input on my scripts and found that they had already discovered a series of bugs that could potentially affect that week’s build and my usability testing. In addition, they suggested a variety of ways to test the unhappy path to account for the variety of our users’ needs.
Over time, our paired relationship resulted in our doing diagramming sessions, during which we outlined all of the user roles and their potential scenarios of use. We were quickly ready to provide this diagram to the team. While this diagram was useful to everyone, it was most helpful to the Product Owner. Although he spent most of his day thinking three months ahead of the current build, our diagrams were effective in pulling him back to the now. They spurred many helpful conversations about context and potential conflicts. This experience proved to me that pairing User Experience and Quality Assurance is exceptionally beneficial to a project.
Once a team has researched, developed, and tested the code, it’s time to release the software. A very important part of the release phase is implementing and tracking analytics. The results can produce business insights, identify patterns of user behavior, and catch bugs in the system.
When you are in beta, these analytics validate your product concept and potentially save you from making a big mistake before your product even hits the market. When you release your product to the market, you can use these analytics to re-establish trust with your customers. You can watch for any mistakes or errors that may occur during usage. User Experience can use these insights to assess user behavior, while Quality Assurance can use them to catch bugs.
Together, these team members can identify key user-behavior patterns and make sure that they are free of bugs and any workflow blocks that users might encounter. If they identify any bugs, they can start from the beginning, reassess their design, and make any changes that are necessary to create a truly usable product.
Throughout the product development process, there are various activities on which User Experience and Quality Assurance can work in tandem, including specification by example, the creation of a style guide, usability testing, and analytics. Quality assurance impacts the user experience. Pairing the User Experience and Quality Assurance functions early and often can help you to close gaps in your understanding and enable you to build a truly usable product. Try this out! It’s definitely worth the conversation!