Introduction to Usability Evaluation in an Agile Software Development Process
The emergence of the agile software development manifesto has changed the practice of user-centered design (UCD).  In contrast to traditional approaches to usability evaluation, iterative usability evaluations within an agile context should be rapid and efficient, but must still reveal any problems that newly added features have introduced. Software engineers have rapidly adopted the agile approach, and embedding usability evaluations in an agile software development process provides timely feedback from users and results in intensive collaboration between usability experts and software developers. [2, 3] In particular, it facilitates a collaborative approach to usability testing.  However, while agile development encourages creative teamwork and fosters effectiveness and maneuverability, the underlying iterative processes are sometimes confusing and hard to distinguish because of the rapidity of the ongoing development process. [5, 6, 7]
While software engineering employs various software-development processes, focusing on usability to different degrees, prior research has rarely disclosed explicit usability-evaluation processes that are integrated into the agile development process. [8, 9] In particular, there has been a lack of research on usability evaluations’ ill-defined iterations within the context of the agile development of personal health–management systems. In contrast with the development of systems other than healthcare systems, the usability evaluation of healthcare systems is more complicated.  With the greater emphasis on iterative, formative evaluation in agile development, there is demand for closely integrating usability evaluation within the development process in engineering healthcare systems. [11, 12] Therefore, our study aimed to describe the explicit process of iterative usability evaluation in the agile development of a personal health–management system.
Because an agile development approach emphasizes the value of collaboration and the rapid feedback that iterative usability evaluation can provide, its effectiveness typically relies more on the involvement of usability experts. This helps to reduce the investment of time and money in developers’ decision making and the delivery user feedback.  However, previous studies have not identified a distinct role for usability experts working with designers. [14, 15]
In examining the role perceptions of usability experts who were collaborating with developers and users in the agile development process, our study was founded on the theory of Role Identity Salience,  which provides a conceptual bridge linking the individual to the larger social structure that emphasizes self-definition, behaviors, and social relations. The concept of role-identity salience plays a critical part in many contemporary discussions of the concept of the self.
The Purpose of Our Study and Its Findings
Our research aimed to discover the explicit processes that constitute iterative usability evaluation in agile development, as well as to increase our understanding the roles usability experts play in an agile context. Two research questions guided our study of the agile development of a personal health–management system:
- What constitutes an iterative usability evaluation process in agile development?
- How do usability experts perceive their roles in an agile development process?
What Constitutes an Iterative Usability-Evaluation Process in Agile Development?
In the case we studied, the development of a personal health–management system, the usability-evaluation process extended across three stages within each two-week sprint. Usability experts, system developers, and users participated during different stages of the usability-evaluation process, as Figure 1 shows.
The system developers and the usability experts resided in geographically dispersed locations. Therefore, during the first week of each sprint, the development team and the usability team participated in a conference call to decide what parts of the application they would be testing—in the form of either paper prototypes or live code. The usability team familiarized themselves with the application, brainstormed ideas, identified their usability-evaluation goals, developed a think-aloud protocol for test sessions, as well as a protocol for user interviews, and recruited participants for the study.
During the second week of each sprint, the usability team conducted usability testing with the participants they had recruited. To facilitate and speed up the delivery of feedback to the team, a facilitator led each usability test session, and a note taker captured users’ comments during each session. Typically, five to eight users participated in each cycle of usability testing.
At the end of each two-week sprint, the usability team synthesized their findings and the participants’ feedback and reported the results of the study to the development team, providing an Excel spreadsheet and video clips that documented the test sessions, as well as their recommendations for improving the application. Then the two teams participated in another conference call to discuss the findings, provide clear answers to any questions, and address any concerns.
How Do Usability Experts Perceive Their Role in Agile Development?
During each two-week sprint’s cycle of usability evaluation, three to five usability experts were assigned as members of the agile development team to plan and facilitate the test sessions. A member of the usability team served as a facilitator, who led the usability testing; a note taker, who recorded users’ feedback; and a team leader, who synthesized users’ feedback. A team leader could also be a facilitator or a note taker. The usability experts who facilitated the user interviews and conducted usability testing described their specific tasks as follows:
- Meet with the system developers to discuss and set the objectives of the usability evaluation.
- Develop protocols for user interviews and usability testing.
- Conduct usability testing to discover usability issues and collect users’ feedback.
- Analyze the findings from usability testing and the user feedback that was collected during interviews.
- Report the findings and user feedback, and make suggestions for improving the software to the system developers.
After examining the usability professionals’ perceptions of their experiences conducting usability evaluations in an agile context, we found that the usability experts played significant roles during each two-week sprint throughout the entire development process, as shown in Figure 2.
The usability experts’ most important role was serving as a bridge connecting system developers and users. In each sprint throughout system development, the usability experts tested developers’ assumptions in their usability-evaluation sessions with users. During the usability evaluations, the usability experts played a vital role in obtaining users’ feedback. Finally, they were responsible for reporting users’ feedback to system developers and provided appropriate recommendations for the system’s improvement.