Optimizing and Extending Existing Research Methods
Practitioners continue to build on proven methodologies and adapt them to real-world scenarios and evolving technologies. There seems to be a tension between those who seek more rigorous approaches to user research and usability evaluation and those who have limited time and budgets and do what they can to convince business stakeholders of the value of research. Hopefully, continuing discussions on this topic can help us do both more effectively.
Express Usability: How to Conduct and Present a Usability Analysis in One Week or Less
In their presentation titled “Express Usability: How to Conduct and Present a Usability Analysis in One Week or Less,” Sarah Weise and Linna Manomaitis discussed an approach to doing a full cycle of user research and analysis activities in one week. The goal was to lower the threshold for trying usability testing by offering business stakeholders abbreviated usability activities they could complete in one week.
In tailoring standard usability techniques to fit into a single week, they essentially created a list of the different activities they could perform and assigned those activities to three phases: data gathering, analysis, and deliverables and presentation. For a given project, they would work with stakeholders to focus the goals of their research and analysis, then select individual activities from each phase that they could complete in a week. For example, if their goal were to improve the navigation on a Web site, they might choose task analysis as the data-gathering technique, do an analysis, then prepare a set of wireframes as the deliverable. By grouping activities in their list in this way, focusing on one goal, selecting the activities that best supported that goal, and getting the job done in one week—that is, in 40 hours—they were able to demonstrate the value of usability testing to the business and build relationships with stakeholders that smoothed the way to their doing usability testing on other projects.
I was especially intrigued by the structure this approach offered, helping them deal with to the inevitable tight turnaround times most projects require, and was impressed by the way they used the one-week timeframe as a way of focusing their research efforts on achievable goals rather than trying to improve everything at once.
Combining Methods: Web Analytics and User Testing
Martijn Klompenhouwer and Adam Cox’s presentation, “Combining Methods: Web Analytics and User Testing,” described how it’s possible to combine these two different, proven methods to achieve greater insights. The room was overflowing with people in advance of the session, an indication to me of great interest in this topic. While many people in the UPA audience have experience with qualitative usability testing, I got the sense that many were less familiar with Web analytics, but interested in its potential for providing a quantitative balance to the qualitative findings from usability testing.
Martijn and Adam talked about several ways in which the two disciplines could work together. For example, when planning a usability test, Martijn used Adam’s analysis of Web analytics to get insights about personas that could potentially represent the people who are visiting their current site. Data such as the other sites from which visitors are coming to their site and search terms people are using to find their site can provide clues about their different audiences and inform scenarios for a usability study. Analytics can also help narrow the focus of a usability study, focusing it on parts of a site that have the biggest drop-off rates, bounce rates, or tendencies to branch to the site search. For example, an analysis of the drop-off rate for a checkout process funnel could help them determine on what part of the process they should focus their research.
In presenting the results they achieved with this method, Martijn and Adam told us that combining the two sets of information had the most impact: The Web analytics validated the findings of the usability study, while the usability study provided color and added explanations of why users did what they did, enriching the metrics. The presentation provided a well-prepared summation of the benefits of using Web analytics and usability studies together. There were many questions from the audience—most relating to details of implementation. Using quantitative metrics to complement usability studies is definitely a compelling approach. I’d like either the presenters or others to extend this discussion and present a tutorial that would help those who are interested in this approach to make sense of where to start—given a mountain of Web analytics that may be daunting to climb.
The Importance of Storytelling
Storytelling was the theme of the UPA conference a couple of years ago, but the topic of storytelling is still relevant and important to the successful practice of user-centered design. Stories can be a very effective way of communicating ideas—both from research participants to researchers and from researchers to project teams or stakeholders. Fortunately, there were talks that covered the full range of this topic.
InfoPal: A System for Conducting and Analyzing Multimodal Diary Studies
Jhilmil Jain presented a talk titled “InfoPal: A System for Conducting and Analyzing Multimodal Diary Studies.” In diary studies, users record their experiences and interactions with a product or service independently, over an extended period of time. Such studies are particularly useful when it is difficult to observe participants directly or over a long period of time. A drawback of traditional diary studies is that text-based diaries are not very dynamic and, often, users cannot easily complete them within the context in which they use an application—for example, while driving. Additionally, participants may forget to record something or might be aware of only a subset of items that may be important. Finally, diary entries are often tied to specific devices. In response to these drawbacks, Jhilmil and her team at HP Labs set out to develop a new system with the following goals in mind:
- flexibility of expression, using one or more devices or modalities
- seamless, collaborative diary creation by multiple participants
The result of their efforts is a system called InfoPal, which allows test participants to create diary entries using either a Web application or a mobile application, capturing text, voice input, pictures, videos, and audio recordings. The good thing is that the multimodal approach to data capture not only makes it easier for participants to create diary entries, it also enhances the researchers’ insights. While this tool is not available to the general public, their experience with InfoPal can provide some helpful guidelines for those looking to develop and conduct their own diary studies.
Using Stories Effectively in User Experience Design
Taking a more holistic look at stories, Whitney Quesenbery and Kevin Brooks discussed “Using Stories Effectively in User Experience Design.” To help the audience understand how stories are effective, they started with listening exercises for the audience that reinforced the power of storytelling in conveying messages. They showed an example that demonstrated the ineffectiveness of personas based on demographics and personality characteristics alone, then compared those personas to much more effective personas based on stories. Since I am a consultant who does research work for external clients, people sometimes asked me how I can convince clients and business owners to act on the merits of my research findings. Whitney and Kevin’s talk reminded me of how powerful stories are in this regard, when communicating with stakeholders. In fact, the most powerful experiences are those in which research participants tell their stories to stakeholders directly, through video clips of research sessions.
Whitney and Kevin also provided some insight into using stories to elicit feedback from participants. As they put it, to complete traditional task analysis and other documentation, you have to ask research participants about their frequency of use, how a design would fit into their work practice or a buying process, for example. But, if you can complement such questions with a simple request to “Tell me a story about the last time you did [scenario or task]…,” the resulting data is likely to be infinitely richer.