UX Analytics, Part I: A Call to Action

March 21, 2011

As UX researchers, our goal is to identify customer pain points and obstacles in a given workflow or process, then tell a compelling story about their risks and provide general recommendations for alleviating those risks. But after eight years in UX research—even having mastered the arts of compelling storytelling and building stakeholder empathy—I still never quite saw the results I’d hoped for. Worse, my work required me to move quickly onto the next project, never to hear again about the outcomes of projects past.

However, my experience on one project was enlightening: When Web analytics data disproved my post-usability test hypothesis, I realized the importance of my being data driven in my practice—as opposed to my just providing UX research data. This revelation was the starting point of my reinvented career in UX analytics—a marriage of right-brain, deep-dive, empathic UX research and left-brain, quantitative analysis of users’ actual behaviors.

Champion Advertisement
Continue Reading…

This transformation has provided a new depth of power to my storytelling, with the result that some of my key issues and recommendations have moved rapidly up to the top of our development team’s list of priorities and are now considered key business initiatives. For example, instead of my saying, “Users are confused about creating an account,” I’d instead say, “Users are confused about creating an account, and 36% of them are abandoning the process, causing an estimated revenue loss of $650K annually.”

In this article, which is the first in my series on UX analytics, I’ll define and describe the value of UX analytics and initiate you in this very fruitful approach to UX research.

The Business Impact of UX Analytics

At the heart of UX analytics is business impact. My stakeholders have rarely denied the importance or even criticality of the obstacles and inconveniences that our Web sites present to our customers. But, as UX professionals, our recommendations for usability improvements must compete with other projects that have revenue projections tied to them. My call to the UX community is that we must compete on the same playing field with other projects by aligning our arguments with the same business logic.

The consequence of my doing this level of business analysis is that I am now more deeply involved in projects, I am often consulted by stakeholders, and my participation on a project is integral to the implementation of my recommendations. In addition, I am responsible for the post-implementation measurements, which enables me to recommend further design iterations if both the data and my design expertise indicate there’s still opportunity to improve the user experience.

This approach has afforded me a better platform for advancing awareness of user needs—whether through further usability testing, customer interviews, or redesign efforts that employ iterative prototyping and testing methods. Now, my case might be, “Users are confused about creating an account, and 36% of them are abandoning the process, causing an estimated revenue loss of $650K annually, so I recommend that we do a deep dive, through a series of usability tests, to understand why users are abandoning.”

Another powerful benefit of UX analysis is that my clients are now key advocates for the changes that I’m advocating. They’ve become empowered, engaged negotiators with development teams, persuading them to implement the solutions that best balance user and business needs.

What Is a UX Analyst?

To give you an idea of what my objectives are as a UX Analyst, here are a few bullet points from my newly crafted job description:

  • Marry multiple sources of data—such as customer experience management, Web analytics, customer satisfaction, call center feedback, and primary or secondary research—with user-centered design best practices to deliver actionable insights and recommendations.
  • Do strategic planning for data gathering and measurement that support Ebusiness initiatives and proactively identify needs for measuring user experiences.
  • Clearly communicate the business and customer experience opportunities, and deliver measurable recommendations that increase value to the business and the customer.
  • Lead the design of new and evolving UX and Voice of the Customer (VOC) dashboards.
  • Draft UX documentation such as wireframes, flowcharts, and taxonomies when useful as a way of supporting recommendations.

Today, I report to our Digital Analytics Director, who was formerly known as our Web Analytics Director and changed the team’s name to communicate that it encompasses both quantitative and qualitative customer behavior research and analysis. My key clients are the Site Managers, each of whom owns of a particular ecommerce Web site and is responsible for its development roadmap, site revenue, and key performance indicators (KPIs). All of these teams reside in Ebusiness.

Getting Started with UX Analytics

Here are some immediate steps you can take to get started down this UX analytics path. I suggest that you approach UX analytics by taking these steps in the order I’ve listed them:

  1. Build a trusted partnership with someone on your company’s Web Analytics team who would be willing to meet with you for an hour on occasion, to pull some reports and explain them to you.
  2. Find out which Web analytics tools your company uses, do a little research on them to understand their utility, and start becoming familiar with their use.
  3. Take an Excel class or some tutorials to become comfortable with the mechanics of manipulating data—like calculating averages, sums, and divisions of data. (You don’t need to worry about formulas and pivot tables right now. Macros are not something I work with.)
  4. Build a strong partnership with your User Acceptance Testing (UAT) analysts who are responsible for the Web sites of interest. They have direct insights into bugs and system issues that you should become intimately aware of. Sometimes recommendations may include fixing a bug that has been sitting at the bottom of a queue for a while.
  5. Reach out to your VOC/customer satisfaction survey analysts to get on their distribution list, including getting open-ended responses from customers.
  6. Reach out to your call center team to get on their distribution list for reports indicating Web site problems, including getting customers’ open-ended responses from call center representatives.
  7. Steep yourself in these open-ended responses. I suggest spending no more than 30 minutes at a sitting. The goal is not to read all of those responses, but to make a practice of rapidly auditing them to build your tacit knowledge of your customers’ needs and experiences. Ultimately, doing this could build your position as one of the authorities on your company’s customer needs.
  8. Consider other sources of data that could reveal user insights and meet the analysts or get on their distribution lists, if they have any. Other sources could be optimization testing or social media analysis.
  9. Brush up on various methods of online qualitative user research and the remote tools the entire UX community is discussing today, including heatmaps, eyetracking, clicktracking, unmoderated usability testing, and remote usability testing. For a fairly complete list of tools, check out the article “24 Usability Testing Tools,” by W. Craig Tomlin.

In Conclusion

I’ve experienced the power of advancing UX findings and recommendations by tying them to business impact. So can you. Just building a trusted partnership with someone on your Web Analytics team may be empowering in itself and open some new doors, spurring thought and innovative ideas for change.

In Part II in this series, I’ll tell you how to get started with business impact analysis by creating a business process funnel. 

Senior Customer Experience / User Experience Analyst at Deluxe Corporation

Denver, Colorado, USA

Kristi OlsonKristi has been doing UX research for ecommerce Web sites since 2001 and information architecture since 2006. After specializing in qualitative research methods for Target and Evantage Consulting, she moved to her current employer, Deluxe, and transformed UX research by blending qualitative and quantitative insights in a new approach called UX analysis. Kristi is an expert in research methodology, designing and blending methods of research, and discovering creative approaches to gathering data that meet business objectives. Kristi has synthesized her long-time, right-brain approach to user research with the left-brain approach of Web analytics to more powerfully communicate with Ebusiness. She has been Deluxe’s Tealeaf owner, administrator, and primary business user since 2009 and is on the road to becoming an Omniture Discover power user. Kristi is currently completing her Master of Liberal Studies in Innovation Studies.  Read More

Other Articles on UX Strategy

New on UXmatters