Site Search Analytics: A Conversation with Lou Rosenfeld

By Kristina Mausser

Published: July 18, 2011

Lou RosenfeldLou Rosenfeld’s newest book, Search Analytics for Your Site: Conversations with Your Customers, has been the subject of more prelaunch buzz than most UX books have gotten this year. It seemed everyone was tweeting, talking, or speculating about it before the ink had even had a chance to dry. And, true to the hype, this book delivers in spades. If you read one book this year to hone your craft, add value to your UX practice, or enable you to help your clients, this is the one! Lou recently found some time in his very hectic schedule to sit down and talk with me about his book and the burgeoning practice of site search analytics (SSA).

KM: This book is so rich with actual case studies and insights. It has obviously been years in the making.

LR: Heh… Thanks, but those “years in the making” actually had much more to do with the many other activities and milestones—both professional and personal—that interrupted my writing. To be blunt, I hate to write and would rather do just about anything else. But the extra time did help me think through the topic a bit more. I hope that shows in the final product.

KM: What inspired you to write this book?

“As an information architect, I sit squarely in the findability corner of user experience. Search is a huge component of findability, and … search analytics is an obvious way to improve search performance.”
—Lou Rosenfeld

LR: As an information architect, I sit squarely in the findability corner of user experience. Search is a huge component of findability, and when you think about it, search analytics is an obvious way to improve search performance. It’s essentially a way to close a feedback loop: users search, and we give them results, but we do very little to find out how well those results are actually serving users. By not analyzing query data, we miss out on all sorts of opportunities to improve search performance. An added bonus is that site search analytics can also help you improve your site’s navigation, metadata, and content. In fact, the book includes separate chapters on these topics.

Yet, for various reasons, little has been written about the topic. Neither the UX nor the Web Analytics communities have shown much interest in this sad orphan, which is too bad, considering how valuable it can be to both. I hope my book—the last chapter, specifically—helps change that.

KM: Lou, I find your writing style to be very approachable and hands-on. Your book does an excellent job of walking people through site search analytics, so once they’ve read the book, in addition to knowing about the topic, they can actually do site search analytics.

LR: Thanks! As Steve Krug would say, it isn’t rocket surgery. Anyone who can operate a spreadsheet and has an hour per month can get something of value from site search analytics.

KM: And that’s one thing you advocate doing in your book—along with many other useful tips we can follow. What’s another technique your book covers that you think every company should apply to its Web site?

“Everyone should play with query data to see what sorts of interesting patterns emerge.”
—Lou Rosenfeld

LR: Pattern analysis. Everyone should play with query data to see what sorts of interesting patterns emerge. Get the most frequent 50 or 100 queries—the short head, as opposed to the long tail—into a spreadsheet and have as many people as possible look for patterns and other interesting things. You’ll be amazed at how quickly insights begin to emerge. It’s really a fantastic and fun way to dig deep into learning about user intent.

It’s also a good idea to regularly review the most frequent queries that retrieve zero results. You’ll immediately go into diagnosis mode. “Do we really not have that content? Should we? Is it there, but not titled properly? Is it mistagged? Jargony? Hey, this would be good ammunition for the next time our content authors balk at following our content guidelines….”

I’m convinced that doing little things like this on a regular basis would negate the need to redesign the structure of most sites—which would be a really good thing.

KM: When writing your book, who did you intend it for?

LR: Like all Rosenfeld Media books, the core audience is user experience practitioners—whether designers or researchers. But this particular title should also appeal to marketers and analytics people.

KM: You’ve touched on how site search analytics can help improve a site’s content, metadata, and messaging. So it seems site search analytics could really be part of a content strategist’s secret arsenal as well. Would you agree?

“You can learn a lot about how your deep content works—or ought to work—in terms of search and contextual navigation.”—Lou Rosenfeld

LR: I think so. You can learn a lot about how your deep content works—or ought to work—in terms of search and contextual navigation. I’d imagine such things are important to content strategists—though, admittedly, I’m not one.

KM: We often focus on what a Web site is doing right and sometimes tend to ignore the numbers that indicate what it’s doing wrong. But your book encourages us to look at these numbers through failure analysis. Does this approach negate the need for additional usability testing?

LR: Absolutely not. In fact, analytics are, in general, good at telling you only what is going on. They do little to explain why things are the way they are.

So, while site search analytics might help you to better understand users’ behaviors and arrive at some good hypotheses for why they behave the way they do, they are only hypotheses. You still need qualitative methods of user research like usability testing to get from hypotheses to conclusions.

KM: Site search analytics results dovetail nicely with persona creation, don’t they?

LR: Absolutely! By using your query data to describe a persona’s information needs, you can make your personas more evidence based—and, perhaps, make it more likely that people will take them seriously.

KM: These days, people are throwing the word silo around a lot, with respect to the various disciplines that make up Web development. In your book, you advocate finding complementary problem-solving practices from the UX and Web Analytics communities. Where is the common ground with site search analytics?

“Site search analytics … [is] semantically rich—search queries essentially express users’ information needs in their own words.”—Lou Rosenfeld

LR: Site search analytics offers common ground in that it has value to both communities. It’s just another flavor of analytics—behavioral data that’s a by-product of Web site use—so Web analytics practitioners know what to do with it.

But it’s also semantically rich—search queries essentially express users’ information needs in their own words. That makes site search analytics quite different from most varieties of analytics—and more like the qualitative studies that UX folks commonly rely upon.

All research findings lose their value when we wall them off in pointless technology ghettoes—for example, “Here’s the Omniture report that you requested.”—or within organization divisions—like “We’re User Research. We don’t do analytics.” When you put these findings together in smart, innovative ways, their sum is far greater than their parts. In the coming years, the organizations that figure out how to synthesize their different types of research are the ones that will destroy their competitors.

KM: Why do you think it’s taken so long for UX professionals to adopt site search analytics in their practice? How can we encourage its adoption?

“The bigger challenge … is that many of us UX folks fear behavioral data and distrust quantitative measurement. This has a lot to do with our own biases….”—Lou Rosenfeld

LR: To be fair, many of us don’t have access to our site’s query data—or perhaps don’t even know it exists. But these are barriers that we can overcome.

The bigger challenge—and I’ll likely take a bit of heat for this blatant stereotype—is that many of us UX folks fear behavioral data and distrust quantitative measurement. This has a lot to do with our own biases and academic backgrounds. But while it’s smart to be skeptical of any particular type of research, it’s a terrible idea to write off whole swaths of data that could lead to new insights—simply out of ignorance or because looking at this data makes us uncomfortable. If we continue to take the attitude that stuff like this is the Analytics team’s bailiwick, we’re doing ourselves—and our clients and employers—a huge disservice.

KM: Where do you think site search analytics could have the most impact for companies?

“Analyzing the query data … will provide you with a variety of actionable insights that will enable you to tune the search system so it actually works.”
—Lou Rosenfeld

LR: Here are two situations where site search analytics could give you significant bang for your buck:

  1. Your company has invested $200,000 in a new search engine, but once IT installed it, they didn’t bother to tune it. Not surprisingly, users hate the search system. When your bosses start wondering why they’ve just wasted all that money, offer to help them save face by analyzing the query data. For a fraction of the installation cost, doing this will provide you with a variety of actionable insights that will enable you to tune the search system so it actually works.
  2. Your company measures its Web site’s performance in all sorts of powerful ways—but none of its KPIs address findability—which is often at the heart of many of your users’ biggest complaints. Site search analytics enables a set of findability-related metrics that can really beef up existing KPIs. You can read more about this in Chapter 7 of my book.

KM: Lou, it’s been a real pleasure talking with you today. You mentioned earlier that you hate writing, but on behalf of the UX community, I’d like to thank you for writing Search Analytics for Your Site. Your book provides such good, step-by-step, how-to instructions for a unique approach that can help us understand people’s information needs in ways that are subtly intricate, but so important. The book is approachable, helpful, and offers an in-depth level of knowledge that is far beyond what we could get from a few blog posts. It’s like having one of your workshops in a book.

Lou Rosenfeld’s New Book: Search Analytics for Your Site: Conversations with Your Customers

To learn more about site search analytics, read Lou’s new book Search Analytics for Your Site: Conversations with Your Customers. For a 20% discount on this and other Rosenfeld Media books, purchase them on the Rosenfeld Media Web site, using the discount code: UXMATTERS.

Join the Discussion

Asterisks (*) indicate required information.