All About Card Sorting: An Interview with Donna Spencer

May 25, 2009

Donna Spencer is one of Australia’s best-known information architects, organizer of the UX Australia conference, and a frequent presenter at UX conferences in Australia, the US, and Europe. I caught up with Donna between her appearances at the IA Summit and RedUX DC to talk about card sorting and her new book, Card Sorting: Designing Usable Categories, which Rosenfeld Media recently published.

SB: Can you tell us a little bit about your background? How did you get started?

DS: I was working for a big government department in the very late 90s. (Actually, I worked there for all of the 90s, but not on the Web.) I moved to the Web team right when they were expanding the Web site from 20,000 to 200,000 pages. At the same time, there was lots of discussion on the Internet about information architecture (IA), and that’s when I found my thing.

But that’s about how I found information architecture. You are probably also interested in how I found card sorting. There’s a story for that, too.

Sponsor Advertisement
Continue Reading…

After dramatically expanding this enormous government Web site, we decided to tackle the top-level information architecture—particularly the first two levels of the Web site. (The expansion hadn’t touched this.) I had heard about the idea of card sorting, so arranged a card sort with internal staff and external users, focusing on the content directly available from the top two levels of the site. I basically jumped in with both feet and had a go at it. It worked out really well, and we got some great ideas on how to regroup the content at that level.

Then, I changed organizations to lead an intranet redesign. With my new-found experience in card sorting, I decided to do a card sort on the entire intranet. Boy, oh boy, did I learn a lot about card sorting and about IA during that project. Everything went wrong, which was a much better learning experience than when everything goes right. Oh, I still shudder at the memory of that project.

SB: I’m sure I’m not the only one who wants to know: What went so wrong in that card sort?

DS: This particular intranet was quite big. I first made the mistake of trying to cover the whole intranet in one card sort, with the result that, when users went to sort the cards, they really looked at them and said, “But none of this really goes together. There is too much here.” Of course, they tried, which meant that my results weren’t very good quality. Then, I paid too much attention to what people had called things in the card sort and used those for labels. I remember a point when others on my team gently pointed out that Computing wasn’t a very good navigation label, and I said, “But that’s what the users called it.”

Underlying all of that was a bigger problem—that I was trying to get a technique to do the thinking for me, instead of using it to help me think.

SB: Card sorting seems to be one of those techniques that is very simple in theory, but complex in practice.

DS: The underlying idea is really, really simple: Write content ideas on index cards and get your users to sort them in ways that make sense to them. The hard things are:

  • What do you actually want and expect to learn from doing this sort of activity?
  • How are you going to pull insights from data?
  • What are you going to do with the data?

These are things that most people forget to think hard about. Lots of people say, Hey, we should do a card sort, without really figuring out what they want to learn from it. To do it well, you need to think about what you want to learn and make sure you construct the activity to actually learn that.

Then, you need to collect the right amount and type of data to meet your research goal. There’s little point involving 300 people in a remote card sort when you would get better insight from a face-to-face sort of 10 groups of 3. All you have is a big, fat analysis problem.

SB: What do you do with information after you’ve collected it?

DS: The main thing to do is to look for interesting patterns. To spot things that people did that you didn’t expect—or that confirm what you did expect. You can spot these patterns by just printing out all the results in a big spreadsheet and sticking them on the wall or by crunching numbers with a statistical program. But, however you do it, you are looking for things that tickle your brain cells and help you think through stuff that will help your project.

The thing you don’t do is take the raw data, stick it into a dendrogram—a slightly scary diagram that comes out of a statistical technique called hierarchical cluster analysis—and implement that as your hierarchy on your Web site. That won’t help you at all.

SB: Isn’t that what you’re after, though? A single answer that you can implement?

DS: Wouldn’t that be lovely? I figure that, if something is so simple that all the users come up with a consistent answer that you can just implement, maybe it wasn’t so hard that you needed to involve the users at all. If it is that simple, you really should be able to do it yourself.

Card sorting is best when you want to learn something specific and getting your users involved will help you learn it. Then, you think about what you learned and do something with it.

SB: There’s card sorting, and then, there’s card sorting. Can you tell us a little about the variations in the technique?

DS: One of the variants that people sometimes do is a card sort with their team, not users. That’s not card sorting. That’s just pushing content ideas around a table with your colleagues. That’s a great thing to do, but doesn’t tell you anything about your users.

The other main variation is to use a closed sort—one where you provide the categories as well as the content—as a way to validate an open card sort or to test an information architecture. I personally think this is really weird. Why would you get users to put content into categories and then say, “Yes, this information architecture works?” Users don’t come to your Web site to put content into categories. They come to find information. So, if you want to find out whether people can find information, you should ask them where they would look if they need information.

SB: Where do you see card sorting heading as technique?

DS: When I started card sorting 10 years ago or so—wow, that’s scary—people were using it as a one-shot answer to developing an information architecture. I don’t see that so much any more. Now, I see people using it to get some general ideas about how people think and combine that with other insightful research.

I think this will continue—using it as a way to learn a bit about how people think. I also think the tools will get better, so it is easier to run and easier to analyze.

But I don’t think it will undergo any profound changes. It is too simple—get people to put stuff into groups that make sense for them. That’s about all there is to it.

SB: Why, then, write an entire book about card sorting?

DS: Even though card sorting is simple in concept, any user research method has little traps. The book goes step by step through all the detailed steps to plan and run a sort—with lots of tips based on my experiences. Then, there are two whole chapters dedicated to the analysis of results. There is definitely plenty to talk about on that alone. But it is still quite a skinny book. People should be able to pick it up as they work through their first card sort and work through it with the book.

Learn More

If you’d like to learn more about card sorting, take a look at Card Sorting: Designing Usable Categories at Rosenfeld Media. 

Principal at Meld Studios

Sydney, New South Wales, Australia

Steve BatyFocusing on the business side of the user experience equation, Steve has over 14 years of experience as a UX design and strategy practitioner, working on Web sites and Web applications. Leading teams of user experience designers, information architects, interaction designers, and usability specialists, Steve integrates user and business imperatives into balanced user experience strategies for corporate, not-for-profit, and government clients. He holds Masters degrees in Electronic Commerce and Business Administration from Australia’s Macquarie University (MGSM), and a strong focus on defining and meeting business objectives carries through all his work. He also holds a Bachelor of Science in Applied Statistics, which provides a strong analytical foundation that he further developed through his studies in archaeology. Steve is VP of the Interaction Design Association (IxDA), a member of IA Institute and UPA, founder of the UX Book Club initiative, Co-Chair of of UX Australia, and an editor and contributor for Johnny Holland.  Read More

Other Columns by Steve Baty

Other Articles on User Research

New on UXmatters