All About Card Sorting: An Interview with Donna Spencer

By Steve Baty

Published: May 25, 2009

Donna SpencerDonna Spencer is one of Australia’s best-known information architects, organizer of the UX Australia conference, and a frequent presenter at UX conferences in Australia, the US, and Europe. I caught up with Donna between her appearances at the IA Summit and RedUX DC to talk about card sorting and her new book, Card Sorting: Designing Usable Categories, which Rosenfeld Media recently published.

SB: Can you tell us a little bit about your background? How did you get started?

DS: I was working for a big government department in the very late 90s. (Actually, I worked there for all of the 90s, but not on the Web.) I moved to the Web team right when they were expanding the Web site from 20,000 to 200,000 pages. At the same time, there was lots of discussion on the Internet about information architecture (IA), and that’s when I found my thing.

But that’s about how I found information architecture. You are probably also interested in how I found card sorting. There’s a story for that, too.

UXmatters readers can receive a 20% discount when purchasing Card Sorting: Designing Usable Categories directly from Rosenfeld Media, by using the discount code UXMATTERSCS.

After dramatically expanding this enormous government Web site, we decided to tackle the top-level information architecture—particularly the first two levels of the Web site. (The expansion hadn’t touched this.) I had heard about the idea of card sorting, so arranged a card sort with internal staff and external users, focusing on the content directly available from the top two levels of the site. I basically jumped in with both feet and had a go at it. It worked out really well, and we got some great ideas on how to regroup the content at that level.

Then, I changed organizations to lead an intranet redesign. With my new-found experience in card sorting, I decided to do a card sort on the entire intranet. Boy, oh boy, did I learn a lot about card sorting and about IA during that project. Everything went wrong, which was a much better learning experience than when everything goes right. Oh, I still shudder at the memory of that project.

SB: I’m sure I’m not the only one who wants to know: What went so wrong in that card sort?

“I was trying to get a technique to do the thinking for me, instead of using it to help me think.”

DS: This particular intranet was quite big. I first made the mistake of trying to cover the whole intranet in one card sort, with the result that, when users went to sort the cards, they really looked at them and said, “But none of this really goes together. There is too much here.” Of course, they tried, which meant that my results weren’t very good quality. Then, I paid too much attention to what people had called things in the card sort and used those for labels. I remember a point when others on my team gently pointed out that Computing wasn’t a very good navigation label, and I said, “But that’s what the users called it.”

Underlying all of that was a bigger problem—that I was trying to get a technique to do the thinking for me, instead of using it to help me think.

SB: Card sorting seems to be one of those techniques that is very simple in theory, but complex in practice.

DS: The underlying idea is really, really simple: Write content ideas on index cards and get your users to sort them in ways that make sense to them. The hard things are:

  • What do you actually want and expect to learn from doing this sort of activity?
  • How are you going to pull insights from data?
  • What are you going to do with the data?
“You need to think about what you want to learn and make sure you construct the activity to actually learn that.”

These are things that most people forget to think hard about. Lots of people say, Hey, we should do a card sort, without really figuring out what they want to learn from it. To do it well, you need to think about what you want to learn and make sure you construct the activity to actually learn that.

Then, you need to collect the right amount and type of data to meet your research goal. There’s little point involving 300 people in a remote card sort when you would get better insight from a face-to-face sort of 10 groups of 3. All you have is a big, fat analysis problem.

SB: What do you do with information after you’ve collected it?

DS: The main thing to do is to look for interesting patterns. To spot things that people did that you didn’t expect—or that confirm what you did expect. You can spot these patterns by just printing out all the results in a big spreadsheet and sticking them on the wall or by crunching numbers with a statistical program. But, however you do it, you are looking for things that tickle your brain cells and help you think through stuff that will help your project.

The thing you don’t do is take the raw data, stick it into a dendrogram—a slightly scary diagram that comes out of a statistical technique called hierarchical cluster analysis—and implement that as your hierarchy on your Web site. That won’t help you at all.

SB: Isn’t that what you’re after, though? A single answer that you can implement?

“Card sorting is best when you want to learn something specific and getting your users involved will help you learn it. Then, you think about what you learned and do something with it.”

DS: Wouldn’t that be lovely? I figure that, if something is so simple that all the users come up with a consistent answer that you can just implement, maybe it wasn’t so hard that you needed to involve the users at all. If it is that simple, you really should be able to do it yourself.

Card sorting is best when you want to learn something specific and getting your users involved will help you learn it. Then, you think about what you learned and do something with it.

SB: There’s card sorting, and then, there’s card sorting. Can you tell us a little about the variations in the technique?

DS: One of the variants that people sometimes do is a card sort with their team, not users. That’s not card sorting. That’s just pushing content ideas around a table with your colleagues. That’s a great thing to do, but doesn’t tell you anything about your users.

The other main variation is to use a closed sort—one where you provide the categories as well as the content—as a way to validate an open card sort or to test an information architecture. I personally think this is really weird. Why would you get users to put content into categories and then say, “Yes, this information architecture works?” Users don’t come to your Web site to put content into categories. They come to find information. So, if you want to find out whether people can find information, you should ask them where they would look if they need information.

SB: Where do you see card sorting heading as technique?

“Get people to put stuff into groups that make sense for them. That’s about all there is to it.”

DS: When I started card sorting 10 years ago or so—wow, that’s scary—people were using it as a one-shot answer to developing an information architecture. I don’t see that so much any more. Now, I see people using it to get some general ideas about how people think and combine that with other insightful research.

I think this will continue—using it as a way to learn a bit about how people think. I also think the tools will get better, so it is easier to run and easier to analyze.

But I don’t think it will undergo any profound changes. It is too simple—get people to put stuff into groups that make sense for them. That’s about all there is to it.

SB: Why, then, write an entire book about card sorting?

DS: Even though card sorting is simple in concept, any user research method has little traps. The book goes step by step through all the detailed steps to plan and run a sort—with lots of tips based on my experiences. Then, there are two whole chapters dedicated to the analysis of results. There is definitely plenty to talk about on that alone. But it is still quite a skinny book. People should be able to pick it up as they work through their first card sort and work through it with the book.

Learn More

If you’d like to learn more about card sorting, take a look at Card Sorting: Designing Usable Categories at Rosenfeld Media.

6 Comments

I agree that card sorting shouldn’t be a structural answer to all of your information architecture, but a supporting exercise to gain insight into how your users group information.

Thanks, Steve and Donna. Great interview!

One would think that only so many things can be said about a card sort, but somehow Donna always has more. Nice!

Though I do feel sorry for Donna. It’s like all she ever gets to talk about?

Thanks, Stu & Ole, for your comments.

Ole—don’t worry about me—I talk about a lot more than just card sorting.

Hi Donna.

I’m interested in understanding how card sorting helps you architect better experiences in complex transactional Web apps, and when I say experiences, I mean the sequential page flows that allow users to interact and achieve their desired goals.

For any Web app with transactional functionality, finding the product that you want to act on in the IA structure is only the start.

Do you use card sorting to help craft better experiences ?

Hi Jason

That’s a good question. I have a couple of thoughts that relate to things in the article:

  • Card sorting helps you to learn some stuff, not give you an answer. So if you think about what stuff you want to learn, you can think about whether you think a card sort can help you with that. It doesn’t do everything. ;)
  • In your high-level example, you may be able to learn about what tasks tend to go together, then clump them together. You could also ask people about the sequence they expect to see and make sure your app follows their expected flow.
  • Card sorting won’t help you understand what people want to do once they find the product they want to act on. And you wouldn’t expect it to do that. You’ll learn that via other user research methods.
  • So yes, I use card sorting to craft better user experiences. But I don’t use it on every project. I only use it when I think it is a really good way to learn the thing I want to learn.

That was a great interview! I’m just finishing up a year long course on usability design and, unfortunately, card sorting was one of the few techniques we didn’t get to try with our test users.

I’m doing a usability test on a finished product—Web site—now, and the main problem seems to be confusion about the navigation labels. The underlying content is often subtly, but significantly different from what users expected. Writing recommendations for these problems is giving me a lot of trouble. How I wish we would have been able to bring users in on the design before the product was finalized.

Join the Discussion

Asterisks (*) indicate required information.