Usability Testing with People on the Autism Spectrum: What to Expect

Verbal Difficulties

The think-aloud protocol poses difficulties for people with autism. Some autistic users don’t speak at all, and even those who do speak may find it difficult to figure out what a UX researcher is asking of them. Here’s an example of a dialogue between a researcher and a subject with ASD:

UX researcher: “How do you play this game?” (Blocks from Auti-mate, shown in Figure 2.)

ASD subject: “You match.”

UX researcher: “Okay. So what is the purpose of this game?”

ASD subject: “To play.”

Another fascinating thing is that some people with autism speak in movie or TV-commercial quotations. (This is called idiosyncratic language.) They will suddenly say something agitatedly, perhaps using a quotation from a movie, and you definitely get the sense that they wanted to express something to you. For example, here are some utterances of an ASD subject who is playing the Auti-mate game shown in Figure 3:

• When he starts to get the hang of the game, he says, “And, as a military man, he is ready to continue right away! This is amazing!”
• But, when he runs into an obstacle that he can’t get over, he says, “What, what, what? Oh no, oh no. Did he put it away? He can walk. No, he can’t, but he can swim.”
• Then, when he finds the solution and solves the level, he says, “Nivea Men, it starts with you! From Loreal, Paris. Because you are worth it!”

It is difficult to decipher such idiosyncratic use of language. You can always ask someone who knows the person really well what he or she means. They may understand because they know in what context that person would typically use these expressions.

Non-verbal Signs

Despite the verbal difficulties with the think-aloud protocol for people with autism, usability testing is very informative. It is surprising that, in usability-testing sessions, the body language of autistic people was far more telling than the body language of typical people.

The emotional reactions of typical people are usually so subtle that you hardly notice them. This is not in the case for someone with autism, so you may observe the following:

• signs of excitement—If the person starts his usual repetitive behavior—for example, hand-flapping, rocking, or vocalizing—the frequency of these repetitive acts increases with excitement. A person with autism might start moving on his chair as though it were very uncomfortable.
• signs of focus—The person may be completely silent and remain still and unmoving. It could seem as though he has shut out the outside world.
• signs of joy in success—The person might jump up and laugh out loud.

During a usability test, when typical people run into an obstacle, they quickly find an alternative approach or take a detour to solve the problem. So you need to pay very close attention to notice these glitches and pinpoint a mismatch between the user’s mental model and the design. In contrast, people with autism often have a certain rigidity in their behavior. That is, if they try to do something and fail, they just try the same thing again—and again and again. It can take a while before they’ll start looking for another strategy. So a UX researcher simply cannot miss these glitches.

Conclusion

To sum up, people on the autism spectrum have atypical ways of processing visual and other types of information. To find out whether an application is appropriate for someone who exhibits these peculiarities, you need to do usability testing.

Strangely enough, few UX researchers have ever done usability testing with people who are autistic. Out of the hundreds of applications that have been created for this group, I know of only six examples that were tested for usability! But you can do such testing with just a few extra preparations and a bit of tact. Even though many people on the ASD spectrum find it difficult to express themselves verbally, non-verbal signs can make up for that, so you will get essential usability or accessibility insights about your software.