Top

Usability Testing with People on the Autism Spectrum: What to Expect

October 5, 2015

In the last year, I’ve conducted about 30 usability tests with adults and children who are on the autism spectrum. I’ve tested Web sites, online software, and games to determine whether they were usable enough for users belonging to this group. Figure 1 shows Abel, a smart, eleven-year-old boy, playing the video game Auti-mate during a test session. He’s very focused, though his favorite puppet is within reach.

Sponsor Advertisement
Continue Reading…
Figure 1—Abel testing a video game
Abel testing a video game

An Atypical Brain

The brains of people with autism process sensory information differently. As a consequence, they may have an advantage in processing vast amounts of complex, static information. On the other hand, they may have issues processing rapidly changing information—such as people moving and talking on crowded streets. In this latter case, they perceive a chaotic rush of impressions, which can make them feel overwhelmed and powerless. They need a greater sense of stability and predictability. This autistic information–processing style affects their experience and interactions with software user interfaces as well. That’s why we test with people with autism. What should you expect when doing usability testing with people on the autism spectrum—that is, people with ASD (Autism Spectrum Disorder)?

Verbal Difficulties

The think-aloud protocol poses difficulties for people with autism. Some autistic users don’t speak at all, and even those who do speak may find it difficult to figure out what a UX researcher is asking of them. Here’s an example of a dialogue between a researcher and a subject with ASD:

UX researcher: “How do you play this game?” (Blocks from Auti-mate, shown in Figure 2.)

ASD subject: “You match.”

UX researcher: “Okay. So what is the purpose of this game?”

ASD subject: “To play.”

Figure 2—A screen in Blocks, a game we tested
A screen in Blocks, a game we tested

Another fascinating thing is that some people with autism speak in movie or TV-commercial quotations. (This is called idiosyncratic language.) They will suddenly say something agitatedly, perhaps using a quotation from a movie, and you definitely get the sense that they wanted to express something to you. For example, here are some utterances of an ASD subject who is playing the Auti-mate game shown in Figure 3:

  • When he starts to get the hang of the game, he says, “And, as a military man, he is ready to continue right away! This is amazing!”
  • But, when he runs into an obstacle that he can’t get over, he says, “What, what, what? Oh no, oh no. Did he put it away? He can walk. No, he can’t, but he can swim.”
  • Then, when he finds the solution and solves the level, he says, “Nivea Men, it starts with you! From Loreal, Paris. Because you are worth it!”
Figure 3—A screen from another Auti-mate game that we tested
A screen from another Auti-mate game that we tested

It is difficult to decipher such idiosyncratic use of language. You can always ask someone who knows the person really well what he or she means. They may understand because they know in what context that person would typically use these expressions.

Non-verbal Signs

Despite the verbal difficulties with the think-aloud protocol for people with autism, usability testing is very informative. It is surprising that, in usability-testing sessions, the body language of autistic people was far more telling than the body language of typical people.

The emotional reactions of typical people are usually so subtle that you hardly notice them. This is not in the case for someone with autism, so you may observe the following:

  • signs of excitement—If the person starts his usual repetitive behavior—for example, hand-flapping, rocking, or vocalizing—the frequency of these repetitive acts increases with excitement. A person with autism might start moving on his chair as though it were very uncomfortable.
  • signs of focus—The person may be completely silent and remain still and unmoving. It could seem as though he has shut out the outside world.
  • signs of joy in success—The person might jump up and laugh out loud.

During a usability test, when typical people run into an obstacle, they quickly find an alternative approach or take a detour to solve the problem. So you need to pay very close attentionĀ to notice these glitches and pinpoint a mismatch between the user’s mental model and the design. In contrast, people with autism often have a certain rigidity in their behavior. That is, if they try to do something and fail, they just try the same thing again—and again and again. It can take a while before they’ll start looking for another strategy. So a UX researcher simply cannot miss these glitches.

Conclusion

To sum up, people on the autism spectrum have atypical ways of processing visual and other types of information. To find out whether an application is appropriate for someone who exhibits these peculiarities, you need to do usability testing.

Strangely enough, few UX researchers have ever done usability testing with people who are autistic. Out of the hundreds of applications that have been created for this group, I know of only six examples that were tested for usability! But you can do such testing with just a few extra preparations and a bit of tact. Even though many people on the ASD spectrum find it difficult to express themselves verbally, non-verbal signs can make up for that, so you will get essential usability or accessibility insights about your software. 

Product Development Coordinator and UX Researcher at Maven7 Network Research

Co-founder and Psychologist at Auti-mate

Budapest, Hungary

Zsombor Varnagy-TothZsombor has worked as a researcher in the Assistive Technology and Eyetracking Lab of Eotvos University, mostly doing research on human-computer interaction projects relating to autism. More recently, he was a UX Researcher at the design company UXstudio.hu. He also co-founded Auti-mate, which makes talent-discovery video games for autistic children.  Read More

Other Articles on Usability Testing

New on UXmatters