These limitations were the well-timed subject of the International Usability Evaluation Special Interest Group (SIG). This session was packed before it even began, as attendees scrambled to find a seat at one of the five or six large round tables. As a first-timer at CHI, and one of the last to find a seat, I experienced a bout of nervousness, wondering how much group participation this would really involve. My fears were quickly assuaged once I learned how the session would progress—with the first half allotted to presentations by the panelists; the second, to small group discussions at each table.
I found the presentation by Apala Lahiri Chavan to be outstanding. She was an enthusiastic and very articulate speaker who also provided good, useful content. Chavan presented case studies on usability methods and requirements gathering that did and did not work in India. She described the reluctance of users to criticize designs for fear of offending the facilitators. Also, in certain contexts where class and/or status were factors, interviewees were reluctant to provide information. Upon encountering these and other issues, her team employed the following new strategies:
- In addressing issues of status, they realized users were willing to provide more information to younger, though less experienced interviewers.
- To address the reluctance of users to criticize user interfaces:
- They created an evaluation forum that was something like a marketplace. By allowing participants—who were accustomed to bartering—to barter, researchers elicited the type of feedback they needed. (Since this method placed participants in what resembled a commercial context, it reminded me of the 7-11 Milk Experiment that Jared Spool and UIE use as the basis of their eCommerce site research.)
- Instead of usability scenarios, they employed movie scripts and popular actors, because people in India are willing to openly discuss and criticize such enactments.
During the facilitated discussions, small groups talked about various usability methods, including surveys, interviews, usability tests, and some novel techniques; what issues they had with classic methods; where the issues related to culture, country, region, or language; and what, if any, strategies they used to overcome such issues. Thanks to the varied group of international participants, a range of issues did arise.
- One practitioner spoke of traveling from a South East Asian country to Australia to work on financial transactions. She found that, in Australia, users were unwilling to discuss banking and money in ways she had not expected.
- To address a participant’s problem with a usability test in an Asian country, the group concluded that some type of co-participant method might have elicited more articipation than a single-person think aloud test session.
I greatly appreciated the willingness of the presenters and discussion participants to share methods that had not worked for them and how, when possible, they modified them.