Top

Transnational HCI: Understanding Immigrant Web Usage

December 21, 2015

Immigrant Web users comprise a huge demographic for User Experience. According to the United Nations, there are more than 230 million immigrants worldwide—more people than the entire population of Brazil—and according to the United Nations International Migration Report 2013, the numbers are projected to increase in the 21st century.

Mobiles phones and the Internet help immigrants to maintain ties to their homeland. As a result, immigrants develop a high degree of cultural and technological awareness. Many identify with multiple cultures at once: the new culture, their homeland, and a third, hybrid identity that is shaped by the technological connectivity between cultures. As immigrants navigate multiple societies, they find that they practice the customs and language of their culture of origin primarily, if not exclusively, in specific situations and among certain people.

The World Wide Web offers a virtual space in which immigrants can construct a hybrid culture. For example, immigrants may express their identity through a virtual diaspora community. Such a virtual community may offer discussion boards, social networking, links, and other content, bringing together the immigrant’s original culture with that of the host society in near synchronous interaction. The sites and types of interactions in which participants choose to engage depend on a complex set of factors that have both cultural and functional import.

Recent research in usability attests to a growing recognition of the importance of understanding how migration affects and is affected by networked technology. HCI (Human-Computer Interaction) researchers Irina Shklovskii, Janet Vertesi, and Silvia Lindtner argue that cross-cultural interconnectedness across the globe necessitates a new approach to usability, which they call transnational HCI. Transnational HCI is the application of knowledge, practices, and methods from the field of Human-Computer Interaction to the analysis of immigrant users or users who routinely interact electronically across cultures that are separated by distance. At the heart of transnational HCI is a user base that has a flexible, multi-faceted sense of cultural identity. Transnational HCI focuses less on anthropological, rhetorical, historical, and sociological factors—the why of cultural interaction—and more on how users interact with technology and each other to accomplish specific tasks.

In this article, I’ll demonstrate how transnational HCI can guide usability for immigrants through data that I gathered during a study of Russian-American users on the World Wide Web. According to Ameredia, the Russian-American population represents the second largest ethnic market in the United States—behind Mexicans—comprising 10.3% of the total foreign-born population.

As people with footholds in two or more cultures and high degrees of cultural awareness, Russian-Americans regularly use a variety of Web resources having different cultural identities to accomplish specific tasks. According to ISO standards and UX methods, to what extent and in what ways do multiple cultural identities affect user interactions among Russian-Americans? In what ways could the type of study that I conducted inform a transnational HCI approach? And finally, what do the results of my study suggest UX professionals could do to better meet the needs of immigrant users?

Measures of Usability

In my study, I applied ISO-based, usability-testing principles to Web sites for the Russian-American immigrant user group. According to the ISO 9241-11 definition of usability, testing measures three things:

  • effectiveness—the accuracy and completeness with which users achieve specific tasks
  • efficiency—the number of resources, including time, that users expend in completing their goals
  • satisfaction—the degree to which users feel an application, service, or site meets their hopes and expectations

In my study, I compared measures of effectiveness, efficiency, and satisfaction across the cultural domains of the Web sites.

Effectiveness

I measured effectiveness in terms of how often users correctly completed the tasks. Using this measure, user performance was best on the Russian sites, with 93% of participants successfully completing tasks. In second place, the US sites had a completion rate of 87%, while the completion rate for the Russian-American sites was 80%.

Efficiency

Often, by setting task goals prior to usability testing, it’s possible to establish optimal completion times for the tasks. For example, a development team may decide that it wants users to be able to complete tasks within a certain timeframe or may ascertain an optimal completion time from previous studies. Because of the scarcity of prior research among this immigrant group and the fact that the Web sites were not developed specifically for this study, it was not possible to establish optimal completion times for tasks.

As an alternative, I used the lostness metric to determine task efficiency. As Tom Tullis and Bill Albert discussed in their book Measuring the User Experience, lostness measures the optimal number of page visits that are necessary to complete a task against the total number of pages a user has visited. Task efficiency depends on the extent to which a user finds an optimal path to information. The lower the lostness score, the less lost the user felt on a Web site. The lostness data I’ve provided here includes only successfully completed tasks.

Users were most efficient with the Russian-American sites, which had a lostness score of 0.14. The Russian sites were next with a score of 0.19; followed by the US sites with a score of 0.21. Participants were more efficient using the Russian-American Web sites than the US sites.

User effectiveness correlated with efficiency somewhat among the Russian-American and Russian sites, indicating that the more users successfully completed tasks, the more efficient they were. For US sites, this correlation was weaker, though still positive, which may indicate that these sites were designed in such a way that inefficient users were still able to complete the tasks, but that this is not true of the Russian and Russian-American sites.

Satisfaction

The study rated user satisfaction in two ways. First, users comparatively rated their satisfaction with a site against the other sites, on a scale from 1, most satisfactory, to 3, least satisfactory. Users rated the Russian sites the best, with a score of 1.7, followed by the US sites, at 1.9, and the Russian-American sites, at 2.1. Users’ satisfaction rankings for Web sites strongly correlated with their effectiveness measures, indicating that the more successful users were in completing tasks, the more likely they were to rank a site highly. No significant correlation existed between user ranking and efficiency.

Second, I rated user satisfaction according to the frequency and content of open-ended user remarks. I subdivided comments into positive and negative categories. Users contributed 305 comments about the Web sites. I totaled the positive and negative comments, then calculated a percentage grade against the total number of comments for each cultural category. The US Web sites had the highest proportion of positive comments, at 75%; followed by the Russian sites, at 50%; and lastly, the Russian-American sites, at 38%.

Table 1—Comparison of Web sites’ study outcomes
  Russian-American Russian American

Effectiveness

3

1

2

Efficiency

1

2

3

Satisfaction

Rank

3

1

2

Number of Positive Comments

3

2

1

Note—1 indicates the best in a category; 3, the worst.

I further analyzed the comments, determining that they fell into three primary categories;

  1. organization—including navigation, findability, and technological quality
  2. content—including the aesthetics, quality, and integrity of the content, the prevalence of advertising, and its relevance to the purported topic
  3. cultural sensitivity—including cultural markers such as artwork and the provision of Russian or English language

The US Web sites received the highest proportion of positive comments, with 81% for organization and 69% for content; followed by the Russian Web sites, with 47% for organization and 42% for content; and last, the Russian-American sites, with 19% for organization and 39% for content. The Russian-American and Russian Web sites had the highest number of positive comments for cultural sensitivity, with 79% of comments being positive.

Friend Advertisement
Continue Reading…

Analysis of the Results

The results of the study suggest that cultural sensitivity plays an important role in usability for Russian-American users—to the extent that it allows them to overcome a site’s structural design deficiencies, as identified by users. When combining and averaging sites’ scores for effectiveness and efficiency, the Russian sites were in the top ranking, with a relative ranking of 1.5; followed by the Russian-American sites, at 2; and in last place, the US sites, at 2.5.

The participants were most effective and efficient on sites with only Russian content. Users also rated their satisfaction with the Russian sites most highly, in comparison to the other sites. Once I’d broken down and categorized the comments, it was clear that the US sites received the most positive remarks for organization and content, while the Russian and Russian-American sites received the most positive comments for cultural sensitivity.

The results suggest that, for an immigrant Web site that uses a theme from the culture of an immigrant’s origin—in this case, Russian—the content and language are more usable for these participants than for a site that they would identify as well organized—or at least, well organized according to the standards of the new culture.

One final issue regarding cultural sensitivity: to this group of Russian immigrants, cultural sensitivity means awareness of Russian society, even though they are also partially immersed in the US culture. In contrast, this group of immigrants perceives a site’s organization, layout, and content as reaching the best standard, according to the standards of US society.

Conclusions and Take-Aways

Because of the cultural sensitivity of Russian-American users, doing usability testing using transnational HCI methods requires great awareness that cultural norms are integral to the functional aspects of Web use. Thus, transnational HCI does not approach Web usability with the idea that cultural differences could meld together or become insignificant during user interactions. Rather, the opposite situation prevails: culture becomes highly significant in certain contexts of use. In fact, culture relates to user interfaces and becomes associated with specific contexts of use.

The task of a UX professional who practices transnational HCI is to understand the contexts in which culture is connected to Web use. Are there certain tasks or sites for which users prefer the new or their original language? Which types of Web usage patterns require high degrees of cultural and linguistic sensitivity? Discovering this is a time-intensive process that requires usability specialists to dig deep into the characteristics of an immigrant population. A method such as contextual inquiry would be well suited to this task. Current research about the sociological patterns of ethnic minorities is also necessary. A transnational HCI team would greatly benefit from including a team member who belongs to the ethnic group under study.

Transnational HCI also requires cultural sensitivity in the design of usability-testing protocols. For my study, screening for suitable participants based on their cultural identity proved a difficult task. For many, the idea of Russian-American identity is highly flexible. Some participants were not technically Russian, having come from former states of the Soviet Union, with large minorities of Russian speakers. Some had ambivalent attitudes toward Russian culture and society because of political tensions. Others defined themselves in religious and ethnic terms first—for example, indigenous, Ukrainian-Russian, Muslim, Tadjik, and so forth—then, secondarily, as Russian-American.

Some participants were reluctant to specify their status for fear of political or bureaucratic repercussions. Because of possible sensitivities around which participants might embrace or reject an immigrant classification, I permitted participants to define for themselves how they fit into the categories Russian immigrant or Russian-American.

Plus, many participants perceived the testing situation in ways that challenged traditional protocols for quasi-experimental usability testing. Many participants asked the test facilitator for personal advice—such as where to find good schools or recommendations for attorneys in their area—while others held religious or cultural customs that required the presence of a friend or relative in the usability lab. Transnational HCI teams may need to be flexible about usability-testing protocols to accommodate such participants and be willing to create a set of modified protocols.

Immigrant and multi-ethnic users are growing in importance and number. When usability specialists encounter such users, relying on the knowledge and use of methods that have their basis in transnational HCI will greatly facilitate the elicitation of usability patterns. 

Method of the Study

My study involved 30 participants, ages 18–55, living in a US city with a large cohort of Russian-speaking immigrants. The participants provided background information about their cultural origin and Web-use habits. I provided data-collection instruments in both Russian and English. Additional characteristics of the participant demographic included the following:

  • Participants’ average age was 28.3 years old. Gender distribution was even.
  • A majority, 93%, of participants identified themselves as having been born in Russia or a Russian-speaking country and had left that country on average 5.5 years prior to the study. The remaining 7% of participants withheld identification of their country of origin.
  • All participants indicated that they were fluent Russian speakers, and all knew some English. Other languages that participants spoke included Ukrainian, Byelorussian, Hebrew, Armenian, German, Kalmyk, and Turkish.
  • All participants regularly participated in Russian-themed activities in the United States. The most popular activities included shopping at Russian stores, reading Russian newspapers and books, and social activities.

Participants completed a series of Web-search tasks involving six different Web sites. Two sites were in Russian and created by Russian developers for Russian users—gov.spb.ru and cooking.ru; two were in English and created by US developers for US users—denvergov.org and allrecipes.com; and two were developed by Russian-Americans for Russian-Americans—russianfoods.com and russiandenver.com. In addition to having a particular cultural identity—Russian, Russian-American, and American—the sites in each of these groups belonged to two different topical domains: city-government services and food/recipe sites. I chose these domains because they were familiar to all participants.

These were pre-existing, or organic, sites rather than sites that were designed and developed specifically for the study. My intent was to increase the sites’ cultural credibility with the participants, as well as to provide Web sites that the participants might actually use in real life. Three independent Russian-American Web professionals assessed the sites, using Jeff Sauro’s System Usability Scale. After conducting several hundred evaluations using this scale, over many years, the average score was 68. During this study, expert evaluators assessed the Web sites as having a score of 69, indicating that their quality was consistent with that of typical Web sites.

One complication that using real sites would have presented was that they change frequently. To mitigate this problem, I mirrored the sites on a test-bed server, thus maintaining consistency over the study’s multiple-month duration.

References

Ameredia. “Russian American Demographics.” Ameredia, undated. Retrieved April 16, 2015.

Sauro, Jeff. “SUStisfied? Little-Known System Usability Scale Facts.” UXPA Magazine, October 3, 2011. Retrieved May 13, 2014.

Shklovskii, Irina, Janet Vertesi, and Silvia Lindtner. “Introduction to This Special Issue on Transnational HCI.” Human-Computer Interaction, Vol. 29, January 21, 2014.

Tullis, Tom, and Bill Albert. Measuring the User Experience, Second Edition. Burlington, MA: Morgan Kaufmann, 2013.

United Nations Department of Economic and Social Affairs / Population Division. International Migration Report 2013. United Nations, December 2013. Retrieved April 19, 2015.

Independent Usability Consultant

Denver, Colorado, USA

Filipp SapienzaFilipp has conducted and published research about international usability in academic and trade publications and has developed Web applications for a variety of organizations. Prior to becoming a consultant, he was a Professor of Technical Communication at the University of Colorado-Denver, where he developed and taught graduate-level courses in User Interface Design, Usability Testing, and International Web Design. He has 20 years of professional experience in information technology for business and higher education.  Read More

Other Articles on Global User Experiences

New on UXmatters