Advice for Conducting International UX Research, Part 2

Practical Usability

Moving toward a more usable world

A column by Jim Ross
April 19, 2021

Conducting international UX research in locations around the world provides some logistical challenges. Plus, there are some important differences between conducting research in your own country and conducting international UX research. In Part 1 of this series, I shared some advice on whether you should conduct international UX research sessions yourself or hire a local moderator and whether you should travel to the sessions or conduct them remotely. I also discussed hiring a local moderator and translator.

Now, in Part 2, I’ll provide some advice about recruiting participants for international UX research, how you can prepare your local moderators to conduct the research, what your local moderators should do to prepare for the research sessions, how to oversee and observe the sessions, and what information your local moderators should provide once the sessions are complete.

Champion Advertisement
Continue Reading…

Recruiting Participants

Use a good recruiting company, with a proven ability to recruit participants in your research locations. Local recruiting companies are usually most effective at reaching participants in their own country. If you’re doing research in multiple countries, it’s easier to work with an international recruiting company that has local offices in all the countries in which you’ll be conducting research. This provides the benefit of one company’s coordinating all the local offices that are doing the recruiting.

Allocating More Time to International Recruiting

It can take longer to find and recruit participants for international research, so allocate more time for recruiting in your project plan. Most recruiting companies recommend two weeks for recruiting participants domestically. So, with the additional complexity of international recruiting, plan on it taking three weeks.

Asking the Recruiting Company to Review Your Screener

Ask the recruiting company to review your screener to make sure it’s realistic and appropriate for their country. Local recruiting companies can be very helpful in adjusting your questions to fit the people in their country. For example, you might have questions that ask about income ranges in your own country’s currency. They can suggest what would be reasonable ranges for their country’s currency.

Accounting for Local Holidays

When scheduling the project, find out whether any holidays occur during the dates on which you’re planning to be recruiting or holding sessions. Holidays can reduce your time for recruiting or make certain dates unavailable for sessions. You can look up international holidays online, but you should also ask your recruiting company about whether there are any dates you should avoid because of holidays. They can tell you which holidays are minor, so wouldn’t affect your research, and which are major holidays that you should avoid.

Scheduling the Sessions Appropriately

Either the recruiting company or a local moderator can advise you on what would be the most appropriate session schedule for each country. Schedule the sessions at times when your participants would be readily available, even though those times might not be convenient for your observers.

Also ask the recruiter or a local moderator to suggest an optimal session duration. In some countries, participants and moderators tend to talk more or need more time to build rapport, so scheduling 90-minute sessions might be more appropriate than 60-minute sessions.

Be sure your schedule includes the appropriate amount of time between sessions, so it won’t get thrown off track by participants who are running late. Different cultures have various expectations about punctuality. In some countries, participants arrive early; in others, they’re very punctual and arrive on time; and in still others, it’s acceptable to arrive at a time you might consider late. Regardless of participants’ intended arrival time, traffic and public transportation can be difficult and unpredictable in some countries, which can cause participants to be late.

If it’s likely that your participants might be late, schedule longer breaks between your sessions—for example, 30 minutes instead of 15 minutes. That way, if a participant is late, the session is less likely to run over the start time of the next session. Another tactic is to give participants an earlier arrival time for their session—for example, you might tell someone to arrive at 10:30am for a session that actually starts at 11am.

Following Legal and Privacy Regulations

You’ll also need to understand and follow the laws and data-privacy regulations of each country. Your recruiting company should be able to advise you about these laws and regulations because they’re equally responsible for following them when interacting with the participants they recruit. For example, in the European Union, you need to protect your participants’ privacy by following the General Data Protection Regulations (GDPR). This may require that you ensure their full name isn’t exposed and they don’t share any personally identifying information. In some countries, you can’t have participants appear on camera, so you might need to ask them to keep their Webcam off.

Preparing Your Local Moderators to Conduct the Research

Even when you hire one or more local moderators, you’re still responsible for planning the research, creating the moderator guide, and overseeing the sessions. However, you need to familiarize your local moderators with the research and the moderator guide.

Finalizing the Moderator Guide Before Sharing It with Your Local Moderators

Avoid confusing your local moderators by continually making changes to the moderator guide. Instead, you should wait until it’s in its final state to give it to them. So, before sharing the moderator guide with your local moderators, get your client and the project team to review it, make any necessary changes to it, conduct your own pilot test to identify any problems and make sure your planned activities fit within the session time, then fix any remaining problems.

If you’re planning to moderate some sessions in your own country or language, conduct those sessions first. You or your clients might want to make changes to the moderator guide after those sessions. Make those adjustments before you give the guide to your local moderators.

Running Through the Moderator Guide with the Local Moderators

Meet with all of your local moderators individually to walk them through both your goals for the research and the moderator guide. Even if you’ve created a very detailed moderator guide that seems obvious and understandable to you, it’s always possible that your local moderators might interpret some of your questions in ways that are completely different from what you intended. Include your research objectives at the front of your guide to make sure the moderators understand the overall purpose of the research, can interpret your questions correctly, and know the right follow-up questions to ask. As you walk them through each of the moderator guide’s sections, explain each question, why you’re asking it, and what you hope to learn from it. If you’ve already conducted a pilot session or have initially conducted sessions in your own language, provide videos of those sessions to the local moderators as examples of how they should conduct their sessions.

Translating the Moderator Guide

Once you’ve run through the moderator guide with your local moderators, it’s time to translate it into each of your international participants’ languages. Some local moderators like to do the translations themselves. Sometimes the recruiting company does the translations. But some moderators like to keep the guide in its original language to avoid any translation problems that could change the meanings of the questions. Be sure to provide at least a couple of extra days in your project schedule for translating the moderator guide.

Once the translations are complete, you could optionally ask a client or project-team member who speaks the other languages to review the translated guide and ensure it’s accurate. For example, the names of technology products, applications, or Web sites sometimes have different names in other countries, but the translators might not know this.

Setting Up a Way to Communicate with Your Local Moderators

Ask your local moderators how they would prefer you to contact them during sessions. You might need to clarify something for them or have some additional questions you want them to ask participants. It’s important to let your local moderators choose whatever method of communication would be least distracting or intrusive to them. You wouldn’t want your communications to interrupt them or throw them off track. Often, a messaging system such as Slack, WhatsApp, or iMessage is best.

You should serve as the single point of contact with the local moderators. If other observers have additional questions, they should send them to you. Then you can then send the questions to the moderator when appropriate.

Preparing for International UX Research Sessions

When you’re preparing for international UX research sessions, be sure that any local moderators or translators are familiar with whatever technology they’ll need to use and that everyone can handle any technical difficulties they might encounter. You should also have local moderators conduct pilot tests.

Ensuring Everyone Knows How to Use the Technology

Make sure any local moderators or translators are familiar with the technology they’ll need to use. For remote sessions, hold a technology-setup meeting and have everyone run through everything they’ll need to do, including connecting to online meetings, recording sessions, sharing their screen, and giving participants remote control of the mouse, if necessary. Ideally, you should use online-meeting software with which the moderator and translator are familiar. Make sure that the software is also available to the participants.

Preparing for Technical Difficulties

Learn about and prepare for potential technical problems you might face. In some countries, research labs are less well equipped and might not have the same amenities to which you’re accustomed in your country. Internet connections might not be as fast or reliable—either in research labs or for participants connecting to remote sessions. Some countries such as China might block participants’ use of certain online meeting software or Web sites. Check with your recruiting company or a local moderator to ask whether such technical problems might occur.

Conducting Local Pilot Tests

In addition to running your own pilot test, which you conducted earlier during your research process, ask your local moderators and translator to conduct a pilot test with a real participant. This can help everyone become familiar with the process. Then your team can fix any technical issues that arise or clarify any misunderstandings. After the pilot test, hold a debrief meeting with the moderator and translator to discuss any changes they should make to the moderator guide or the way they run the sessions.

Observing the Sessions

If you’re using local moderators to conduct UX research sessions, your role is to observe the sessions and take notes. It’s much easier to take detailed notes when you’re simply observing and not moderating sessions yourself. List any additional questions that you want a moderator to ask the participants. However, be judicious in deciding when to communicate these questions. Ideally, you should wait until later in a session so you don’t interrupt the flow.

As a UX researcher, it can feel very strange and sometimes frustrating to observe someone else moderating your research sessions. You can drive yourself crazy by thinking about how you would have done something differently. For example: I wouldn’t have asked the question that way. That’s not what I meant by that question! Why didn’t she ask that follow-up question? However, you can’t interrupt the moderator every time you feel that way. Between sessions, you can offer the moderator key notes about how to do things differently. But don’t nitpick about every little issue you have with the way they moderated the sessions. Instead, pick your battles and focus on the most important issues. When you hire someone to moderate your sessions, you have to trust them and just let them do their best. That’s why it’s so important to be sure you make the right decision when you hire a local moderator.

Having the Moderator Provide a Summary or Debrief

Most moderators offer to provide a summary report or discuss their findings in a debrief meeting after the sessions end. You’ll probably already have very detailed notes, so you won’t need too much detail from the moderator. However, it is helpful to have a discussion about the findings and their implications, so you might gain the most benefit from having a debrief meeting.

What I find most useful is getting more information about cultural issues that I might not have noticed or didn’t understand properly. Perhaps you’ll encounter similar situations. For example, you might not have understood something to which a participant referred, or you might notice that the participants in a certain country have different reactions or behaviors from the participants in other countries. Your local moderators can be very helpful in understanding such findings and their implications.

Having Fun

Conducting international UX research can be fun and is an interesting change of pace from doing UX research in your own country. If you’re able to travel to the sessions, you can experience the excitement of exploring another country and culture. Be sure to build in at least a little time to enjoy yourself and see the local sights. Even if you must conduct the sessions remotely, you’ll still be able to talk with or hear from participants in other countries. While international UX research does require some additional work and can sometimes be challenging, the rewards are worth the extra effort. 

Principal UX Researcher at AnswerLab

Philadelphia, Pennsylvania, USA

Jim RossJim has spent most of the 21st Century researching and designing intuitive and satisfying user experiences. As a UX consultant, he has worked on Web sites, mobile apps, intranets, Web applications, software, and business applications for financial, pharmaceutical, medical, entertainment, retail, technology, and government clients. He has a Masters of Science degree in Human-Computer Interaction from DePaul University.  Read More

Other Columns by Jim Ross

Other Articles on Usability Testing

New on UXmatters