In Participants’ Own Voices: Using Quotations from User Research Ethically

By Amaya Becvar Weddle

Published: March 4, 2013

“Quotations from user-research participants are a powerful form of qualitative data. They provide invaluable perspectives, in participants’ own words, on the value and meaning of products and solutions….”

Quotations from user-research participants are a powerful form of qualitative data. They provide invaluable perspectives, in participants’ own words, on the value and meaning of products and solutions—perspectives that have a high level of credibility. Such quotations often provide stakeholders’ only direct glimpses into what happens during research sessions, as participants experience UX designs. Sometimes these influential quotations trickle into marketing and sales presentations. What better way to market and sell products to customers and partners than using words that come directly from the mouths of potential users?

As UX researchers, we have an ethical commitment to represent what transpires during a usability study in an objective manner. How we choose to record and represent what happens during research sessions has a direct bearing on a study’s outcome and influence. Therefore, it is essential that we be thoughtful about how we represent the ideas, perspectives, and impressions of study participants.

Choosing Quotations

“The process that we follow in choosing the quotations to highlight in a report, while neither exhaustive nor formal, involves a synthesis of perspectives and requires us to maintain our objectivity.”

The process that we follow in choosing the quotations to highlight in a report, while neither exhaustive nor formal, involves a synthesis of perspectives and requires us to maintain our objectivity. Two important reasons for including a quotation in a report are

  1. representativeness—We might cite one or two representative quotations as evidence of a prevailing opinion or trend. When presenting quotations in this way, we should try to ensure that many participants share the same opinion.
  2. inspiration—Sometimes participants articulate meaning in new or surprising ways. Or participants may express their emotional responses in an authentic, captivating manner. Sharing participants’ voices with stakeholders or a product team can play an important role in shaping or transforming product strategy. New ideas can spring from participants’ talking about what a product means to them, how they might imagine using the product in their life, or what they need that the product in question does not currently offer.

Refining Messy Human Speech

“People simply are not as gracefully articulate in conversation as they can be in writing. … Therefore, when transforming spoken, qualitative data into the text of a presentation, there is often a smoothing process that takes place.”

Unfortunately, human speech is rarely print perfect. People simply are not as gracefully articulate in conversation as they can be in writing. Even in written statements from surveys, participants may not always be as clear and grammatically correct as a copyeditor might wish them to be. Therefore, when transforming spoken, qualitative data into the text of a presentation, there is often a smoothing process that takes place. We may also be tempted to augment a statement by providing photos or descriptions of participants to build a story around the quoted material.

However, by adding quotation marks, we are implying a truthful replication of what a participant actually said, on the record. So, what’s the best way to balance the demand for accuracy in representation with the demand for clarity in storytelling when creating a presentation? Use representative quotations that are very close to those in the written log or video recording of a session. I realize that the need for accuracy could be at odds with the desire to produce neat, tight sound bites and concise takeaways. However, there is a fuzzy line beyond which a crafted quotation goes beyond being a cleaned-up version of a participant’s spoken statement and becomes something that is artificial; that changes from truth to fiction.

Additionally, the photos and descriptions that we choose to employ should accurately represent the context in which a participant uttered a quotation and the participant who uttered it. While it is reasonable to expect that we might alter the names of participants to preserve their anonymity, any demographic information that we provide should reflect reality.

Example Descriptions and Quotations

“Let’s look at the possible consequences of the transformation process that happens as we turn raw data into print-ready material.”

Let’s look at the possible consequences of the transformation process that happens as we turn raw data into print-ready material. To demonstrate these consequences, I’ll provide three examples that were inspired by actual events.

Example 1: The Clubber

This example is from an on-site study. The participant was a 23-year-old male. We asked him to view a series of videos on a mobile device, then respond verbally to an online questionnaire asking about his impressions. A moderator, who was present during the latter part of the study, asked him to comment generally about his experience and impressions. She recorded the interview on videotape and typed some notes on her notebook computer during the interview.

Here is an example of a representative participant quotation from the interview, which the moderator transcribed into her typewritten notes:

“I felt like the video gave you the feeling you were in a club. You had all your senses stimulated. The feature enhances the enjoyability of watching video on the phone. Because the screen is small, the feature helps evoke the feeling that you’re there. This phone had better resolution. It helped give a better feeling of being there. Like the music was louder.”

The following quotation derived from the transcribed data, and the researcher added it to a PowerPoint presentation that served as a report to stakeholders:

“I felt like the video gave you the feeling you were in a club. You had all your senses stimulated. The feature enhances the enjoyability of watching video on the phone.”

And finally, here is the quotation that appeared in a published whitepaper on the research results:

“The video gave you the feeling that you were in a club. All of your senses were stimulated.” (An image of a cheering crowd in a nightclub accompanied the quotation.)

In the transformation that occurred between the transcript and the presentation, we see a trimming down of the quotation; between the presentation and the whitepaper, further trimming and refinement of the verbiage. Furthermore, we see the addition of an image that provides a visual context for the quotation. Should we be ethically concerned about the changes that transpired between the first and third iterations of this quotation?

In my view, these changes are in line with the original intent of the quotation, and a reasonable smoothing of the remarks into something more readable has occurred. Moreover, although the image accompanying the quotation was not from the video that the moderator recorded during the session, the context is similar to what the participant experienced during the study.

Example 2: The Dancer

This example is from on-site user research. The participant was a 43-year-old, stay-at-home mom. We asked her to view a series of videos on a mobile device, then answer questions about her impressions in a print survey. No moderator was present when the participant recorded her responses to the questions in the survey. Following the study, the researchers transcribed the survey responses into a spreadsheet.

Here is an example of a written response from the survey sheet:

“I remember this video from last time. I liked it more before—which really made me feel more involved and like I wanted to dance. Not so much this time though. It was a big difference in the music. It’s like the difference of making you want to dance and not dance.”

The following quotation appeared as a bullet point on a PowerPoint slide in a research report:

“It was a big difference in the music. It’s like the difference of making you want to dance and not dance.”

Finally, here is how the quotation appeared on a printed sheet of collateral for a customer event:

“It’s like the difference between making you want to dance and not dance.”—Kimberly, 19 (An image of two young girls looking at a tablet device appeared beside the quotation.)

In the transformation that occurred between the survey responses and the report, we see a paring down of the quotation; between the report and the print collateral, further trimming of the text.

More important, we see an image in which the subjects belong to quite a different age demographic from the participant, and the text identifies the participant as a teenager. Both misrepresent the potential appeal of the technology to a younger audience. Also, as a researcher, I found it quite striking that this technology compelled a more mature, stay-at-home mom to dance—a sentiment that gets lost in the final version of the quotation.

Example 3: The Skydiver

This example came from an on-site study. The participant was a male, 37-year-old operations director. We asked him to view a series of videos on a mobile device, then tell us his impressions by verbally responding to an online questionnaire. A moderator, who was present during the latter part of the study, asked him to comment generally about his experience and impressions. The moderator recorded the interview on videotape and typed some notes on her notebook computer during the interview.

This is a participant quotation that the moderator captured in her typewritten notes during the interview:

“It was like supplementing or helped me recall a true experience. It kind of brought me back to the memory of something—like driving a cool car or like skydiving. Yeah, in that last video, the—the vibration of the wind, I remember that. So it brings you back to that true life experience.”

Here is how that quotation appeared in a bullet point on a PowerPoint slide in a research report:

“It was like supplementing or helping me recall a true experience. It brought me back to the memory of something—like driving a cool car or skydiving.” (A still image from a video of a skydiver appears on the slide.)

Finally, here is how that quotation was represented in printed collateral for a customer event:

“It feels like I’m there again!”—Arthur, 22 (An image of a snowboarder jumping appears on the sheet.)

In the transformation that took place between the moderator’s notes and the report, we see a smoothing process that removed a stutter and some redundant information and made the quotation more concise. Stuttering is common in human speech, but in print, text is easier to read when we remove it. However, in the final transformation to print collateral, the text has lost its authenticity. A statement the participant expressed in past tense is now in present tense—as though the participant were describing what is currently happening. The collateral material identifies the participant as belonging to quite a different age demographic, and the imagery accompanying the quotation presents a different visual context from that the participant experienced during the study.

Best Practices

“A useful approach to handling quotations from user research comes from another discipline that often faces the same dilemma: journalism.”

A useful approach to handling quotations from user research comes from another discipline that often faces the same dilemma: journalism. Every day, journalists must accurately represent the statements of their interview subjects while rapidly producing perfectly written representations of broader human opinion.

An article by Killenbery and Anderson [1] suggests a practical test for alleviating the multitude of ethical dilemmas facing journalists in the handling of quotations. They suggest adapting Sissela Bok’s test of publicity, [2] which asks journalists to willingly imagine themselves under scrutiny by asking: Is my behavior such that I’d be comfortable having all of my audiences—interview subject, editor, public, and self—simultaneously monitor my choices, knowing that someone might later ask me to justify my actions? 

Asking ourselves how research participants might feel upon seeing their quotation in print is quite useful. However, while a journalist is actually quite likely to have this happen, UX research participants rarely see the outcomes of studies in print. As a consequence, some might not feel compelled to observe the same level of professional rigor. But we should. What if a participant did see that presentation? How would that person feel? Would he respond with indignity—or tacit acquiescence? Would she feel empowered and glad to have her voice heard? As a researcher, when speaking for the user, I hope my work holds to the highest standard.

References

[1] Killenberg, G. Michael, and Rob Anderson. “What Is a Quote? Practical, Rhetorical, and Ethical Concerns for Journalists.” Journal of Mass Media Ethics: Exploring Questions of Media Morality, Volume 8, Issue 1, 1993.

[2] Bok, Sissela. Lying: Moral Choice in Public and Private Life. New York: Vintage Books, 1979.

Join the Discussion

Asterisks (*) indicate required information.