Top

New York Times Virtual Reality: How Innovation Can Simultaneously Connect and Separate

Service Design

Orchestrating experiences in context

A column by Laura Keller
December 21, 2015

I recall vividly, when I was about twelve years old, going to Disney-MGM Studios and visiting a sound-based exhibit called Soundsations with my dad. I walked into a small booth and sat on a padded seat. I was instructed to put headphones on, close my eyes, and imagine I was R.J. McBean, a newly hired executive at a major motion-picture studio. The lights dimmed to complete darkness, then, suddenly, I heard the sound of a door opening and a male voice boomed in my ear. The sound was so clear and vivid and oriented so perfectly that I felt I was the person to whom he was speaking. He walked around, going farther away, then nearer—eventually opening a refrigerator door to get a drink. He opened a can of soda, seemingly 18 inches from me. It sounded so real that I recall reaching my hand out to take it. Then, I quickly reminded myself that this wasn’t real—just amazing 3D sound.

Sponsor Advertisement
Continue Reading…

The most impressive part of this five-minute audio was when I heard someone giving me a haircut. This was part humorous and part thrilling. I sincerely believed someone was just millimeters from nicking my ear. I recall slowly moving my head away from the imaginary scissors, while my dad chuckled next to me—likely because he was doing exactly the same thing I was.

While Disney is known for its innovations and creating immersive experiences, what strikes me about this exhibit—now that I’m older and have had time to reflect upon it—was its simplicity. The usual Disney fare involves sights, sounds, touch, smells, motion—creating imaginary worlds by affecting all of your senses. But by appealing to just one of the senses, Soundsations made me believe I was R.J. McBean on my first day on the job. Beyond Disney’s speakers, there was no need for special hardware or screens or any other intermediary interface. I simply had to close my eyes.

Fast forward more than 25 years and now every industry is using virtual reality—from the military and healthcare to sports and fashion. Recently, The New York Times Magazine (The Times) launched NYT VR, a mobile app “that can be used—along with your headphones and, optionally, a cardboard viewing device—to simulate richly immersive scenes from across the globe.” [1]

The Times initially launched this app with a short film, “The Displaced,” about three children from South Sudan, Ukraine, and Lebanon who were driven from their homes because of war and persecution.

In this column, I’m going to provide my experience assessment of The Times’ NYT VR and describe whether, for me, it achieved what editor Jake Silverstein calls “an uncanny feeling of connection with people whose lives are far from our own.” [1]

My Expectations

Before I walk you through using NYT VR, it’s important that I explain what I’d expected from this experience. When using new technologies and products, I’m very focused on my needs and am often highly skeptical. This does not mean I’m not open minded, but that I have a keen self-awareness about what I’ll find valuable and what I won’t. I’ll loyally use something new if it’s a complement to the way I live—not just another thing to own or do. While this can be frustrating to others around me—for example, I’ve never joined Facebook—it makes me pretty good at what I do for a living: understanding what experiences different people will want out of a product or service.

I also recognize that, sometimes, people use new technologies just for fun, and this was my perspective when using NYT VR. But even when it’s just for fun, people, myself included, still have expectations of using a new product or service. Descriptive phrases like “richly immersive,” “uncanny feeling of connection,” and “a new form of storytelling” made me expect an experience similar to the one that made Soundsations so amazing and memorable. For example:

  • I wanted to forget where I was and feel as though I was in the story, seeing it firsthand.
  • Any intermediary user interface should feel invisible to me.
  • I should feel something for the characters in the story—feel connected to them and their world.
  • The experience should be simple and unencumbered.

It’s also important to note that The Times indicated, “You can use the app on its own. But the experience is even better with a special virtual reality viewer.” Because I didn’t have a VR viewer, I tempered my expectations slightly. The Times was telling me that, while the cardboard VR viewer is optional, it will make the experience even more impressive. [1] These expectations provide context for what I was thinking when using NYT VR.

My First Viewing: Watching “The Displaced” Without a VR Viewer

I have an iPhone. The installation of the NYT VR app was easy. I downloaded the app, opened it, and indicated whether I had a viewer. Once the film downloaded, I immediately played it and, as an image of a young boy appeared on the screen, I quickly turned off the lights in my living room. I did this because the The Times had built up the experience so much that I wanted to become as deeply immersed in the film as possible and avoid distractions.

What instantly struck me was that turning the phone afforded me a 360-degree, panoramic view of the boy’s environment. I saw both the boy and everything he was seeing in his surroundings. That was certainly impressive. And, much as with Soundsations, I found myself conflating my own reality with that of the film, so I turned my whole body 360 degrees instead of just turning the phone or my head. For the first few seconds, I sincerely felt as though I had forgotten where I was and wanted to turn my body to see more to the right or the left of where the boy was. This instinctual behavior was certainly a strong indication of the immersiveness The Times had promised.

But, after those initial seconds, white subtitles began appearing over the film to communicate what the boy was saying. Because I could turn to see a 360-degree view, the same subtitles needed to appear many times across the film’s panoramic view. Sometimes, I couldn’t read them very well if the white text was over the sky or something with a lot of detail. Pretty quickly, I began to find small usability issues with the format, and I started taking screenshots. So I stopped the film, asking myself: couldn’t I refrain from making a usability critique for just eleven minutes so I could watch the film?

On one hand, it felt ridiculous that my first instinct was to begin finding user experience flaws that marred this new technology, as well as the execution of this new journalistic approach. This was, after all, a story about three children and their experiences being refugees. Surely, my self-aggrandizing usability critique could not compare to the content of their life stories.

On the other hand, I didn’t want to start evaluating the user interface. I wanted to be so immersed in the story that font colors and text alignment wouldn’t even be noticeable. But perhaps it wasn’t fair to expect NYT VR to make me forget who and where I was. Having an immersive experience didn’t mean I had amnesia. So I decided I would start over, and, this time, I wouldn’t take screenshots. I would simply watch the film.

My Second Viewing: Take 2

On my second viewing, I was able to finish the film and, thankfully, found no additional usability issues to distract me. The panoramic style of the film-making and its ability to connect viewers to the people in the film continued to impress me. However, there’s an obvious divide between the viewers and the refugees that no technology could overcome—no matter how innovative. For example, in one scene, hundreds of families were waiting for airplanes to drop food. As I began viewing this sequence, the families were waiting behind a tape, chattering anxiously and staring up at the sky.

My natural inclination was to turn my phone, so I could see what they were seeing. They were looking up at something, and I wanted to do the same. The film-making approach and technology made me feel as though I were there. However, as the viewer, I was situated in front of the tape, not behind it, and this instantly separated me from the refugees. They were desperate for food and had to be cordoned off, while I could be trusted to stand outside the tape. So, ironically, the realism of the technology that created a connection between me and the people in the film also reminded me of just how different and separate I was from them.

When I turned the phone, giving me the same perspective as the refugees, I saw and heard airplanes approaching in the distance. As the planes flew nearer, I began seeing hundreds of white rectangles—most likely bags of grain—falling to the ground, over dozens of acres. But, as the planes flew overhead, then away from the people, there was calm and quiet.

As the viewer, I had expected to experience the chaos of people running toward me and surrounding me as they left the cordoned-off area to reach the bags of grain. However, the filmmaker had decided to re-orient the viewer’s perspective toward the middle of the area where the planes had dropped the bags. Suddenly, I saw hundreds of people running toward the bags around me. The view and its perspective were impressive, and I could appreciate just how many refugees were taking advantage of the food drop. But the film’s perspective reminded me that I was just an observer of their situation and was not really there.

Watching “The Displaced” with a Cardboard Viewer

While I had already had many thoughts about my experience with NYT VR and “The Displaced,” it was only fair for me to evaluate it using their recommended approach: a virtual reality (VR) cardboard viewer.

The recommended cardboard viewers were all intended for Android use, so I went on Amazon and found one that was recommended for all phones with screens between four and six inches. It arrived in two days. (Thank you Amazon Prime!) However, as I opened the package and began putting the cardboard viewer together, I was again struck by the irony of the situation: I was literally putting pieces of cardboard together that I had ordered for $16, using a digital app, and that arrived within just two days. I was to use this cardboard viewer in front of my mobile phone, in the hope that I would feel more connected to these people who didn’t have a home and were clamoring for air-dropped food. The simple act of acquiring the cardboard viewer reinforced my perception that the same experience that could make me feel as though I were immersed in the refugees’ environment could also make me feel just how different I was and how unconnected my life was from their life and situation.

Before allowing my cynicism to get the better of me, I reminded myself not to get too serious about this. I finished putting together the cardboard pieces and followed the instructions in the NYT VR app to indicate I would be watching “The Displaced” using the VR viewer. I quickly slipped my phone into the cardboard, so I wouldn’t miss the beginning of the film. But I fumbled a bit because there was no way to use the phone’s controls once it was in the cardboard. I kept thinking that I shouldn’t need to choose any more options to start the film, so secured the phone in the viewer, only to realize that I had to take it out yet another time. This was not the simple, unencumbered, “invisible” user interface I had expected and wanted.

Once I pressed Play, I again quickly put the phone back into the viewer one final time, again turned off my lights, and anxiously waited to see how different this experience would be with the cardboard viewer. As the film began, I at first thought I noticed a 3D effect to the film and got excited. But then, as the film proceeded, there was no 3D effect at all. The film was blurry. Straight up blurry. So blurry, in fact, that I couldn’t make it past the first three minutes. It hurt my brain. You’ve got to be kidding me? I tried everything: re-aligning the cardboard, making it tighter, and so on. Nope, nothing improved it. Perhaps it was just a poor product. I asked my friend to test hers, but it was blurry for her as well.

Then, I thought, Well, maybe because I didn’t use the setup recommended by The Times, it didn’t work. So I did some research online. I read that, Sunday subscribers to The Times had received a complimentary Google Cardboard in early November. But it turned out that even those people had issues: “In reviews for NYT VR, many iPhone users complained of seeing double. It turns out that, if the stereoscopic effect that is intended to make the videos appear as if they’re in 3D isn’t precisely configured for your phone, it can end up making people sick.”

The Times indicated they were working on the issue and, overall, NYT VR seems to be performing well for most people. “The NYT VR app currently has an average rating of four out of five stars on both app stores, so clearly it didn’t give every single one of its users a headache. Many of the people who were able to get it working were effusive with praise and astonishment at the experience.” [2]

Storytelling: The Medium Is the Message

One detail I left out earlier, when describing my experience of using NYT VR, was that, when I first started moving my phone to experience the initial minutes of the film, I said out loud, “That’s cool!” And it was. The use of technology and the film-making approach were very impressive. I had found myself thinking I was situated with the people in the film. When detailing the scene with the airplanes earlier in this column, I used language to refer to myself, not just as the viewer of the film, but a viewer who was standing outside the tape, as though I were there.

I question how genuine a connection one could truly have with the people in “The Displaced”—or in any future films The Times produces—when the technology and the necessary devices are so obviously and literally in your face. The aspiration should be for the message and story to trump the medium, but it’s obvious that Marshall McLuhan was right: you cannot separate the two.

The headline for the launch of NYT VR was: “NYT VR: How to Experience a New Form of Storytelling from The Times.” The key words are form and storytelling. I would argue that NYT VR put form first. For example, the only way I could find “The Displaced” was by doing a search or visiting the Technology section. I couldn’t find the film through content that related to the refugee crisis.

Also, when I visited the FAQs for NYT VR, I scrolled through ten questions relating to troubleshooting the technology and when new videos would be available. Then, when I reached the bottom of the screen, I read: “I’d like to make a charitable donation to help support those impacted by the refugee crisis. Does The Times have a recommended charity? Yes, visit this page for a list of recommendations compiled by Charity Navigator, a nonprofit group that evaluates American charities.” So, if the NYT VR was a success, and you felt a connection with the people in the film and compelled to take action—through the story—the information you needed to do so was trumped by the technology—the form.

Over time, I hope The Times can more seamlessly weave the VR form into their content. Once the excitement over the launch of NYT VR passes, The Times should gradually stop promoting and marketing the technology and simply let their audience experience it.

“The Displaced” actually exists as a lengthy feature article on their site, supported by numerous photos. I read the article after I had experienced using NYT VR and found that the combination of the two—getting immersed in the refugees’ world both through the app on my phone and by reading the static prose and imagery—provided a more holistic view of the refugees’ world than either accomplished on its own. It certainly will take time—and, honestly, a solid experience design strategy—for virtual reality to blend with other more traditional forms of storytelling to create the best, most immersive experience. Nevertheless, I’m eager to see that future. 

References

[1] The New York Times Magazine. “NYT VR: How to Experience a New Form of Storytelling from The Times.” The New York Times Magazine, November 5, 2015. Retrieved November 17, 2015.

[2] Leswing, Kif. “Why ‘The New York Times’ VR App Gave Some People Double Vision This Weekend.” Fortune, November 9, 2015. Retrieved November 17, 2015.

Director of Strategy & Experience Design at NTT Data

Woodbridge, New Jersey, USA

Laura KellerLaura’s 10 years of experience have focused on representing the human element in any interaction with a brand through actionable, business-impacting insight gathering and design. At NTT Data, Laura leads cross-channel experience design strategy engagements for clients. Clients have included AstraZeneca, Hachette Book Group, GlaxoSmithKline, Prostate Cancer Foundation, Honeywell, and the NBA. In addition to her Service Design column for UXmatters, Laura has written articles for the Service Design Network’s Touchpoint: The Journal of Service Design, User Experience Magazine, Communication Arts, and Johnny Holland. She has presented on service design at SDN’s Global Service Design Conference, the Usability Professionals’ Association International Conference, IxDA New York City, and IxDA New Jersey.  Read More

Other Columns by Laura Keller

Other Articles on Software Reviews

New on UXmatters