Storytelling, and its advancements, has mostly been in the pursuit of making things seem more and more real, to immerse the listener in an experience that seems lived, to have them believe what they see on a screen is actually happening, or can happen. Artworks using Virtual Reality and Augmented Reality often succeed in breaking the fabled fourth wall and helping audiences embody the experience of going through what is on display. Naturally enough, the documentary form—which often seeks to create empathy in viewers—finds a home within these mediums, moving away from traditional 2D formats, where the space between the screen and viewer is often underutilized. If the recently concluded Tribeca Immersive is anything to go by, documentarians are increasingly taking to these formats to lessen the unclaimed space that lies between the audience and the message.
Ashraki Mussa Machano and Steven-Charles Jaffe’s Zanzibar: Trouble in Paradise is about female seaweed farmers in Zanzibar who pivot to sponge farming because of the impact of climate change. It is one of the first holographic films ever produced in the world, and the first ever to be displayed on a Looking Glass display. What this translates to, is that the film of course appears in 3D (without glasses) but also responds to the distance between the screen and the viewer—as one moves, the size and angle of objects change. “We are so confident about the story, the characters, the themes, we know that if this were to be a 2D film, it would have an impact,” explains Jaffe. “But the thing is, The holographic display gives you depth. We were interested in something that could, even if through an illusion, give people a feeling of being in the same space, literally, as the people in a film.” Hindu and Nasiri, the two women in Zanzibar, speak to the camera beginning with, “My mother was a seaweed farmer,” and as one walks closer and further away from the screen, one gets immersed in the depths of the ocean that surrounds them, always under the threat of climate change and exploitation brought in by the booming tourism industry. “Zanzibar is really a paradise,” says Machano, who is from there and has, for years, worked as a photographer, “And these women depend on the ocean, which has provided for their livelihood. We use the ocean as our playground but access to this same ocean has now been denied to us.” Of course this could’ve been another documentary on climate change but Machano insists that this technology was the best means of telling the story of these women in their own words. “Technology is like an ocean,” he says, “It divides people but also connects them. It gives me the chance to tell stories about my community to people in New York.”
Tani Ikeda and Michelle Kumata’s AR project Emerging Radiance, which won the Best Immersive prize, honors the Nikkei farmers who lived in Bellevue, Washington, most of whom had family members being detained in concentration camps during World War II. Using Spark AR Instagram filters over Kumata’s hand-painted murals of three survivors—Toshio Ito, Rae Matsuoka Takekawa and Mitsuko Hashiguchi—Emerging Radiance helps the murals come to life as we hear them (in their own voice) relate memories of World War II and their farms. “It allows the people to speak, allows our community to not just be beautiful background images but also gives people political context to our storytelling, to who we are,” explains Ikeda. The voice recordings are a part of her father Tom’s massive archive documenting Japanese American histories in the country—perhaps the largest one in the world. The people on the murals were people Ikeda’s grandparents knew, people whose predecessors she grew up with. The seed packets that were handed out at the venue all contained heirloom seeds to crops the Nikkei farmers grew. While she works on a related documentary film, Ikeda thought AR to be the best medium for this project because it made the story accessible to folks (especially younger audiences) who already have a basic understanding of the technology through filters on Instagram. “The idea was to have people use the QR code on the murals and experience the stories of our elders, and then share the stories,” Tani Ikeda says. “That way people can interact with the art, remix it, and then recreate with their own experiences. This is a new way of utilizing technology to try and make a lasting impact which people can adapt to.” It is an intergenerational project, a family project rooted in a very specific history but, as Ikeda says, by making the protagonists more “real” to the people, “it brings together people from different communities to share their histories of racism, experiences of incarceration. These stories are deeply interconnected. This project helps me use my community’s resilience to create a roadmap to justice that other communities look at and draw up for themselves.”
“A lot of people call VR the empathy machine,” says Antonia Forster who, with Thomas Terkildsen, created another nonfiction VR project, LGBTQ+ VR Museum, that premiered at Tribeca. For the queer community that has been denied a sense of material history, this project creates a crowd-sourced VR museum of objects that illustrate an important part of people’s personal queer histories—for some, it’s a teddy bear, a James Baldwin book, a pair of wedding stilettos, a bottle of nail polish, and so on. “I have very strong feelings of not overstating that VR can induce empathy,” Forster continues, “But, done carefully and done correctly, it does give you the ability to put yourself in someone else’s shoes in an almost literal sense. That sense of presence really has a lot of impact on people, and I think you can use that for good.” For Forster, it was important to have the museum be self-curated by the queer community. “It was very important for me not to fall victim to my own bubble. Because I’m white, female, cis-gendered. So, there was a very high statistical risk that I would end up sourcing more stories like mine,” she adds, before explaining the very difficult process of procuring objects (through Facebook groups and Twitter threads), creating 3D scans of them, and then working with the contributor to create a version everyone was happy with–all during a lockdown. As documentaries step into this other reality, it is heartening to notice questions of ethics and duty of care being addressed, organically.
For Thomas, who is a psychologist by training, it was important to be able to measure the viewers’ emotional states through psychophysiological measures. “So I attach a bunch of sensors and then I use that data to teach computers to recognize human emotions. This kind of physical presence and experience of immersion is central to creating emotions and emotional activity in people. It requires that we feel like we are physically there and we have something at stake. And the best way of doing that is putting our actual bodies inside the story,” he adds. As one “walks” through the museum, walking through various exhibits, announcements tell you when the museum is about to shut. If you are too tired of walking around, there is a courtyard for you to “sit.” Although in a parallel world, LGBTQ+ VR Museum is very much a “real” museum, with, as the creators tell me, a rolling roster of exhibits that get changed from time to time. It was even accompanied by a biometric experience that measured the viewer’s emotional quotient in real time. Both the creators are keenly working on taking the Museum to audiences worldwide, and the fact they can shorten or expand the project, depending on logistics and constraints, makes it easily portable.
“I don’t think one technology is exclusively better than another. And none of that is more important than good content. A good story has to be the first and foremost reason you make anything—from painting a photograph to a documentary,” says Jaffe, finding a resonance in Ikeda. “Telling our stories allowed our community to get reparation, so no matter what the medium is, I will continue to solidify my belief that all our stories are valuable,” she adds. Documentaries, and storytelling in general, strive to represent people—like the Nigerian gay man at the LGBTQ+ Museum who cried because he had never seen a tactile representation of Nigerian queer history. Documentaries drive us to draw parallels, connect dots, and then learn from these connections. As long as we continue doing that respectfully, we can never have enough mediums that help us forge these connections.
Bedatri D. Choudhury is Managing Editor of Documentary magazine.
