Home
VR/AR
After Solitary UNSUNG When Lighting The Voids What They Carried We Who Remain Ocean To Plate
Film
When I Say Africa Untitled Intimate Partner Violence Project Conflicted: The Fight Over Congo's Minerals Death In Plain Sight The Fight For Native Families Food For Nine Billion Sound Tracks Tulia, Texas FRONTLINE/World America's Investigative Reports / MSNBC Investigates
About Get In Touch

© All rights reserved

What They Carried Augmented Reality Series | McClatchy New Ventures Lab

Co-Director of a three-episode AR experience created as part of the first McClatchy New Ventures Lab Storytellers in Residence cohort. Explored creating AR nonfiction storytelling experiences for mobile with a team of artists, engineers and designers. Experimented with volumetric photo and video using photogrammetry and Depth Kit, as well as creating 3d illustrations and animations with Google’s Tiltbrush and Oculus Quill. Collaborated with users to develop best practices and toolkits for capturing and delivering mobile AR experiences.

What They Carried tells the story of migrants crossing the Mexico-U.S. border through the personal belongings they choose to carry with them.  Viewers can place these intimate objects in their homes, examine them from every side and discover the hopes, fears and stories imbued in them.

Our first episode, “Belongings,” is a tabletop experience that introduces confiscated items through photographer Tom Kiefer.

Episode two, “Crossing,” flashes back to the precarious journey across the Sonoran Desert that so many migrants make, including the Dreamer we interview in episode three — Karina Ruiz.

Using a process called photogrammetry where you take hundreds of pictures from different angles using a basic DSLR camera, and a software called Reality Capture, we were able to produce a high resolution, 3D model of each object. We loved the idea of using 3D illustrations and animations to enhance and contextualize the photo-real objects, create background environments like the desert, and give our text captions a handwritten, creative feel. 

To achieve this, we used two VR painting and animation tools, Google’s Tilt Brush and Oculus Quill, and worked with Micah 404, a talented designer and artist based in New York.

All of these elements were brought into Unity, a game engine platform, where our developer Julian Rojas assembled and scripted the diverse parts into a coherent narrative.

Using Format