With Alejandro G. Inarritu’s Carne y Arena (Virtually Present, Physically Invisible) debuting at this year’s Cannes as the festival’s first-ever Virtual Reality (VR) installation, and with Kathryn Bigelow co-directing a VR documentary about elephant poaching called The Protectors, it is clear that some of the world’s biggest talents are exploring the filmmaking possibilities of virtual reality.
You might have seen (or even tried out) an Oculus Rift or an HTC Vive, but how will this burgeoning technology affect the film industry? And how can South African filmmakers begin to prepare for the challenges of working in this new medium?
Murray Walker, a VR developer based in Cape Town, explains that although a lot of money is being invested in VR hardware and software development (especially in the United States and China), so far this has largely been concentrated on the gaming industry, with “business areas like training, education, medicine and travel all attracting investment now, too.”
Indeed, part of what is so exciting about VR’s infiltration into the film industry is that the question of how exactly it should be used is still being worked out. “Storytelling in VR is so exciting to me precisely because there are no answers right now,” Walker says. “It’s really like being back in France when the Lumieres were fiddling with their cinematography and there were no rules or best practices yet.”
Walker explains that there are two main avenues for filmmakers to explore: virtual reality and 360-degree video. He explains that, “VR is made in a game engine and rendered in real-time at 90 frames per second. 360 video is not the same as VR but is also consumed via a headset. 360 video is captured by an array of cameras, anything from 2 to 16, and the images are then stitched together to make what seems like a screen that entirely envelops the viewer. Both avenues are interesting from the point of view of film, with VR emerging more from the animation angle, and 360 video from the live capture side.”
Contemporary conversations around VR in the film industry centre around the related issues of interactivity and immersivity: of the audience member being ‘surrounded’ by the media experience, and able to navigate it themselves. As far back as 2013 (in an interview with Variety magazine), Steven Spielberg declared, “We’re never going to be totally immersive as long as we’re looking at a square, whether it’s a movie screen or whether it’s a computer screen. We’ve got to get rid of that and put the player inside the experience, where no matter where you look you’re surrounded by a three-dimensional experience. That’s the future.”
Walker agrees with these sentiments and is particularly enthusiastic about the interactivity that VR could provide. In fact, he thinks that this is precisely the challenge that filmmakers should embrace: “Think about giving up control as to where you want the viewer to focus and imagine non-linear storytelling. In the context of VR, I think this is really the only viable direction film can take, to be honest. When the viewer has the freedom to look anywhere, it’s incredibly hard to get them to focus where you want them to focus.”
Instead, he feels the drive should be towards “bringing gaming technology into the equation and incorporating the ever-improving conversational AIs that are out there. That way you can create a kind of film/game hybrid that will offer a unique experience for every viewer.”
While a project like this might sound far removed from traditional filmmaking, there are already products on the market that could pave the way for such experimentation. One exciting innovation has been the development of light-field cameras (such as the Lytro Immerge), which offer new ways of digitising and spatially mapping physical spaces such as film sets. As Walker puts it, “With these cameras, even when using traditional methods of capturing action, your output will be entirely 3D – allowing the viewer to change their field of view at any point.”
There are, furthermore, some interesting physiological challenges involved with working in the medium of VR. Walker says that he has observed that “suddenly immersing someone in a different space has this stunning effect, where the viewer has this almost cognitive dissonance while the brain tries to figure out what’s just happened. The first 30 seconds or so, the viewer is really spending all their energy just orientating themselves. So I’ve found that starting them in a simple space and not having much happen until they are comfortable and paying proper attention works well. Also remember that high-end VR headsets only allow movement in a two-by-two metre area, so you can’t ask the viewer to move around too much.”
He also points out that “the sense of personal space exists in VR just like in real life. It’s pretty uncomfortable to have someone right in your face. And in VR, your face is essentially the camera, so positioning stuff at least an arm’s length away is a solid guideline.”
Finally, he draws attention to the crucial role that sound plays in the VR experience. “Sound is extremely important to immerse the viewer effectively. In VR, the viewer is spatially aware in a 360-degree sense, so if sounds don’t come from the correct place in space, it can break the immersion pretty severely.”
Battling to understand how all of these components might come together to form a coherent narrative experience, I asked Walker to describe the best example of VR film that he’s seen. “It has to be Gnomes & Goblins by Jon Favreau. It was only a demo, but I think it really shows the way that film in the age of VR will go. In it, you need to befriend a little, shy goblin. You do this by gently giving him an acorn. Move too quickly and he runs away. It can take ages to get him to trust you enough, and the essence of the experience is that he really starts to feel like he’s a living being. When you finally do win him over, he gives you a present in return. It’s a really simple concept, but it was beautifully put together – obviously entirely in CGI, made in a game engine – and it properly impressed me. I was laughing out loud thinking of the potential that this little ‘film’ demonstrated.”
Although the possibilities suggested by this kind of ‘media melding’ are genuinely exciting, Walker doesn’t see VR as a replacement for film at all.
“I think it will just be another option for filmmakers. I don’t believe traditional 2D film will be consigned to the entertainment dump anytime soon. VR is a new storytelling platform, not a direct replacement of an existing platform. For one thing, headsets are – by design – isolating, which takes the social aspect out of watching a film, which I think many people enjoy. For another, most of us watch films to relax: we don’t necessarily want to be the actor. Imagine how exhausting it would be to ‘play out’ Die Hard! I think, at least for the next few decades VR will run parallel to normal ways of filmmaking, offering an entirely different set of challenges and opportunities. It won’t suddenly make normal filmmaking and storytelling redundant.”
So even if the future of film is ‘safe’ from major disruption at the hands of VR, it will be interesting to see how South African filmmakers respond to the advent of new these technologies – especially in the light of the recent growth and continued international interest in our animation industry.