Johannesburg-based branded entertainment creators, whatwewant, who have been making extensive use of fast-growing virtual reality technology to weave highly immersive new content, expound their ideas on the future of the medium.
In cinema, the relationship between the filmmaker and the audience member has been integral to justifying the art’s existence. Without it, any given piece would be meaningless, let alone be able to thrive past itself in the interpretation of the filmmaker’s statement by the audience, who interpret, magnify and in some instances create meaning…
But the landscape of this exchange is radically shifting, or not, with the fruition of virtual reality as a new storytelling medium. We’ll elaborate on why it’s a radically different means of communication in a moment, but it’s important to understand how virtual reality got here, and why it took so long to come into its own.
Humanity’s endeavour to replicate the process of stereopsis, the brain’s mechanism of merging two separate images from two inputs, has been active since Sir Charles Wheatstone’s invention of the first stereoscope in 1838. The simulation of binocular depth perception, where two static images were reflected at an angle of 45 degrees, was an important first step in our pursuit to make stories more and more immersive. This technology has evolved throughout the past century by slipstreaming behind the exponential curve of tech development. From Ivan Sutherland’s gargantuan device in 1965, the Ultimate Display, arguably the beginning of augmented reality, to the crude View Master that you used as a child, and of course, cinema, we have arrived at the point where numerous technologies have converged to create the format of 360 video.
So why the hype? Why has this tech accumulated $4 billion in funding since 2010? And why did AR and VR startups see a 347 per cent increase in funding by venture capitalists in the first quarter of this year alone? We’re arguing that immersion is key to this, not to mention the potential applications for virtual reality across the socioeconomic landscape. It’s disrupting everything from training to therapy to tourism, let alone entertainment media, where it’s finding itself in the pockets of more and more consumers, thanks to the exponential tech inside your smartphone.
The power of immersion cannot be understated. In well-executed experiences, you’ll find participants using personal pronouns, exclaiming, “wow, I’m here…” or, “wow, that character is talking to me.” The replication of bioscopic vision has bridged the divide between the viewer and the filmed world. They are a part of it. The power of this in fictional storytelling has put stars in the eyes of storytellers. Erroneous assumptions about the ‘disruption of cinema’ have been flung about, with some even imagining the cinema of the future, where viewers will don head mounted displays as opposed to sitting in cinema seats. The problem here is that the popularity of immersive 360 video content has hoodwinked storytellers into assuming that virtual reality is the evolution of cinema, whereas it’s actually the evolution of gaming. As virtual reality experience designers, we’re arguing that cinema and 360 video are not the same thing. Key to this argument is the role of the protagonist and the role of 360 video as an experience based platform.
Cinema has spent more than a century developing the sophisticated meanings and associations of film grammar in order to show an objective view of a narrative world. Through the exercise of empathy, sympathy and the vicarious experience of narrative conflict, audience members interpret the filmmakers’ meaning and the protagonist is key to this engagement with the visual medium. Few mediums can do this as well as cinema can. Even games integrate cinematic cutscenes in order to piggyback on cinema’s prowess and create an emotional connection between the player and the characters they control (or simply as an expositional tool).
If we contextualise cinema as objective, and virtual reality as subjective (try to ignore the fact that filmmakers use the term ‘subjective camera’ to describe a character’s point of view in a scene), we can start to dissect why filmmakers are struggling to match cinema’s narrative principles in virtual reality experiences, because in VR, you navigate the 360 sphere autonomously, whereas in film, you are guided through the world by a director.
Let’s use Disney’s virtual reality experience, Through Mowgli’s Eyes Part II: Cold Lairs to contextualise this argument. In the 2016 remake of The Jungle Book, Mowgli meets the fearsome King Louis, a gargantuan orangutan, that tries to strongarm the mancub into creating man’s red-flower (fire, for the uninitiated). Jon Favreau demonstrates Mowgli’s vulnerability in the scene because he is unable to create the red flower. The viewer sees his impotent cry for help, his desperation as a helpless child. Whereas in the 360 video experience of the same scene, you are Mowgli and all you see is King Louis and the surroundings. That’s it. There’s no drama. As a companion experience to the film, the VR experience gives us an appreciation for the ape’s scale as he encroaches uncomfortably close to us with a wild, greedy glare in his eyes. It makes the viewer feel small and vulnerable. It’s powerful stuff because it’s immersive and real, but in isolation, it’s dramatically impotent. Why? There’s no protagonist. The absence of the protagonist and the antagonist interaction means that there is no conflict. Some would argue, “but you are the protagonist,” when you actually are not. You’re not even a participant, because the experience is passive, not interactive. In 360 video you’re simply a viewer. A viewer and a protagonist are two very different things. As a result, the experience feels gimmicky, utilising one of the virtual reality experience designer’s favourite tools – discomfort.
Currently, VR experiences are dominated by characters glaring at the lens. Think of what happened when 3D first came to audiences (also enabled by the process of stereopsis, by the way). It didn’t take long for a director to bounce a berry off of Dwayne Johnson’s popping pec, right at the audience, in Journey 2: The Mysterious Island. Hell, even the title’s a gimmick. This is symptomatic of storytellers’ attempts to try inject a sense of emotional engagement into 360 video experiences. This ‘Being John Malkovich’ effect doesn’t get any more engaging when you’re being driven by the character (when the camera moves). In fact, as virtual reality experience designers, we’ll tell you how problematic it becomes because of stitch lines and nausea. Simply put, filmed 360 video content is not at the point where you can strap a rig to Optimus Prime’s head without making your audience chuck their cookies. Furthermore, the whole point of virtual reality is for the viewer to autonomously navigate the 360 canvas, commanding their field of view by changing the direction of the character’s head kind of defeats the purpose.
Some VR experience designers have sidestepped these problems by simply placing the viewer in the scene, allowing them to objectively observe character interactions. But that’s stepping into the Rhineland between cinema and VR, which is problematic because of the fundamental differences between the mediums. Thanks to the century-old crafts of editing and cinematography, the objective view of a narrative world is cinema’s domain. There’s no close-up in VR. This is why the relationship between the audience and the storyteller is shifting. In VR, the experience designer hands over control to the viewer. Although the experience designer may tempt the viewer to certain points in the 360 sphere, they must relinquish control of the field of view to the viewer, not dictate it, as is the case in film.
Having said that, there’s no question that VR is establishing its own domain of powerful storytelling. Chris Milk famously dubbed VR the ultimate empathy machine when he presented a United Nations VR experience to members of the World Economic Forum. This experience was called Clouds Over Sidra (available on the Within app and website), where a young Syrian girl guides you through the Zaatari Refugee Camp in Jordan. He remarked on how unprecedented their reactions were, saying in a Ted Talk that they were affected by it. As a window to other people’s worlds, virtual reality is driving transformation as a medium of documentation, because you can be present in the subject’s bedroom. The power of documentaries lies in the visual testimony, and the immersive power of virtual reality is only enhancing the medium’s power.
Virtual reality is however, the cheese to gaming’s macaroni. True immersion in a fictional world is what the first person shooter subgenre has been trying to do for 20 odd years. Driving a character, as opposed to being driven by a character in a passive 360 video, allows the participant to have a direct, autonomous impact on the forces of antagonism in the narrative world. In this way, the dissonance with the function of the protagonist that film is experiencing is not present in gaming, and the digital world has only become more real because of VR’s immersive power. Jon Favreau is one of many film directors that are creating interactive experiences in virtual reality with Gnomes and Goblins, an experience where the viewer forges a friendship with a tiny, inquisitive goblin by helping it forage for food. An immediate emotional connection is created when you help this tiny character get a peach that’s too out of reach (our creative developer had to jump to reach the peach because he was too short). The use of the HTC Vive’s sophisticated calibration mechanisms allows you to feel truly present in the Goblin’s fantastical forest home, because your distance from every object in the 3-Dimensional environment is measured relative to the space your physical body is in. This demonstrates how there are degrees of interactivity or immersion in virtual reality when more and more sensory stimuli are activated. Some experience designers, such as the pioneers at the VR theme park The Void have used haptic feedback technology and environmental effects to expand on this idea.
Ultimately, 360 video is in its infancy. Although we’re skeptical about some of its narrative applications, we’re eager to see how this powerful storytelling tool will grow and become more accessible as more consumers became aware of its potential and increase its popularity.
You may wonder why we have such a big opinion on virtual reality. As content creators specialising in exponential technologies, we’ve been introducing the market to virtual reality for the past 18 months and have found success in experiences that operate outside of the mass market. In this small interim before everyone with a smartphone has a VR headset, such as the Google Cardboard, the Samsung Gear VR or Google’s up and coming Daydream VR, the only equitable way for marketers to get audiences interacting with virtual reality is to take it to them. In below the line activations, sales and training, we’ve produced work for organisations such as Internet Solutions, Vodacom, Sun International and more.
By whatwewant
