Virtual reality has been around longer than you know. In fact, it may surprise you to find that the concept was first referred to in the 1935 science fiction novel Pygmalion’s Spectacles by Stanley G. Weinbaum, which uncannily described a goggle-based reality wherein the user experienced a holographic recording of fictional realism, including smell and touch. Fast forward 81 years and Weinbaum’s work of fiction has undoubtedly become reality – virtual reality.
The science and technology realm has always had a loose yet distinct relationship with the dreamers of science fiction and in almost every case their dreams were only realised into the mainstream when technology was sufficiently advanced to provide the means. This often took decades, or, in the case of some of Leonardo Da Vinci’s work, centuries.
Technology today has afforded us the ability to accomplish things only dreamed about just a few years ago. From flat panel displays, to speaking to people around the globe via video calls to handheld devices such as smart phones that boast millions of times more computing power than the systems that launched the Apollo missions to the moon. In some ways we are moving toward a Star Trek world of Holodecks and brave new worlds where no man has gone before. VR is one of those worlds.
So, we now have the processing power, programming prowess, understanding of the human brain/eye/ear relationship and display technology to provide immersive, hyper-real experiences through HMDs, or Head Mounted Displays, either in monoscopic or stereoscopic views. Add rich and compelling audio into the mix and you’re there. You’re really there. This is Weinbaum’s dream.
But how is the content delivered to the user and how is it produced?
Applications and technology
Virtual reality is defined in the dictionary as “a realistic and immersive simulation of a three-dimensional environment, created using interactive software and hardware, and experienced or controlled by movement of the body.”
As of 2016 we have numerous HMDs and platforms on the market, such as the HTC Vive and the Oculus Rift, which are tethered (connected via a cable) to a PC and experienced via the SteamVR and Oculus platforms, respectively; the Sony PlayStation VR, which is console-based and tethered on the Playstation 4 platform; and the Samsung Gear VR, which is powered by Oculus and works on the Android mobile platform. There is even a super-affordable and nifty cardboard headset from Google called Cardboard that sells for a quid per unit and uses your smartphone as the display, similar to the Samsung Gear VR.
More advanced systems like the HTC Vive incorporate other features in a more comprehensive package such as a built-in front-facing camera, two motion controllers, haptic sensors for forced feedback and two base-stations for defining the space wherein you’ll be using the VR. The sensors then track your movements in a defined area instead of from a seating position. Systems such as these require massive amounts of processing power so if you’re interested in getting a PC-based VR system just be warned that HTC recommends minimum system requirements of an Intel Core i5-4590 CPU and a GeForce GTX 970 GPU. Add that to the cost of the headset itself and costs rise astronomically.
Despite most of these platforms being around for some time now the VR industry is still somewhat of a wild, wild west in terms of new ways of exploring the applications of the experience and the production of the content. What once was a novel idea has become bleeding-edge and now there are myriad applications from education and training, to fine arts and engineering. Med students are even using the technology to work on virtual cadavers instead of the real thing and strides are being made in exposure therapy to help people overcome their phobias.
At the moment the foremost application worldwide is gaming. Gamers crave the most immersive experience possible and VR provides that. In a lot of ways the gaming industry has driven the platform causing content developers in other application genres have taken a lot of cues from it, the result being that game engines such as Unreal and Unity3D are being employed in other areas to create an interactive world outside of the gaming genre. Advertising has seen a massive upturn in the amount of VR being implemented as of late and many big brands have hired VR companies to supply their customers with virtual experiences of their products. It is clear that the value of VR has finally been realised in the minds of the corporate world.
VR production
A common misconception in VR – particularly VR that has used cameras to provide a real-life experience – is that it is akin to directing a film. This cannot be further from the truth. In fact, a VR experience has far more in common with gaming so new ways have to be conceptualised to drive the user to interaction while at the same time telling an immersive story.
Locally we have a few companies that are pioneering this technology, such as Bladeworks based in Bryanston and whatwewant, based in Parkwood. Admittedly, the VR gaming industry in South Africa is in its infancy so companies such as these are leveraging the VR platform in other directions such as in advertising and exciting new platforms such as live concerts. Imagine putting on your headset and being immersed in a live concert from the comfort of your living room. This is happening. Now. In South Africa.
So where does the process start? As mentioned, this is not a traditional filmmaking exercise. An entire rethink of the approach is needed to drive the content home to the user in an effective way and that often requires mental acrobatics. A large part of that is getting over the novelty of the technology itself, which is still front and centre at this early stage.
“I always go back to the (3D) movie trailer for Journey 2: the Mysterious Island,” remarks Cole Matthews, creative developer at whatwewant. “The character throws a berry that bounces towards the viewer. That perfectly encapsulates the kitsch idea of, ‘I don’t know what to do with this new tech so I’m going to throw it in your face and it’s going to be very gimmicky.’ The majority of narrative-based VR experiences at the moment feature characters just glaring at you with incredible make-up and art direction. But because it’s an incredibly immersive medium it does feel uncomfortable but that is novelty. It’s not sophisticated story-telling yet.”
Matthews and Bladeworks VR specialist Jaco Kraamwinkel go on to reiterate that VR will take more and more cues from the gaming world as content producers start to make the experience more interactive and experiential for the user.
“The lines will start blurring between gaming and films,” says Kraamwinkel. “As soon as you start interacting with something it’s more of a crossover into gaming.”
Creative lead at whatwewant, Erik De Jager, goes on to explain that despite the tilt of the proverbial deck towards gaming in VR, the similarities to film cannot be outright disregarded but they are, however, limited.
“You’ll probably find in the production process that there are a lot of similarities between film and VR,” says De Jager. “I think the biggest difference is that the filmmaker no longer has ultimate control over where the viewer is going to look. So I think you become experience designers instead of filmmakers and that is the big difference.”
The best place to start when approaching this kind of content creation is a brief from the client. Getting your head around how to accomplish that in the 360 degree realm is the trick but generally, in the case of Bladeworks, the process starts with an array of cameras, usually around six to twelve Go Pro 4K cameras, mounted in a frame that allows the entire 360 degree panorama to be filmed. The resulting videos are then stitched together in specialised software into a 360 panorama that encompasses the entire environment. Once the video has been stitched together the entire capture has to be calibrated and smoothed due to the lighting and exposure differential between each shot.
“The alternative is to use a game engine such as Unity3D to produce the VR environment. But as both Kraamwinkel and De Jager admit, this sort of approach in a gaming and/or interactive VR experience is still in development in South Africa. For now, however, companies like Bladeworks and whatwewant remain on the cutting edge of VR content and Bladeworks is currently using game engines like Unity3D in other ways.
“We are capable of creating fully immersive VR environments with our current software compliment and our 3D team is highly skilled at that,” remarks Kraamwinkel.
Conclusion
VR is an exciting and bewildering form of content delivery, be it in the gaming world, advertising or any of the other myriad potential applications for the platform. The most exciting thing about it, perhaps, is that its full potential has not been realised yet and we are only beginning to view its full impact as it unfolds. VR experiences such as Google’s Tiltbrush, that the author was given an opportunity to experience, is one example of a revolutionary 3D painting tool for the HTC Vive that puts the user right in the middle of their creation, in real time. Once you experience something like that the mind boggles at the possibilities. It is clear that companies like Bladeworks and whatwewant are part of a select few of pioneers in this field in South Africa and we can only imagine what the future holds for them.
By Greg Bester
