News & Opinion | February 22, 2016 9:00 am

How the Heck Does Virtual Reality Work?

Five burning questions on the tech toy that will define 2016

By The Editors

Oculus Rift, Project Morpheus, HTC Vive … if you’re thinking the language of virtual reality sounds a bit like some hastily penned Battlestar Galactica lingo, you’re not alone.

But it’s a language that has nonetheless breached the chrysalis and hijacked our collective consciousness, and whether you’ve already ordered your headset or are just a curious bystander, now’s the time to learn it.

Herein: a brief but thorough primer to help you get caught up.

OK, so I’ve seen videos of people clumsily groping at thin air with giant space apparatus attached to their heads. But what the heck is actually going on? How does VR technology work?

VR consists of three basic components of VR: a PC or game console, a game or smartphone app and an HMD (head-mounted display) to project the images.

Humans have stereoscopic vision, meaning each eye processes imagery distinctly and the subtle differences are layered into one field of vision. In the headset, each eye has a separate lens matching a slightly altered image, and through the binocular combination of each image, your brain is tricked into thinking it’s perceiving a three-dimensional space.

Images are fed from the game console or PC to the headset via an HDMI cable; smartphones are typically placed directly inside the HMD. Either the headset or your smartphone has motion-trackers, so when you move your head, the sensors follow your movement.

There are two varieties of virtual reality content itself: graphics-oriented games and mobile apps that immerse you into an avatar world, and cinematic virtual reality, which places you inside a panoramic space with imagery captured from real life. Then there’s VR’s kid sister, augmented reality, which projects advanced holographic renderings into real-world spaces.

What’s difference between smartphone VR and heftier devices like Oculus Rift? And why is the latter so darn expensive?

The VR industry is essentially serving two parallel markets: the technophiles and the rest of us. Think of the watered-down smartphone version of VR as a sort of gateway drug into the heavy lifting (Oculus Rift, Project Morpheus, etc.).

Mobile VR integration is a cheaper alternative because its functionality is less sophisticated: you simply download an app and place your phone inside an HMD.

Google Cardboard is dominating the mobile sector because of an effective marketing campaign. It’s an open-source and DIY project, so developers have a lot of freedom to redesign and manipulate the model’s basic template. Samsung’s Gear VR, on the other hand, is proprietary and more expensive: you have to have one of a few specific Samsung devices to use the host of apps they’ve created.

Other open-source startups have cropped up in Google’s wake, namely Durvois, Freefly VR, and Mattel, who have repurposed their retro View-Master to accommodate the trend (sold on Apple’s platform but with no official partnership).

But the purest experiences belong to Oculus Rift, Sony Playstation VR (formerly Project Morpheus), HTC Vive and Microsoft’s HoloLens (augmented reality), all of which cost hundreds or even thousands of dollars. These models are souped-up with sensitive hand- and motion-tracking systems, and all require a high-performance PC or game console. The VR apps that have been designed for these products come with highly developed audio, visual displays and interactivity components.

HoloLens

What’s the difference between virtual reality and augmented reality?

Virtual reality isolates the user in a fabricated dimension. Augmented reality, also called mixed-reality, allows users to interact with virtual artifacts in an actual three-dimensional space.

Microsoft’s HoloLens is a new pillar of this technology, while Apple recently acquired a small California startup called Metaio. Magic Leap, a secret company in the outskirts of Florida that investors are foaming at the mouth for, is at the fore, though. They’re currently developing a “cinematic reality” with classified technology.

Then there’s Google’s Project Tango, which utilizes the same kind of computer vision without the need for GPS or other signals.

Paragliding 360 VideoParagliding in the canary islands3:46

Where can I find VR content? Obviously it’s not as easy as flipping on my TV and surfing through the channels.

There’s a bazaar of mobile applications out there to satisfy your virtual curiosity. The essential 360 pano apps to acquaint yourself with are VRSE, The New York Times’ NYT VT and Youtube — just type #360 in the search parameters.

Innovators are also developing content houses to store virtual reality videos. Vrideo already has a healthy buffet of VR content, while the heavily backed Wevr Transport is still in beta mode.

Does VR have other applications? Beyond, you know, entertainment?

VR has influenced a new frontier in training applications and other modes of entertainment.

The healthcare industry is using the technology as a diagnostic tool; 3D artifacts of CAT scans and ultrasounds are being developed as a simulation model for surgeons.

The military is strapped in as well: soldiers are placed in virtual hostile environments to learn how to respond to an injured soldier under fire, or operate a parachute, or scout a snipe shot, etc. NASA is utilizing the technology to train and potentially control robots via motion sensors on space missions, like simulating a landing on Mars.

Just Another Day in the Office at Magic Leap1:35

Meanwhile, Magic Leap, the seminal augmented reality startup, has partnered with Wevr and major Hollywood studios to develop virtual reality experiences. Sundance has even sponsored a residency program called New Frontier for nascent virtual reality filmmakers. And four VR films shot with HTC Vive were featured in this year’s festival lineup. And the sports world, specifically the NFL, is leveraging the technology to prepare athletes for on-field scenarios.

And then, of course, there’s VR sex.

You can thank Rule 34 for that.

—Eric Brown