FutureTalk: Mixing Realities With AR, VR, and MR [Video]

The concept of mixed reality (MR) has been around for a while—and so has the belief that it’s about to “go mainstream” any minute now. Of course, that hasn’t happened yet. But the industry is innovating hard, pushing the technology forward in exciting ways, and edging us ever closer to the day when our physical and virtual worlds are meaningfully blended.

At our recent FutureTalk in Portland, Oregon, Paul Reynolds shared the latest news from the world of MR (sometimes known as hybrid reality, MR merges “real and virtual worlds to produce new environments and visualizations where physical and digital objects co-exist and interact in real time,” according to Wikipedia). A startup entrepreneur, software engineer, and product designer specializing in 3D/spatial platforms, Paul boasts more than 15 years of experience working with 3D software. In 2014, he joined augmented reality pioneer Magic Leap, making the leap from video games to the world of AR. And last year he co-founded Torch, a 3D content-creation platform.

Paul’s appreciation for the power of AR began in Magic Leap’s first Florida office, when he watched a tiny animated dragon fly across the room and land on his outstretched finger. His reaction? “Childlike, hysterical laughter!” That little dragon convinced Paul that this technology had the potential to give us “unprecedented access to perception.”

Paul’s talk began with some introductions and definitions, even as he warned that many of the industry’s key terms will—like “stereophonic sound” and “color TV” before them—eventually become obsolete.

VR, AR, and MR

Virtual reality (VR) refers to technology that fully overtakes the viewer’s eyesight and hearing. Right now, this is the most mature and accessible tech of the three, for developers and consumers alike. Fully immersive, VR is often powered by mobile devices fitted into special headsets, like Samsung Gear VR, or by standalone headsets like Facebook’s Oculus Rift, which are powered by external computers. Performance is improving rapidly, with issues of user discomfort and motion sickness—which were still problematic as recently as a year ago— already well on their way to being resolved. VR’s main strength, Paul said, is that it promotes activity: “It gets gamers off the couch.”

Augmented reality (AR), meanwhile, allows the user to see the physical world inhabited by digital animations. Rarely wearable, this technology is usually deployed using what Paul called a “digital view-through window” such as a tablet or smartphone. The most famous example of AR is the mobile game Pokémon Go.

Until recently, Paul explained, mixed reality, or MR, typically referred to a continuum, running from the real physical environment at one end to a completely virtual one at the other. This continuum includes AR and AV (augmented virtuality), and covers the full spectrum of technology in the field.

More recently, however, MR has acquired a more specific meaning, as modern VR practitioners have begun to use the term to refer to the practice of compositing a video camera feed with a virtual camera feed. This confusion over terminology may be symptomatic, Paul suggested, of the inevitable growing pains associated with a nascent industry.

Hacking Perception to Create Magical Moments and Magical Worlds

The effectiveness of MR technology depends, Paul said, on “hacking the perception system.” Specifically, it involves navigating the complex balance of proprioception (the sense of one’s own body) and exteroception (the sense of external stimuli) central to the human understanding of “presence.”

Early iterations of 3D displays failed to do so properly, sending alarming signals to the brain that placed the perception system on high alert. Hence, discomfort and sickness. The genius of AR and its “view-through” displays is that it allows the user to process the virtual content in a more relaxed state of perception, as if it were physical.

“True MR happens,” Paul said, “when the digital signals are sufficient to pass the checks and balances of the perception system.”

The power of MR is tangible. But with great power comes great responsibility. To hear Paul’s thoughts on how MR can (and should) be used for the greater good, see some incredible demos, and discover the crucial difference between “magical moments” and “magical worlds,” be sure to watch the full FutureTalk in the video below:

Don’t Miss Our Next FutureTalk

For more information about our FutureTalks series, make sure to join our Meetup group, New Relic FutureTalks PDX, and follow us on Twitter @newrelic for the latest developments and updates on upcoming events.

Note: Event dates, participants, and topics are subject to change without notice.