For Noah Falstein, the future of virtual reality depends not only on understanding the technology and the market, but understanding the fundamental underpinnings of the human brain.

“It’s like you’re going through a mountain pass,” he said at VRDC Fall 2017 in San Francisco today, speaking to an audience made up of game developers as well as people in other industries. Right now, VR is still new territory, and on the other side of that mountain pass, a fertile valley might open up, or maybe not. “Along the way, it’s easy to get stuck,” he said.

But Falstein, who is a true believer in the future of VR, AR, and MR, says that neuroscience is the compass to point this new technology in the correct direction.

A history of conveying images

Falstein is a true veteran of game development, working at companies including Williams Electronics and LucasArts, and most recently was chief game designer at Google. He now runs his own design consultancy company, The Inspiracy.

Falstein briefly went over humankind’s quest to share images, using the example of a 20,000-year-old cave painting. “Our ancestors have been struggling with this for a long time,” he said—the desire to convey something one might personally see to other people. From cave paintings to camcorders, to smartphones with advanced cameras, to VR today, humans have been trying all kinds of ways to convey images that inform or evoke emotions in other people.

Falstein’s approach to understanding the uses and applications of immersive computing lies in trying to understand how the brain itself works and how it has evolved.

Motion

“One of the first things that comes up in VR is the potential of motion sickness,” Falstein said. “I think we’re always going to have some people at the end of that spectrum who just have trouble in VR when they move too swiftly,” he said, but there are some ways to minimize motion sickness.

Falstein explained how motion sickness is evolutionary—when a person is poisoned, it disrupts the inner ear, creating a disconnect between actual movement and the movement one feels in their head. This leads to nausea, which is a great way to throw up and purge a poison mushroom or food that has turned.

He acknowledged that there’s always a person in the studio who’s the most susceptible to motion sickness who is used as the motion sickness guinea pig. “I frankly don’t think we’ll have a better [motion sickness testing] system than that for some time to come,” he said.

He stated some well-known (among VR game devs) facts about preventing VR motion sickness: you need a fast frame rate (90+ is best); devs must minimize lag when the head moves (20ms or less); they should get all visual cues right; minimize acceleration; and come up with creative anti-sickness solutions based on how our visual field and vestibular system interact.

He also explained how blurring or eliminating peripheral vision during acceleration can help fight motion sickness. Some games and Google Earth in VR use this method, and as eye tracking systems become more advanced, users will have more comfortable VR experiences, he said.

Emotion

“It turns out that VR is really good at scaring people,” he said. Movie directors figured this out fairly early in film, and VR developers have found this out too.

“Why is horror in VR so strong?” Falstein explained how the human brain—particularly the amygdala—does the quick, raw processing of fear, anger, and aggression (fight of flight), and also arousal and intimacy.

“In video games, we’re really good at the fight or flight stuff, but the intimacy and empathy stuff, we’re still working on that,” he said.

But with VR, as it “tricks” more of your senses, there’s opportunity for more intimacy and it also appeals stronger to empathy, he said.

Falstein also talked about the possibilities in storytelling when it comes to movies shown in VR, such as short stories in VR (like Google’s Spotlight Stories series), 180-degree movie viewing, or shared or single-viewer experiences. “There’s going to be strong ‘replay’ value for things they missed [on first view],” Falstein said. There are also opportunities for monetization through ads and product placement.

Beyond Emotion

Falstein briefly pointed out how games as medicine is a new market with massive opportunity, and VR can be part of this in treating issues like phobias, PTSD, acute pain, and strokes, as well as training doctors and caregivers.

“It’s really exciting stuff,” he said, “and the future is in your hands.”