Facial Expressions Could Make VR More Accessible and Immersive GA S REGULAR Menu Lifewire Tech for Humans Newsletter! Search Close GO News > Smart & Connected Life

Facial Expressions Could Make VR More Accessible and Immersive

Experts can’t believe their eyes

By Mayank Sharma Mayank Sharma Freelance Tech News Reporter Writer, Reviewer, Reporter with decades of experience of breaking down complex tech, and getting behind the news to help readers get to grips with the latest buzzwords. lifewire's editorial guidelines Published on February 24, 2022 11:00AM EST Fact checked by Jerri Ledford Fact checked by Jerri Ledford Western Kentucky University Gulf Coast Community College Jerri L. Ledford has been writing, editing, and fact-checking tech stories since 1994. Her work has appeared in Computerworld, PC Magazine, Information Today, and many others. lifewire's fact checking process Tweet Share Email Tweet Share Email Smart & Connected Life Mobile Phones Internet & Security Computers & Tablets Smart Life Home Theater & Entertainment Software & Apps Social Media Streaming Gaming

Key Takeaways

Researchers have devised a mechanism to control VR avatars through facial expressions.The researchers found that facial expressions created far more immersive experiences.The technique can help make VR accessible to people with disabilities.
Westend61 / Getty Images After revolutionizing biometrics, faces are all set to invigorate another technology: virtual reality (VR). In a recent study, an international team of researchers from Australia, New Zealand, and India used common facial expressions, such as a smile and a frown, to interact and trigger specific actions in VR environments, with surprising results. “Overall, we expected the handheld controllers to perform better as they are a more intuitive method than facial expressions,” noted Professor Mark Billinghurst from the University of South Australia, one of the researchers involved in the experiment, in a news release. “However people reported feeling more immersed in the VR experiences controlled by facial expressions.”

Intuitive Immersion

The researchers, led by University of Queensland researcher Dr. Arindam Dey, who works with Prof. Billinghurst at the Australian Research Centre for Interactive and Virtual Environments, argued that most VR interfaces require physical interactions using handheld controllers. In their paper, the researchers note that they set out to use a person’s expressions to manipulate objects in VR without using a handheld controller or touchpad. They devised a mechanism to identify various facial expressions, including anger, happiness, and surprise, with the help of an Electroencephalogram (EEG) headset. For instance, a smile was used to trigger the command to move the user’s virtual avatar, while a frown would trigger a stop command and a clench was used to perform a predefined action, instead of using a handheld controller to control the avatar, explained Prof. Billinghurst in the press release. Geber86 / Getty Images As part of the research, the group designed three virtual environments, two that were happy and scary and a third that was neutral. This enabled the researchers to measure each participant’s cognitive and physiological state while they were immersed in each of the three scenarios. In the happy environment, participants smiled to move through a park to catch butterflies with a clenched jaw and frowned to stop. Similarly, in the scary environment, the same expressions were used to navigate through an underground base to shoot zombies, while in the neutral environment, the facial expressions helped users move across a workshop, picking up various items. The researchers then collated the neurological and physiological effects of the user’s interaction in the three VR environments using facial expressions and compared them with interactions conducted via commonly used handheld controllers. Prof. Billinghurst noted that at the end of the experiment, the researchers concluded that although relying on facial expressions alone in a VR setting is hard work for the brain, it gives participants a more immersive and realistic experience than using handheld controllers.

A Mere Gimmick

The researchers contend that interacting with VR through facial expressions not only provides a novel way to use VR, but the technique will also make it more accessible. By ditching handheld controllers, people with disabilities, from those with motor neuron disease to amputees, will finally be able to experience VR. Even as they work to make it more usable, the researchers suggest the technology can also be used to complement handheld controllers, especially for VR environments where facial expressions are a more natural form of interaction. "Most of human communication is actually body language [and] facial microexpressions that we're often unaware of, so proper facial tracking can surely take virtual social interactions to a whole new level." Lucas Rizzotto, intrepid creator and YouTuber, told Lifewire over email. Rizzotto, whose most famous creation is a VR time machine, believes facial tracking definitely has a role to play when it comes to social VR and Augmented Reality (AR) and the metaverse, though he has his reservations about it gaining mainstream acceptance. "As far as purely controlling experiences with your face, I'm sure there's some creative possibilities here when it comes to art and accessibility," Rizzotto opined. "But it could also easily just end up being a gimmick when we have so many more reliable forms of input."
Was this page helpful? Thanks for letting us know! Get the Latest Tech News Delivered Every Day Subscribe Tell us why! Other Not enough details Hard to understand Submit More from Lifewire How Meta's New VR Headset Could Read Your Face and Bring Privacy Risks Meta Quest Pro: Price, Release Date, Specs, and News Meta/Facebook's Bet on Virtual Reality How to Set Up and Use Meta (Oculus) Quest 2 Metaverse vs. Virtual Reality: How They Differ and What That Means for You What Is Virtual Reality? (VR Definition) What Is the Metaverse? What Is Mixed Reality? The 10 Best VR Puzzle and Escape Room Games Pansonite VR Headset Review: A Solid Start Your VR Journey Star Wars: Squadrons Review: Immersive Space Combat The Best VR Apps for the iPhone in 2022 The 10 Best Scary Virtual Reality Games The Best Google Cardboard Apps of 2022 The 9 Best Free VR Games of 2022 Expert Tested: The 7 Best VR Headsets in 2022 Newsletter Sign Up Newsletter Sign Up Newsletter Sign Up Newsletter Sign Up Newsletter Sign Up By clicking “Accept All Cookies”, you agree to the storing of cookies on your device to enhance site navigation, analyze site usage, and assist in our marketing efforts. Cookies Settings Accept All Cookies