How VR Could Capture and Communicate Your Feelings GA S REGULAR Menu Lifewire Tech for Humans Newsletter! Search Close GO News > Smart & Connected Life
How VR Could Capture and Communicate Your Feelings
Watching you, watching me
By Sascha Brodsky Sascha Brodsky Senior Tech Reporter Macalester College Columbia University Sascha Brodsky is a freelance journalist based in New York City. His writing has appeared in The Atlantic, the Guardian, the Los Angeles Times and many other publications. lifewire's editorial guidelines Updated on August 9, 2021 01:21PM EDT Fact checked by Rich Scherr Fact checked by Rich Scherr University of Maryland Baltimore County Rich Scherr is a seasoned technology and financial journalist who spent nearly two decades as the editor of Potomac and Bay Area Tech Wire. lifewire's fact checking process Tweet Share Email Tweet Share Email Smart & Connected Life Mobile Phones Internet & Security Computers & Tablets Smart Life Home Theater & Entertainment Software & Apps Social Media Streaming Gaming Key Takeaways
Researchers are investigating new ways to track and communicate human emotion in VR. A new device called NeckFace can be worn as a necklace to monitor facial expressions. Facebook recently released a paper on "reverse passthrough VR" to make VR headsets less physically isolating. Flashpop / Getty Images If no one can see you smile in virtual reality (VR), did it really happen? A research team at Cornell University has developed a device that can be worn as a necklace and tracks facial expressions. The NeckFace uses infrared cameras to capture images of the chin and face from beneath the neck. It’s part of a growing wave of innovations aimed at capturing and expressing emotions in VR. "Current VR implementations have advantages and disadvantages versus other remote communication forms like webcams," Devon Copley, the CEO of VR company Avatour, told Lifewire in an email interview. "Body language, for example, can be more expressively captured and communicated than by video. But the lack of real facial expressions is a massive loss of communication bandwidth, and these emotion-sensing technologies are really having to compensate for that." Tracking Your Face
VR is all about new ways to experience digital environments. But the NeckFace concept could be one way to get more feedback from users. "The ultimate goal is having the user be able to track their own behaviors, through continuous tracking of facial movements," Cheng Zhang, a Cornell University researcher who was one of the authors of the paper, said in a news release. “And this hopefully can tell us a lot of information about your physical activity and mental activities.” Aside from emotion-tracking, Zhang sees many applications for this technology: virtual conferencing when a front-facing camera is not an option, facial expression detection in virtual reality scenarios, and silent speech recognition. NeckFace also has the potential to change video conferencing. "The user wouldn’t need to be careful to stay in the field of view of a camera," François Guimbretière, another member of the Cornell research team, said in the news release. "Instead, NeckFace can recreate the perfect headshot as we move around in a classroom, or even walk outside to share a walk with a distant friend." Bringing Emotion to VR
Other companies are working to bridge the gap between the real and the virtual worlds. Natural, in-person communication between people includes information channels well beyond the text of utterances. Facebook recently released a paper on "reverse passthrough VR" to make VR headsets less physically isolating. The researchers describe a method of translating your face onto the front of a headset, although it’s only in a testing phase. VR is getting more realistic, but expressing the emotions of users is still a challenge, experts say. "Natural, in-person communication between people includes information channels well beyond the text of utterances," Copley said. "Tone of voice and body language is crucial, but an often-overlooked and really important aspect of communication is gaze. The direction of an interlocutor’s gaze is massively important." Many companies are trying to discern human emotion in virtual reality. HP’s new Omnicept headset, for example, tracks pupil size, pulse, and muscle movements. The company MieronVR uses the Omnicept for healthcare applications. "VR has the ability to connect people and build empathy for the self and others," Jessica Maslin, the president of Mieron, told Lifewire in an email interview. "Self-empathy is connected to much higher levels in self-care and care of future outcomes." Tracking emotion in VR one day even could help detect whether users will commit criminal acts in the future. Flashpop / Getty Images "If we can detect emotion, we can create virtual scenarios in which we locate people, in order to understand their risk better," forensic psychologist Naomi Murphy, who works with VR, told Lifewire in an email interview. "For instance, we could create scenes in which there is fire present to detect how emotionally aroused someone who has a history of arson is pre-and post-treatment." On the lighter side, emotion tracking could also make gaming more fun. "We’re still learning how to interpret this data correctly, but one can imagine signaling physical states in creative ways such as changing color or even choosing a different avatar, based on the emotional state of the user," Copley said. "Imagine turning into a vengeful dragon when the various sensors indicate anger." Was this page helpful? Thanks for letting us know! Get the Latest Tech News Delivered Every Day Subscribe Tell us why! Other Not enough details Hard to understand Submit More from Lifewire Meta/Facebook's Bet on Virtual Reality The 10 Best Scary Virtual Reality Games The Best Google Cardboard Apps of 2022 Are Facebook’s Codec Avatars Too Realistic? Experts Say It’s a Good Thing What Is Virtual Reality? (VR Definition) How AI Could Track and Use Your Emotions HTC Vive Pro Headset Review: The Best Consumer VR Prescription Lenses for VR Could Set Your Eyes Free How New Hologram Tech Could Change Interactions in the Future How Face Tracking could Make VR Better Can AI Teach Us to Be More Human? How VR Could Be the Future of Travel Why You'll Want to Get the New PS5 VR Headset How VR Can Help You Get Fit How An Ultralight Apple Headset Could Transform VR How New Technology Could Create a 3D Map of the World Newsletter Sign Up Newsletter Sign Up Newsletter Sign Up Newsletter Sign Up Newsletter Sign Up By clicking “Accept All Cookies”, you agree to the storing of cookies on your device to enhance site navigation, analyze site usage, and assist in our marketing efforts. Cookies Settings Accept All Cookies