Australian boffins use facial expressions to navigate VR

Researchers from Australia, New Zealand and India have developed experimental technology that allows users to manipulate objects in virtual reality using only facial expressions.

The research team led by The University of Queensland’s Dr Aridam Dey developed neural processing techniques to capture a person’s smile, frown and clenched jaw, using each expression to trigger specific actions in VR environments.

Professor Mark Billinghurst from the University of South Australia said the system was designed to recognise these expressions using an EEG headset.

“A smile was used to trigger the ‘move’ command, a frown for the ‘stop’ command and a clench for the ‘action’ command, in place of a handheld controller performing these actions,” he said.

“Essentially we are capturing common facial expressions such as anger, happiness and surprise and implementing them in a virtual reality environment.”

Prof Billinghurst said users involved in the experiments reported feeling more immersed in the VR experiences controlled by facial expressions than those controlled using the handheld controllers specifically designed for use in VR.

In addition to providing a novel way to navigate VR environments, the techniques could potentially allow people with disabilities, including amputees or those with motor neuron disease, to interact hands-free in VR.

Image credit: ©stock.adobe.com/au/Damir Khabirov