Eyes as a window to listening effort with virtual reality
Kristina DeRoy Milvae – klmilvae@buffalo.edu
Department of Communicative Disorders and Sciences, University at Buffalo, Buffalo, NY, 14214, United States
Additional authors: Ian Phillips, Mythili Thamilchelvam, Shifali Chambers, Uzaira Sethi, Jacob Lefler, Stefanie E. Kuchinsky, Douglas S. Brungart
Popular version of 2pPP4 – Virtual reality potential as a platform to measure listening effort
Presented at the 188th ASA Meeting
Read the abstract at https://eppro01.ativ.me/web/index.php?page=Session&project=ASAICA25&id=3868338
–The research described in this Acoustics Lay Language Paper may not have yet been peer reviewed–
Most people take their ability to understand speech for granted until they have a hearing loss. However, approximately 25 million Americans with clinically normal hearing still report experiencing hearing difficulties substantial enough to negatively impact their daily lives. The extra mental effort that these individuals exert during listening appears to come at the cost of reduced ability to participate in social interactions, greater fatigue, and poorer performance on other daily life functions, such as memory for speech. There is currently no standard objective test that quantifies the increased listening effort that these individuals experience, and we are exploring possibilities for this with our research.
Changes in the pupil size of the eye during difficult listening can be used to measure listening effort. Typically, this measurement requires listeners to keep their heads still while a specialized, expensive infrared camera system tracks changes in the listener’s pupils while they listen. In recent years, virtual reality headsets have emerged as a possible alternative to these laboratory-based eye-tracking systems. Virtual reality headsets were also originally developed as research tools that were primarily used in the laboratory, but they are now widely marketed for entertainment and gaming applications. The most advanced virtual reality headsets now incorporate eye-tracking technology capable of making continuous measurements of eye gaze direction and pupil size. Virtual reality headsets are less expensive than laboratory eye-tracking systems, they are more portable, and they require fewer movement restrictions during the eye measurements. All of these factors could make virtual reality headsets more appealing than traditional eye trackers for clinical applications. However, it is not known how comparable listening effort measurements are across these research-grade and virtual-reality eye-tracking systems.
The purpose of our study was to directly compare the ability of research and virtual reality systems to monitor changes in listening effort via changes in pupil dilation. Participants with normal hearing listened to and recalled digits played in the right ear. Sometimes they had to ignore distracting digits that were also played in the left ear. As in our previous studies, we found that the addition of the distracting digits caused a systematic increase in the pupil size measured by a laboratory-grade eye tracker, presumably because it required an increase in listening effort. We also found that the same effort-related increase in pupil size could be measured with a low-cost virtual-reality headset, opening up the possibility that these systems could be used to conduct objective measures of listening effort in clinical environments.
Our research also revealed important potential limitations to consider with using virtual reality headsets to measure listening effort in a clinical setting. These include subtle quality differences in the measurements, rapidly changing technology, and the programming requirements to set up these systems for this application. Our ongoing work aims to address these concerns as we see potential for clinical use of virtual reality systems for listening effort applications in audiology. The ability to validate patient complaints of hearing difficulties with objective clinical metrics will allow for the development interventions to mitigate listening effort and ultimately improve the quality of life for the millions of people experiencing hearing difficulties.
Disclaimer: The views expressed in this article are those of the authors and do not necessarily reflect the official policy of the Department of Defense or U.S. Government.