OTTAWA, Ontario, May 16, 2024 – Oscar Wilde once said that sarcasm was the lowest form of wit, but the highest form of intelligence. Perhaps that is due to how difficult it is to use and understand. Sarcasm is notoriously tricky to convey through text — even in person, it can be easily misinterpreted. The subtle changes in tone that convey sarcasm often confuse computer algorithms as well, limiting virtual assistants and content analysis tools.
Xiyuan Gao, Shekhar Nayak, and Matt Coler of Speech Technology Lab at the University of Groningen, Campus Fryslân developed a multimodal algorithm for improved sarcasm detection that examines multiple aspects of audio recordings for increased accuracy. Gao will present their work Thursday, May 16, at 10:35 a.m. EDT as part of a joint meeting of the Acoustical Society of America and the Canadian Acoustical Association, running May 13-17 at the Shaw Centre located in downtown Ottawa, Ontario, Canada.
Using text recognition, incorporating emoticons, and introducing audio analysis, researchers designed a robust system for detecting sarcasm in human speech. Image credit: This image was created with the assistance of DALL•E 3.
Traditional sarcasm detection algorithms often rely on a single parameter to produce their results, which is the main reason they often fall short. Gao, Nayak, and Coler instead used two complementary approaches — sentiment analysis using text and emotion recognition using audio — for a more complete picture.
“We extracted acoustic parameters such as pitch, speaking rate, and energy from speech, then used Automatic Speech Recognition to transcribe the speech into text for sentiment analysis,” said Gao. “Next, we assigned emoticons to each speech segment, reflecting its emotional content. By integrating these multimodal cues into a machine learning algorithm, our approach leverages the combined strengths of auditory and textual information along with emoticons for a comprehensive analysis.”
The team is optimistic about the performance of their algorithm, but they are already looking for ways to improve it further.
“There are a range of expressions and gestures people use to highlight sarcastic elements in speech,” said Gao. “These need to be better integrated into our project. In addition, we would like to include more languages and adopt developing sarcasm recognition techniques.”
This approach can be used for more than identifying a dry wit. The researchers highlight that this technique can be widely applied in many fields.
“The development of sarcasm recognition technology can benefit other research domains using sentiment analysis and emotion recognition,” said Gao. “Traditionally, sentiment analysis mainly focuses on text and is developed for applications such as online hate speech detection and customer opinion mining. Emotion recognition based on speech can be applied to AI-assisted health care. Sarcasm recognition technology that applies a multimodal approach is insightful to these research domains.”
ASA PRESS ROOM In the coming weeks, ASA’s Press Room will be updated with newsworthy stories and the press conference schedule at https://acoustics.org/asa-press-room/.
LAY LANGUAGE PAPERS ASA will also share dozens of lay language papers about topics covered at the conference. Lay language papers are summaries (300-500 words) of presentations written by scientists for a general audience. They will be accompanied by photos, audio, and video. Learn more at https://acoustics.org/lay-language-papers/.
PRESS REGISTRATION ASA will grant free registration to credentialed and professional freelance journalists. If you are a reporter and would like to attend the in-person meeting or virtual press conferences, contact AIP Media Services at media@aip.org. For urgent requests, AIP staff can also help with setting up interviews and obtaining images, sound clips, or background information.
ABOUT THE ACOUSTICAL SOCIETY OF AMERICA The Acoustical Society of America is the premier international scientific society in acoustics devoted to the science and technology of sound. Its 7,000 members worldwide represent a broad spectrum of the study of acoustics. ASA publications include The Journal of the Acoustical Society of America (the world’s leading journal on acoustics), JASA Express Letters, Proceedings of Meetings on Acoustics, Acoustics Today magazine, books, and standards on acoustics. The society also holds two major scientific meetings each year. See https://acousticalsociety.org/.
ABOUT THE CANADIAN ACOUSTICAL ASSOCIATION/ASSOCIATION CANADIENNE D’ACOUSTIQUE
fosters communication among people working in all areas of acoustics in Canada
promotes the growth and practical application of knowledge in acoustics
encourages education, research, protection of the environment, and employment in acoustics
is an umbrella organization through which general issues in education, employment and research can be addressed at a national and multidisciplinary level
The CAA is a member society of the International Institute of Noise Control Engineering (I-INCE) and the International Commission for Acoustics (ICA), and is an affiliate society of the International Institute of Acoustics and Vibration (IIAV). Visit https://caa-aca.ca/.
M. Fernanda Alonso Arteche – maria.alonsoarteche@mail.mcgill.ca
Instagram: @laneurotransmisora
School of Communication Science and Disorders, McGill University, Center for Research on Brain, Language, and Music (CRBLM), Montreal, QC, H3A 0G4, Canada
Instagram: @babylabmcgill
Popular version of 2pSCa – Implicit and explicit responses to infant sounds: a cross-sectional study among parents and non-parents
Presented at the 186th ASA Meeting
Read the abstract at https://doi.org/10.1121/10.0027179
–The research described in this Acoustics Lay Language Paper may not have yet been peer reviewed–
Imagine hearing a baby coo and instantly feeling a surge of positivity. Surprisingly, how we react to the simple sounds of a baby speaking might depend on whether we are women or men, and whether we are parents. Our lab’s research delves into this phenomenon, revealing intriguing differences in how adults perceive baby vocalizations, with a particular focus on mothers, fathers, and non-parents.
Using a method that measures reaction time to sounds, we compared adults’ responses to vowel sounds produced by a baby and by an adult, as well as meows produced by a cat and by a kitten. We found that women, including mothers, tend to respond positively only to baby speech sounds. On the other hand, men, especially fathers, showed a more neutral reaction to all sounds. This suggests that the way we process human speech sounds, particularly those of infants, may vary significantly between genders. While previous studies report that both men and women generally show a positive response to baby faces, our findings indicate that their speech sounds might affect us differently.
Moreover, mothers rated babies and their sounds highly, expressing a strong liking for babies, their cuteness, and the cuteness of their sounds. Fathers, although less responsive in the reaction task, still rated highly their liking for babies, the cuteness of them, and the appeal of their sounds. This contrast between implicit (subconscious) reactions and explicit (conscious) opinions highlights an interesting complexity in parental instincts and perceptions. Implicit measures, such as those used in our study, tap into automatic and unconscious responses that individuals might not be fully aware of or may not express when asked directly. These methods offer a more direct window into the underlying feelings that might be obscured by social expectations or personal biases.
This research builds on earlier studies conducted in our lab, where we found that infants prefer to listen to the vocalizations of other infants, a factor that might be important for their development. We wanted to see if adults, especially parents, show similar patterns because their reactions may also play a role in how they interact with and nurture children. Since adults are the primary caregivers, understanding these natural inclinations could be key to supporting children’s development more effectively.
The implications of this study are not just academic; they touch on everyday experiences of families and can influence how we think about communication within families. Understanding these differences is a step towards appreciating the diverse ways people connect with and respond to the youngest members of our society.
Northwestern University, Communication Sciences & Disorders, Evanston, IL, 60208, United States
Jeff Crukley – University of Toronto; McMaster University
Emily Lundberg – University of Colorado, Boulder
James M. Kates – University of Colorado, Boulder
Kathryn Arehart – University of Colorado, Boulder
Pamela Souza – Northwestern University
Popular version of 3aPP1 – Modeling the relationship between listener factors and signal modification: A pooled analysis spanning a decade
Presented at the 186th ASA Meeting
Read the abstract at https://doi.org/10.1121/10.0027317
–The research described in this Acoustics Lay Language Paper may not have yet been peer reviewed–
Imagine yourself in a busy restaurant, trying to focus on a conversation. Often, even with hearing aids, the background noise can make it challenging to understand every word. While some listeners manage to follow the conversations rather easily, others find it hard to follow along, despite having their hearing aids adjusted.
Studies show that cognitive abilities (and not just how well we hear) can affect how well we understand speech in noisy places. Individuals with weaker cognitive abilities struggle more in these situations. Unfortunately, current clinical approaches to hearing aid treatment have not yet been catered to these individuals. The standard approach to setting up hearing aids is to make speech sounds louder or more audible. However, a downside is that hearing aid settings that make speech more audible or attempt to remove background noise, can unintentionally modify other important cues, such as fluctuations in the intensity of the sound, that are necessary for understanding speech. Consequently, some listeners who depend on these cues may be at a disadvantage. Our investigations have focused on understanding why listeners with hearing aids experience these noisy environments differently and developing an evidence-based method for adjusting hearing aids to each person’s individual abilities.
To address this, we pooled data from 73 individuals across four different published studies from our group over the last decade. In these studies, listeners with hearing loss were asked to repeat sentences that were mixed with background chatter (like at a restaurant or a social gathering). The signals were processed through hearing aids that were adjusted in various ways, changing how they handle loudness and background noise. We measured how these adjustments applied to the noisy speech affected the ability of the listeners to understand the sentences. Each of these studies also used a measurement to capture how the hearing aids and background noise together alter the speech sounds (signal fidelity) heard by the listener.
Figure 1. Effect of individual cognitive abilities (working memory) on word recognition as signal fidelity changes.
Our findings reveal that listeners generally understand speech better when the background noise is less intrusive, and the hearing aids do not alter the speech cues too much. But there’s more to it: how well a person’s brain collects and manipulates speech information (their working memory), their age, and the severity of their hearing loss all play a role in how well they understand speech in noisy situations. Specifically, those with lower working memory tend to have more difficulty understanding speech when it is obscured by noise or altered by the hearing aid (Figure 1). So, improving the listening environment by reducing the background noise and/or choosing milder settings on the hearing aids could benefit these individuals.
In summary, our study indicates that a tailored approach that considers each person’s cognitive abilities could lead to better communication, especially in noisier situations. Clinically, the measurement of signal fidelity may be a useful tool to help make these decisions. This could mean the difference between straining to hear and enjoying a good conversation over dinner with family.
Emma Holmes – emma.holmes@ucl.ac.uk
X (Twitter): @Emma_Holmes_90
University College London (UCL), Department of Speech Hearing and Phonetic Sciences, London, Greater London, WC1N 1PF, United Kingdom
Popular version of 4aPP4 – How does voice familiarity affect speech intelligibility?
Presented at the 186th ASA Meeting
Read the abstract at https://doi.org/10.1121/10.0027437
–The research described in this Acoustics Lay Language Paper may not have yet been peer reviewed–
It’s much easier to understand what others are saying if you’re listening to a close friend or family member, compared to a stranger. If you practice listening to the voices of people you’ve never met before, you might also become better at understanding them too.
Many people struggle to understand what others are saying in noisy restaurants or cafés. This can become much more challenging as people get older. It’s often one of the first changes that people notice in their hearing. Yet, research shows that these situations are much easier if people are listening to someone they know very well.
In our research, we ask people to visit the lab with a friend or partner. We record their voices while they read sentences aloud. We then invite the volunteers back for a listening test. During the test, they hear sentences and click words on a screen to show what they heard. This is made more difficult by playing a second sentence at the same time, which the volunteers are told to ignore. This is like having a conversation when there are other people talking around you. Our volunteers listen to many sentences over the course of the experiment. Sometimes, the sentence is one recorded from their friend or partner. Other times, it’s one recorded from someone they’ve never met. Our studies have shown that people are best at understanding the sentences spoken by their friend or partner.
In one study, we manipulated the sentence recordings, to change the sound of the voices. The voices still sounded natural. Yet, volunteers could no longer recognize them as their friend or partner. We found that participants were still better at understanding the sentences, even though they didn’t recognize the voice.
In other studies, we’ve investigated how people learn to become familiar with new voices. Each volunteer learns the names of three new people. They’ve never met these people, but we play them lots of recordings of their voices. This is like when you listen to a new podcast or radio show. We’ve found that people become very good at understanding these people. In other words, we can train people to become familiar with new voices.
In new work that hasn’t yet been published, we found that voice familiarization training benefits both older and younger people. So, it may help older people who find it very difficult to listen in noisy places. Many environments contain background noise—from office parties to hospitals and train stations. Ultimately, we hope that we can familiarize people with voices they hear in their daily lives, to make it easier to listen in noisy places.
Making Table Tennis Accessible for Blind Players #Acoustics23
Object tracking combined with a speaker array can provide real-time audio feedback in three dimensions.
SYDNEY, Dec. 6, 2023 – Table tennis has been played for decades as a more accessible version of tennis. The sport is particularly beginner-friendly while maintaining a rich level of competitive play. However, like many sports, it remains inaccessible to people who are blind or have low vision.
Phoebe Peng, an Engineering Honours student at the University of Sydney, is researching ways to allow people with low vision and blindness to play pingpong using sound.
The process uses neuromorphic cameras and an array of loudspeakers, designed to allow players to track the ball and movements based on sound. Peng will present her work Dec. 6 at 10:20 a.m. Australian Eastern Daylight Time, as part of Acoustics 2023 Sydney running Dec. 4-8 at the International Convention Centre Sydney.
Motion tracking cameras and an array of linked speakers give real-time audio feedback to table tennis players with low vision. Credit: Phoebe Peng
According to Peng, table tennis makes a perfect test case for this kind of technology.
“The small size of the ball and table, along with the movement of the ball in 3D space, are things that make table tennis difficult to play for those with low vision and complete blindness,” said Peng, who completed the work as part of her Honours thesis. “Making this sport more accessible while also exploring the potential of neuromorphic cameras were my two biggest motivators.”
The neuromorphic cameras Peng employed are ideal for tracking small objects like table tennis balls. Unlike normal cameras that capture complete images of a scene, neuromorphic cameras track changes in an image over time. Using two perfectly positioned cameras, Peng could identify and track a ball in three dimensions in real time. She then fed that data into an algorithm controlling an array of loudspeakers along the sides of the table, which created a sound field matching the position of the ball.
While this system works well, Peng says more experimentation is needed before it will be ready for actual play.
“An ongoing technical challenge is the matter of human perception of sound,” said Peng. “There are limitations on how accurately people can perceive sound localization. What type of sound should be used? Should the sound be continuous? This is a technical challenge we’ll be tackling in the next stage of development.”
The Acoustical Society of America is joining the Australian Acoustical Society to co-host Acoustics 2023 Sydney. This collaborative event will incorporate the Western Pacific Acoustics Conference and the Pacific Rim Underwater Acoustics Conference.
ASA PRESS ROOM In the coming weeks, ASA’s Press Room will be updated with newsworthy stories and the press conference schedule at https://acoustics.org/asa-press-room/.
LAY LANGUAGE PAPERS ASA will also share dozens of lay language papers about topics covered at the conference. Lay language papers are summaries (300-500 words) of presentations written by scientists for a general audience. They will be accompanied by photos, audio, and video. Learn more at https://acoustics.org/lay-language-papers/.
PRESS REGISTRATION ASA will grant free registration to credentialed and professional freelance journalists. If you are a reporter and would like to attend the meeting or virtual press conferences, contact AIP Media Services at media@aip.org. For urgent requests, AIP staff can also help with setting up interviews and obtaining images, sound clips, or background information.
ABOUT THE ACOUSTICAL SOCIETY OF AMERICA The Acoustical Society of America (ASA) is the premier international scientific society in acoustics devoted to the science and technology of sound. Its 7,000 members worldwide represent a broad spectrum of the study of acoustics. ASA publications include The Journal of the Acoustical Society of America (the world’s leading journal on acoustics), JASA Express Letters, Proceedings of Meetings on Acoustics, Acoustics Today magazine, books, and standards on acoustics. The society also holds two major scientific meetings each year. See https://acousticalsociety.org/.
ABOUT THE AUSTRALIAN ACOUSTICAL SOCIETY The Australian Acoustical Society (AAS) is the peak technical society for individuals working in acoustics in Australia. The AAS aims to promote and advance the science and practice of acoustics in all its branches to the wider community and provide support to acousticians. Its diverse membership is made up from academia, consultancies, industry, equipment manufacturers and retailers, and all levels of Government. The Society supports research and provides regular forums for those who practice or study acoustics across a wide range of fields The principal activities of the Society are technical meetings held by each State Division, annual conferences which are held by the State Divisions and the ASNZ in rotation, and publication of the journal Acoustics Australia. https://www.acoustics.org.au/
Roland, Woolworth & Associates, Oxford, MS, 38655, United States
Bennett Brooks and Brigitte Schulte-Fortkamp
Popular version of 4pAAb1 – Introduction to Soundwalking – an important part of the soundscape method
Presented at the 185th ASA Meeting
Read the abstract at https://doi.org/10.1121/10.0023505
Please keep in mind that the research described in this Lay Language Paper may not have yet been peer reviewed.
Our acoustic environment is a critical part of our everyday experience; it is often unconsciously processed with all other stimuli to form an impression of a place and time, but its impact is not always fully understood. Soundscape is a method of assessing the acoustic environment where perception is prioritized. The soundscape method and the soundwalk tool integrate measurements of the human perception of sound with other observations that characterize the environment, such as the sound levels, the type of location and the various sound sources. The combination of these perceptual measurements with other observations helps us to understand how the acoustic environment impacts the people there and can provide directions for possible changes that can improve their quality of life.
The soundscape method suggests assessing all sounds which occur in an environment using collected data related to human perception, the physical acoustic setting, and context. Context includes visual cues, geographic, social, psychological and cultural aspects, including one’s mental image or memory of a place. Soundscape transcends the common studies of noise and sound levels, and is a powerful tool for effecting positive results with regard to the quality of life for stakeholders in the acoustic environment; standardized methodology has been developed that can be adapted to various applications, using sound as a resource. Soundwalks are an important part of the soundscape method and are a useful way to engage stakeholders who participate by consciously observing and evaluating the soundscape.
Figure 1
A soundwalk is an element of the soundscape method that typically will include a walking tour of observation locations over a predetermined route to solicit perceptual feedback from the participants regarding the acoustic environment (see Figures 1 and 2). The participants of the soundwalk typically include stakeholders or “local experts”: members of the community that experience the soundscape daily, users/patrons of a space, residents, business people, and local officials. Soundwalks can be performed from urban areas to wilderness settings, indoors and outdoors; the information collected can have many applications including ordinances and planning, preservation or improvement of the acoustic environment, and building public/self-awareness of the acoustic environment.
Figure 2
The perceptual information collected during a soundwalk includes the sounds heard by the participants and often directed questions with scaled answers; this along with objective sound level measurements and audio recordings can be used to assess an acoustic space(s) in an effort to effect the purpose of the soundwalk. (see Figures 3 and 4) In some cases, the participants are interviewed to get a deeper understanding of their responses or the data can be taken to a lab for further study.
Figure 3
The soundwalk and post processing of collected information is flexible relative to soundscape standard methods to target an acoustic space and purpose of the investigation. This makes it an adaptable and powerful tool for assessing an acoustic environment and improving the quality of life for the those that live in or use that environment, using their own perceptions and feedback.
It’s much easier to understand what others are saying if you’re listening to a close friend or family member, compared to a stranger. If you practice listening to the voices of people you’ve never met before, you might also become better at understanding them too.
Figure 1
Figure 2
Figure 3
Figure 4