4aPPa4 – Perception of Vowels and Consonants in Cochlear Implant Users

Melissa Malinasky – Melissa_Malinasky@rush.edu

Popular version of paper 4aPPa4
Presented Thursday morning, December 10 , 2020
179th ASA Meeting, Acoustics Virtually Everywhere

Understanding individual phoneme sounds is vital to speech perception. While cochlear implants (CI) can improve speech understanding, they also introduce distortions to sound input due to the limits of technology versus the human ear. Understanding which phonemes are most commonly misunderstood, and in what context this occurs can lead to the development of better signal processing strategies in Cis and better audiologic rehabilitation strategies post-implantation. The objective of this study was to evaluate perceptual differences in accuracy of specific vowels and consonants in experienced CI users. This study looked at 25 experienced adult CI users that were a part of a larger study by Shafiro et al. Participants were presented with a word, and given closed-set responses that tested their ability to distinguish between individual consonants of vowels. To determine if they can make these distinctions, each multiple-choice response was varied by one phonemic sound (i.e. bad vs bat, hud vs hid).

Cochlear implant users achieved 78% accuracy overall for consonant sounds compared to 97% for normal hearing participants. This shows that CI users are quite successful at identifying individual consonants sounds. Consonants at the beginning of the word were identified with 80.5% accuracy, while consonants at the end of the word were identified with 75.4% accuracy. This is not as great of a variation as we would have predicted.

For correct identification of vowels, cochlear implant users had 75% accuracy, while normal hearing users had 92% accuracy. Vowels were analyzed based on accuracy, as well as other vowels they were confused with. Some vowel sounds had over 80% accuracy, while others had as low as 45%.

Overall, this study shows that CI users have fairly good consonant and vowel recognition. These results are consistent with what has been previously reported by Rodvik et al. (2018). While CI users do perform quite well, they are still outperformed by their normal hearing, age-matched peers. The presence of a single consonant can affect someone’s entire understanding of a word, and it is important to understand where the most difficulty lies for CI users. Improvement in identification of some of these more difficult consonants can give this population greater access to language understanding. These findings can also help tailor auditory training programs, and help improve speech intelligibility in CI users.

References:

Hillenbrand, J., Getty, L. A., Clark, M. J., & Wheeler, K. (1995). Acoustic characteristics of American English vowels. The Journal of the Acoustical Society of America, 97(5), 3099–3111. doi: 10.1121/1.411872

House, A. S., Williams, C. E., Hecker, M. H. L., & Kryter, K. D. (1965). Articulation testing methods: Consonantal differentiation with a closer response set. The Journal of the Acoustical Society of America, 37(1), 158–166. https://doi.org/10.1121/1.1909295

Peterson, G. E., & Barney, H. L. (1952). Control Methods Used in a Study of the Vowels. The Journal of the Acoustical Society of America, 24(2), 175–184. doi: 10.1121/1.1906875

Rødvik AK, von Koss Torkildsen J, Wie OB, Storaker MA, Silvola JT. Consonant and Vowel Identification in Cochlear Implant Users Measured by Nonsense Words: A Systematic Review and Meta-Analysis. J Speech Lang Hear Res. 2018 Apr 17;61(4):1023-1050. doi: 10.1044/2018_JSLHR-H-16-0463. PMID: 29623340.

Shafiro V, Hebb M, Walker C, Oh J, Hsiao Y, Brown K, Sheft S, Li Y, Vasil K, Moberly AC. Development of the Basic Auditory Skills Evaluation Battery for Online Testing of Cochlear Implant Listeners. Am J Audiol. 2020 Sep 18;29(3S):577-590. doi: 10.1044/2020_AJA-19-00083. Epub 2020 Sep 18. PMID: 32946250.

5aPPb1 – The freedom to move around – Hearing aid research takes a big step towards the real life

Stefan Klockgether – stefan.klockgether@sonova.com
Diego Ulloa Sanchez
Charlotte Vercammen
Peter Derleth
Sonova AG
Laubisrütistrasse 28
CH 8712 Stäfa
Switzerland

Popular version of paper 5aPPb1
Presented Friday morning, December 11, 2020
179th ASA Meeting, Acoustics Virtually Everywhere

One major aspect of hearing aid development is audiological performance. This describes the benefit in hearing a hearing impaired person can have from using a hearing aid.

Measuring audiological performance depends on the perception of individuals. To reduce the impact of individual behavior of subjects on measured results – a lack of experiment control, the degrees of freedom during an audiological study are usually strongly limited and important aspects of perception in real life are sacrificed in favor of control.

In recent years, efforts have been taken to substantiate the performance of hearing aids in real life situations. It is important to understand the listeners behavior in realistic acoustic environments, especially the potential differences between normal hearing and hearing impaired people.

The new “Real Life Lab” at Sonova brings the freedom to move around to controlled laboratory conditions. The lab provides a stage where persons can move around freely and interact with sound sources. The stage is surrounded by loudspeakers to present sound from all directions. Any motion by persons on the stage can be tracked in real time to regain the control. The motion data can be passively tracked or actively used to trigger audio and video reproduction.

real life lab

Figure 1: The Sonova Real Life Lab with loudspeakers at the sides, below the floor and at the ceiling.

The lab is also used to investigate the behavior in acoustic scenes. A pilot study has been done, to find differences between normal hearing and hearing impaired. The subjects had to find different acoustic targets in a complex scene (Video 1). Their motion as well as there performance was tracked with motion capturing (Video 2).

Video 1: The subject has to find different acoustic targets (a crying baby, a barking dog or a ringing telephone). The subject wears a hairband to track head position and orientation, a vest to track the torso and a controller to point to the found target.

Video 2: Motion capturing view of the task. The three tracked objects are the head position in pink, the torso in orange and the pointer in light blue with a pink beam indicating when and where a target has been found.  

Five normal hearing (NH, age ≈ 27), five subjects with hearing loss wearing hearing aids (HL, age ≈ 74) and five age-matched persons with age-appropriate hearing (AA, age ≈ 71) participated in the study.

Number of found targets in an allowed time. Weighted amount of head movements and accuracy.

The results show clear differences in the performance as well as in the search strategy. The young normal hearing were fast, accurate and moved their heads a lot. The age-appropriate hearing group was slower, but as accurate and moved their heads a lot. The hearing impaired were slower, less accurate and moved their heads less. Hearing impaired seem to benefit less from the gain in acoustic information which is provided by head movements and may therefore reduce the movements.

5aPPb2 – Using a virtual restaurant to test hearing aid settings

Gregory M Ellis – gregory.ellis@northwestern.edu
Pamela Souza – p-souza@northwestern.edu

Northwestern University
Frances Searle Building
2240 Campus Drive
Evanston, IL 60201

Popular version of paper 5aPPb2
Presented Friday morning, December 11th, 2020
179th ASA Meeting, Acoustics Virtually Everywhere

True scientific discoveries require a series of tightly controlled experiments conducted in lab settings. These kinds of studies tell us how to implement and improve technologies we use every day—technologies like fingerprint scanners, face recognition, and voice recognition. One of the downsides of these tightly controlled environments, however, is that the real world is anything but tightly controlled. Dust may be on your fingerprint, the light may make it difficult for the face recognition software to work, or the background may be noisy making your voice impossible to pick up. Can we account for these scenarios in the lab when we’re performing experiments? Can we bring the real world—or parts of it—into a lab setting?

In our line of research, we believe we can. While the technologies listed above are interesting in their own right, our research focuses on hearing aid processing. Our lab generally asks: what factors, and to what extent do those factors, affect speech understanding for a person with a hearing aid? The project I’m presenting at this conference is specifically looking at environmental and hearing aid processing factors. Environmental factors include the loudness of background noises and echoes. Processing factors involve the software within the hearing aid that attempts to reduce or eliminate background noise and amplification strategies that make relatively quiet parts of speech louder so they’re easier to hear. We are using computer simulations to look at both the environmental and the processing factors. We can examine the effects of the environmental and processing factors on a listener by seeing how speech intelligibility is affected by those factors.

The room simulation is first. We built a very simple virtual environment pictured below:

virtual restaurant

The virtual room used in our experiments. The red dot represents the listener. The green dot represents the speaker. The blue dots represent other people in the restaurant having their own conversations and making noise.”

We can simulate the properties of the sounds in that room using a model that has been shown to be a good approximation of real recordings of sounds in rooms. After passing the speech for the speaker and all of the competing talkers through this room model, you will have a realistic simulation of the sounds in a room.

If you’re wearing headphones while you read this article, you can listen to an example here:

A woman speaking the sentence “Ten pins were set in order.” You should be able to hear other people talking to your right, all of whom are quieter than the woman in front. All of the sound has a slight echo to it. Note that this will not work if you aren’t wearing headphones!”

We then take this simulation and pass it through a hearing aid simulator. This imposes the processing you might expect in a widely-available hearing aid. Here’s an example of what that would sound like:

Same sentence as the restaurant simulation, but this is processed through a simulated hearing aid. You should notice a slightly different pitch to the sentence and the environment. This is because the simulated hearing loss is more extreme at higher pitches.”

Based on the results of hundreds of sentences, we would have a better understanding of how the environmental factors and the hearing aid processing interact. We found that for listeners with hearing impairment, there is an interaction between noise level and processing strategy, though more data will need to be collected before we can draw any solid conclusions. While these results are a promising first step, there are many more factors to look at—different amounts of echo, different amounts of noise, different types of processing strategies… and none of these factors include anything about the person listening to the sentences either. Does age, attention span, or degree of hearing loss affect their ability to perform the task? Ongoing and future research will be able to answer these questions.

This work is important because it shows that we can account for some environmental factors in tightly-controlled research. The method works well and produces results that we would expect to see. If you want results from the lab to be relatable to the real world, try to bring the real world into the lab!

4pPPa6 – Benefits of a Smartphone as a Remote Microphone System

Dr. Linda Thibodeau, thib@utdallas.edu
Dr. Issa Panahi
The University of Texas at Dallas

Popular version of paper 4pPPa6
Presented Thursday afternoon, December 5, 2019
178th ASA Meeting, San Diego, CA

A common problem reported by persons with hearing loss is reduced ability to hear speech in noisy environments. Despite sophisticated microphone and noise reduction technology in personal amplification devices to address this challenge, speech perception remains compromised by factors such as distance from the talker and reverberation. Remote microphone (RM) systems have been shown to reduce the challenges hearing aid users face with communicating in noisy environments. The RMs worn by the speaker can stream their voice wirelessly to the users’ hearing aids which results in a significant improvement in the signal-to-noise ratio and make it easier to hear and understand speech.

Given that the additional cost of a RM may not be feasible for some individuals, the possible use of applications on a smartphone has been explored. In the past five years, it has become increasingly common for hearing aids to connect wireless to smartphones. In fact, one desirable feature of the connection to the Apple iPhone has been an application called ‘Live Listen’ (LL). This application allows the iPhone to be used as an RM with made for iPhone hearing aids.

The Statistical Signal Processing Research Laboratory at The University of Texas at Dallas has developed an application for the iPhone that is also designed to be used as an RM. The application, called SHARP, has been tested with persons with normal and impaired hearing and with several types of hearing aids in the Hearing Health Laboratory at the University of Texas at Dallas. A study was conducted to compare the benefit of LL and the SHARP application for participants with and without hearing loss on sentence recognition tasks in noise when listening through hearing aids connected to an iPhone. A video summary of the testing protocol is show in the following short video clip.

Both the LL feature and the SHARP app provide a range of benefits in speech recognition in noise from no benefit to 30% depending on the degree of hearing loss and type of aid. The results suggest that persons can improve speech recognition in noise and perhaps increase overall quality of life through the use of applications such as SHARP on the smartphone in conjunction with wirelessly connected hearing aids.

4pPPb2 – Phantom words are heard more frequently as coming from the right side of space

Diana Deutsch – ddeutsch@ucsd.edu
Dept. of Psychology,
University of California, San Diego,
La Jolla, CA , 92093, USA

Kevin Dooley
Dept. of Psychology,
California State University, Dominguez Hills,
Carson, CA, 90747, USA

Trevor Henthorn
Dept. of Music,
University of California, San Diego,
La Jolla, CA, 92093, USA

Popular version of paper 4pPPb2
Presented Thursday afternoon, Dec 5, 2019
178th ASA Meeting, San Diego, CA

When we listen to speech, we draw on an enormous amount of experience to make inspired guesses as to what’s being said. But this very process of guesswork can lead us to perceive words and phrases that are not, in fact, being spoken. This paper reports a study in which two sequences of words arise simultaneously from different regions of space. The subject sits in front of two loudspeakers, with one to his left and the other to his right. A sequence is played consisting of two words, or a single word that is composed of two syllables, and these are repeated continuously. The same sequence is presented via both loudspeakers, but the sounds are offset in time, so that when one sound (word or syllable) is coming from the speaker on the left, the other sound is coming from the speaker on the right.

On listening to such a sequence, people often begin by hearing a jumble of meaningless sounds, but after a while distinct words and phrases emerge perceptually. Those that are heard as from the speaker on the left often appear different from those heard as from the speaker from the right. Later, different words and phrases emerge. In addition, people often hear a third stream of words or phrases, apparently coming from some location between the two speakers. Nonsense words, and musical, often rhythmic sounds, sometimes seem to be mixed in with the meaningful words.

People often report hearing speech in strange or “foreign” accents—presumably they are perceptually organizing the sounds into words and phrases that are meaningful to them, even though they seem to be distorted in consequence. To give an example of the variety of words that people hear when listening to these sounds, here are some reports from students in one class that I teach at UCSD, when presented continuously with the word nowhere.

window, welcome, love me, run away, no brain, rainbow, raincoat, bueno, nombre, when oh when, mango, window pane, Broadway, Reno, melting, Rogaine.

Click here (MP3 file) to listen to a phantom word

Click here to listen to another phantom word

It has been shown that, in listening to speech, most righthanders tend to focus more on the right side of space, which is represented primarily in the left hemisphere. However, non-righthanders are more varied in the direction of their focus. So we surmised that righthanders would perceive more phantom words and phrases as though coming from their right, and that non-righthanders would not show this difference in perceived location. To ensure that any effect of spatial location could not be attributed to a difference in loudspeaker characteristics, we divided our subject population into two handedness groups – righthanders and non-righthanders – and for each group half of the listeners were seated facing forward – that is, toward the speakers – and the other half were facing backward. The subjects were asked to write down each new word or phrase when they heard it, and to indicate whether it appeared to be coming from their left, from their right, or from somewhere between the speakers.

Forty UCSD students served as subjects. These were twenty righthanders and twenty non-righthanders. The righthanders were 5 male, and 15 female, with an average age of 21 years and 5 years of musical training. The non-righthanders were also 5 male and 15 female, with an average age of 22 years and 6.6 years of musical training. Our results showed no effect of age, musical training, or gender.

phantom words setup

Setup in study exploring the number of phantom words heard as from the left, center, and right.

Figure 1 shows the setup for the experiment. Seven phantom word sequences were presented, separated by 30 sec. pauses.

phantom words reported

Average number of phantom words reported for each sequence, classified by the direction from which the phantom word appeared. The data from the forward-facing subjects are here presented.

Figure 2 shows, for the forward-facing subjects, the average number of phantom words that were reported for each sequence, classified by whether the phantom word was perceived as coming from the left, from the right, or as centered between the speakers. As shown here, the righthanders reported more phantom words as from the right, and this difference in perceived location was highly significant. In contrast, the non-righthanders showed no difference in the number of phantom words they reported as from the left or from the right.

phantom words reported

Average number of phantom words reported for each sequence, classified by the direction from which the phantom word appeared. The data from the backward-facing subjects are here presented.

Figure 3 shows the results for the backward-facing subjects. The righthanders again reported more phantom words as coming from the right, and this difference in perceived location was again significant. And again, the non-righthanders showed no difference in the number of phantom words they reported as coming from the left or from the right.

So this study confirmed our surmise that righthanders would tend to hear more phantom words as from the right side of space, which for them is represented primarily in the left hemisphere. It further implies that, in righthanders, the left hemisphere is more involved in constructing meaning from ambiguous speech sounds.

Footnote
For an extended discussion of the ‘Phantom Words’ illusion, see Deutsch, D. (2019) Musical illusions and phantom words: How music and speech unlock mysteries of the brain,. Oxford University Press. https://global.oup.com/academic/product/musical-illusions-and-phantom-words-9780190206833

1aPP – The Role of Talker/Vowel Change in Consonant Recognition with Hearing Loss

Ali Abavisani – aliabavi@illinois.edu
Jont B. Allen – jontalle@illinois.edu
Dept. of Electrical and Computer Engineering
University of Illinois at Urbana-Champaign
405 N Mathews Ave
Urbana, IL, 61801

Popular version of paper 1aPP
Presented Monday, May 13, 2019
177th ASA Meeting, Louisville, KY

Hearing loss can have serious impact on social life of individuals experiencing it. The effect of hearing loss becomes more complicated in environments such as restaurants, where the background noise is similar to speech. Although hearing aids in various designs, intend to address these issues, users complain about hearing aids performance in social situations, where they are mostly needed. Part of this problem refers to the nature of hearing aids, which do not use speech as part of design and fitting process. If we somehow incorporate speech sounds in real life conditions into the fitting process of hearing aids, it may be possible to address most of the shortcomings that irritates the users.

There have been many studies on the features that are important in identification of speech sounds such as isolated consonant + vowel (CV) phones (i.e., meaningless speech sound). Most of these studies ran experiments on normal hearing listeners, to identify the effects of different speech features in correct recognition. It turned out that manipulation of speech sounds, such as replacing a vowel, or amplifying/attenuating certain parts of sound in time-frequency domain, leads to identification of new speech sounds by the normal hearing listeners. One goal of current study is to investigate whether there are similar responses to such manipulations from listeners who have hearing loss.

We designed a speech-based test that may be utilized by audiologists to determine susceptible speech phones for each individual with hearing loss. The design includes a perceptual measure that corresponds to speech understanding in background noise, where the noise is similar to speech. The perceptual measure identifies the noise level in which the speech sound is recognizable by an average normal hearing listener, at least with 90% accuracy. The speech sounds within the test include combinations of 14 consonants {p, t, k, f, s, S, b, d, g, v, z, Z, m, n} and four vowels {A, ae, I, E}, to cover different features that are present in speech. All the test sounds have pre-evaluated to make sure they are recognizable by normal hearing listeners in the noise conditions of the experiments. Two sets of sounds named T$_1$ and T$_2$ having same consonant-vowel combinations of sounds but different talkers, had been presented to the listeners at their most comfortable level of hearing (not depending to their specific hearing loss). The two speech sets had distinct perceptual measure. When two sounds with similar perceptual measure, and with the same consonant but different vowel are presented to a listener with hearing loss, their response can show us how their particular hearing function, may cause errors in understanding this particular speech sound, and why this function led to recognition of a specific sound instead of the presented speech. Also, presenting sounds from the two sets constitute the means to compare the role of perceptual measure (which is based on normal hearing listeners), on listeners with hearing loss. When the recognition score for a particular listener increases as the result of a change in presented speech sounds, it is an indication on how the fitting process of hearing aid should follow, regarding that particular (listener, speech sound) pair.

While the study shows that improvement or degradation of the speech sounds are listener dependent, on average 85% of sounds are improved when we replaced the CV with same CV but with a better perceptual measure. Additionally, using CVs with similar perceptual measure, on average 28% of CVs are improved when we replaced the vowel with vowel {A}, 28% of CVs are improved when we replaced the vowel with vowel {E}, 25% of CVs are improved when we replaced the vowel with vowel {ae}, and 19% of CVs are improved when we replaced the vowel with vowel {I}.

The confusion pattern in each case, provides insight on how these changes affect the phone recognition in each ear. We propose to prescribe hearing aid amplification tailored to individual ears, based on the confusion pattern, the response from change in perceptual measure, and the response from change in vowel.

These tests are directed at the fine-tuning of hearing aid insertion gain, with the ultimate goal of improving speech perception, and to precisely identify when and for what consonants the ear with hearing loss needs treatment to enhance speech recognition.