Popular version of poster, “Writer recognition with a sound in hand-writing”
172nd ASA Meeting, Honolulu
We can notice a car approaching by noise it makes on the road or can recognize a person by the sound of their footsteps. There are many studies analyzing and recognizing these noises. In the computer security industry, studies have even been proposed to estimate what is being typed from the sound of typing on the keyboard  and extracting RSA keys through noises made by a PC .
Of course, there is a relationship between a noise and its cause and that noise, therefore, contains information. The sound of a person writing, or “hand writing sound,” is one of the noises in our everyday environment. Previous studies have addressed the recognition of handwritten numeric characters by using the resulting sound, finding an average recognition of 88.4%. Based on this study, we seek the possibility of recognizing and identifying a writer by using the sound of their handwriting. If accurate identification is possible, it could become a method of signature verification without having to ever look at the signature.
We used the handwriting sounds of nine participants, conducting recognition experiments. We asked them to write the same text, which were names in Kanji, the Chinese characters, under several different conditions, such as writing slowly or writing on a different day. Figure 1 shows an example of a spectrogram of the hand-writing sound we analyzed. The bottom axis represents time and the vertical axis shows frequency. Colors represent the magnitude – or intensity – of the frequencies, where red indicates high intensity and blue is low.
The spectrogram showed features corresponding to the number of strokes in the Kanji. We used a recognition system based on a hidden Markov model (HMM) – typically used for speech recognition –, which represents transitions of spectral patterns as they evolve in time. The results showed an average identification rate of 66.3%, indicating that writer identification is possible in this manner. However, the identification rate decreased under certain conditions, especially a slow writing speed.
To improve performances, we need to increase the number of hand writing samples and include various written texts as well as participants. We also intend to include writing of English characters and numbers. We expect that Deep Learning, which is attracting increasing attention around the world, will also help us achieve a higher recognition rate in future experiments.
Zhuang, L., Zhou, F., and Tygar, J. D., Keyboard Acoustic Emanations Revisited, ACM Transactions on Information and Systems Security, 2009, vol.13, no.1, article 3, pp.1-26.
Genkin, D., Shamir, A., and Tromer, E., RSA Key Extraction via Low-Bandwidth Acoustic Cryptanalysis, Proceedings of CRYPTO 2014, 2014, pp.444-461.
Kitano, S., Nishino, T. and Naruse, H., Handwritten digit recognition from writing sound using HMM, 2013, Technical Report of the Institute of Electronics, Information and Communication Engineers, vol.113, no.346, pp.121-125.
The term ‘soundscape’ is widely used to describe the sonic landscape and can be considered the auditory equivalent of a visual landscape. Current soundscape research looks into the view of sound assessment in terms of perception and has been the subject of large scale projects such as the Positive Soundscapes Project (Davies et al. 2009) i.e. the emotional attributes associated with particular sounds. This research addresses the limitations of current noise assessment methods by taking into account the relationship between the acoustic environment and the emotional responses and behavioural characteristics of people living within it. Related research suggests that a variety of objective and subjective factors influence the effects of exposure to noise, including age, locale, cross-cultural differences (Guyot at el. 2005) and the time of year (Yang and Kang, 2005). A key aspect of this research area is the subjective effect of the soundscape on the listener. This paradigm emphasises the subjective perception of sound in an environment – and whether it is perceived as being positive or negative. This approach dovetails with advancing sound and music classification research which aims to categorise sounds in terms of their emotional impact on the listener.
Annoyance is one of the main factors which contribute to a negative view of environmental noise, and can lead to stress-related health conditions. Subjective perception of environmental sounds is dependent upon a variety of factors related to the sound, the geographical location and the listener. Noise maps used to communicate information to the public about environmental noise in a given geographic location are based on simple noise level measurements, and do not include any information regarding how perceptually annoying or otherwise the noise might be.
Figure 1 Selected locations for recording – image courtesy of Scottish Noise Mapping
This study involved subjective assessment by a large panel of listeners (N=167) of a corpus of sixty pre-recorded urban soundscapes collected from a variety of locations around Glasgow City Centre (see figure 1). Binaural recordings were taken at three points during each 24 hour period in order to capture urban noise during day, evening and night. Perceived annoyance was measured using Likert and numerical scales and each soundscape measured in terms of arousal and positive/negative valence (see figure 2).
Figure 2 Arousal/Valance Circumplex Model Presented in Listening Tests
Coding of each of the soundscapes would be essential process in order to test the effects of the location on the variables provided by the online survey namely annoyance score (verbal), annoyance score (numeric), quadrant score, arousal score, and valence score. The coding was based on the environment i.e. urban (U), semi-open (S), or open (O); the density of traffic i.e. high (H), mid (M), low (L); and the distance form the main noise source (road traffic) using two criteria >10m (10+) and <10m (10-). The coding resulted in eight different location types; UH10-, UH10+, UM10+, UL10-, SM10+, SL10-, SL10+, and OL10+.
To capture quantitative information about the actual audio recordings themselves, the MIRToolkit for MATLAB was used to extract acoustical features from the dataset. Several functions were identified that could be meaningful for measuring the soundscapes in terms of loudness, spectral shape, but also rhythm, which could be thought of in not so musical terms but as the rate and distribution of events within a soundscape.
As expected, correlations between extracted features and locations suggest where there are many transient events, higher energy levels, and where the type of events include harsh and dissonant sounds i.e. heavy traffic, resulted in higher annoyance scores and higher arousal scores but perceived more negatively than quiet areas. In those locations where there are fewer transient events, lower energy levels, and there are less harsh and possibly more positive sounds i.e. birdsong, resulted in lower annoyance scores and lower arousal scores as well as being perceived more positively than busy urban areas. The results shed light on the subjective annoyance of environmental sound in a range of locations and provide the reader with an insight as to what psychoacoustic features may contribute to these views of urban soundscapes.
Davies, W., Adams, M., Bruce, N., Cain, R., Jennings, P., Carlyle, A., … Plack, C. (2009, October 26). A positive soundscape evaluation system. Retrieved from http://usir.salford.ac.uk/2468/1/Davies_et_al_soundscape_evaluation_euronoise_2009.pdf
Guyot, F., Nathanail, C., Montignies, F., & Masson, B. (2005). Urban sound environment quality through a physical and perceptive classification of sound sources : a cross-cultural study Methodology.
It is 5 o’clock in the morning and only a hint of sunlight is visible on the horizon. Besides the sound of a light breeze swirling through the grass, all is quiet on the Nebraska prairie. Everything seems to be asleep. Then, suddenly, “whhooo-doo-doooohh” breaks the silence. The prairie-chickens have arrived.
The Greater Prairie-Chicken is a medium-sized grouse that lives on the prairies of central North America (Figure 1a) (Schroeder and Robb 1993). Prairie-chickens are well-known for their breeding activities in which the males congregate in groups each spring and perform elaborate courtship displays to attract females (Figure 1b). The areas where the males gather, called “leks,” are distributed across the landscape. Female prairie-chickens visit leks every morning to observe and compare males until a suitable one is chosen. After mating, females leave the leks to nest and raise their broods on their own, while the males remain on the leks and continue to perform courtship displays. Click the link to watch a video clip of prairie chickens lekking.
Figure 1a: A male Greater Prairie-Chicken. Figure 1b: A male prairie-chicken performs a courtship display for a female.
These complex courtship behaviors do not occur in silence. Vocalization plays an important role in the mate choice behavior of prairie-chickens. As part of a larger study addressing the effects of electricity producing wind turbine farms on prairie-chicken ecology, we wanted to learn more about the acoustic properties of prairie-chicken calls. We did this by recording the sound of prairie-chicken vocalizations at leks in the Nebraska Sandhills. We visited the leks in the very early morning and set up audio recorders, which were placed close enough to prairie-chickens on their leks to obtain high quality recordings (Figure 2a). Sitting in a blind at the edges of leks (Figure 2b), we observed prairie-chickens while they were lekking and collected the audio recordings.
Figure 2a: We used audio recorders to record male prairie-chicken vocalizations at the leks. Figure 2b: We observed lekking prairie-chickens and recorded vocalizations by sitting in a blind at the edge of a lek.
Male Greater Prairie-Chickens use four prominent vocalizations while on the leks: the “boom,” “cackle,” “whine” and “whoop.” The four vocalizations are distinct and serve different purposes.
The boom is used as part of the courtship display, so one function is to attract mates. Booms travel a long distance across the prairie, so another purpose of the call is to advertise lek location to other prairie-chickens (Sparling 1981, 1983). Click to listen to a boom sound clip
or to watch a boom video clip we recorded at the leks.
The “cackles” are short calls typically given in rapid succession. Prairie-chickens use the cackle as an aggressive or territorial call (Sparling 1981, 1983) or as a warning to alert other prairie-chickens of potential danger, such as an approaching prairie falcon, coyote or other predator. Click to listen to a cackle sound clip.
The “whine” is slightly longer in duration than the cackle; whines and cackles are often used together. The purpose of the whine is similar to that of the cackle. It serves as an aggressive and territorial call, although it is thought that whines are somewhat less aggressive than cackles (Sparling 1981, 1983). Click to listen to a whine sound clip
or to watch a video clip of cackles and whines (the cackles are the shorter notes and the whines are the longer notes).
The “whoop” is used for mate attraction. Males typically use the whoop when females are present on the lek (Sparling 1981, 1983). Click to listen to a whoop sound clip
or to watch a whoop video clip.
We measured acoustic characteristics of the vocalizations captured on the recordings so we could evaluate their features in detail. We are using this information about the vocalizations in a study of the effects of wind turbine sound on Greater Prairie-Chickens (Figure 3). We hope to determine whether the vocalizations produced by prairie-chickens near a wind farm are different in any way from those produced by prairie-chickens farther away. For example, do the prairie chickens near wind turbines call at a higher pitch in response to wind turbine sound? Also, do the prairie chickens near wind turbines vocalize louder? Ultimately we would like to know if components of the prairie-chickens’ vocalizations are masked by the sounds of the wind turbines.
Figure 3: We are conducting a study of the effects of wind turbine noise on Greater Prairie-Chickens.
The effect of anthropogenic noise is an issue not limited to Greater Prairie-Chickens and wind turbines. As humans create increasingly noisy landscapes through residential and industrial development, vehicle traffic, air traffic and urban sprawl, the threats posed to birds and other wildlife are likely to be significant. It is important to be aware of the potential effects of anthropogenic sound and find ways to mitigate those effects as landscapes become noisier.
Schroeder, M. A., and L. A. Robb. 1993. Greater Prairie-Chicken (Tympanuchus cupido). In The Birds of North America, no. 36 (A. Poole, P. Stettenheim, and F. Gill, Eds.). Academy of Natural Sciences, Philadelphia, and American Ornithologists’ Union, Washington, D.C.
Sparling, D. W. 1981. Communication in prairie grouse. I. Information content and intraspecific functions of principal vocalizations. Behavioral and Neural Biology 32:463-486.
Sparling, D. W. 1983. Quantitative analysis of prairie grouse vocalizations. Condor 85:30-42.
Investigations into the benefits of green roofs have shown that such roofs provide many environmental benefits, such as thermal conditioning, air cleaning and rain water absorption. Analysing the way green roofs are usually constructed suggests that they may have also two interesting acoustical properties: sound insulation and sound absorption. The first property would provide protection of the house’s interior from environmental noise produced outside the house. Sound absorption, on the other hand, would reduce the environmental noise in the environment itself, by dissipating sound energy that is being irradiated on to the roof from environmental noise sources. Thus, sound absorption can help to reduce environmental noise in urban settings. Despite of being an interesting characteristic, information regarding acoustic properties of green roofs and their effects on the noise environment is still sparse. This work looked into the sound absorption of two types of green roofs commercially available in Brazil: the alveolar and the hexa system.
Fig 1: illustration of the alveolar system (left) and hexa system (right)
Sound absorption can be quantified by means of a sound absorption coefficient α, which ranges between 0 and 1 and is usually a function of frequency. Zero means that all incident energy is being reflected back into the environment and α = 1 means that all energy is being dissipated in the layers of the material, here the green roof. To find out how much sound energy the alveolar and the hexa system absorb standardized measurements were made in a reverberant chamber according to ISO-354 for different variations of both systems. The alveolar system used a thin layer of 2.5 cm of soil like substrate with and without grass and a 4 cm layer of substrate only. The hexa system was measured with layers of 4 and 6 cm of substrate without vegetation and 6 cm of substrate with a layer of vegetation of sedum. For all systems, high absorption coefficients (α > 0.7) were found for medium and high frequencies. This was expected due to the highly porous structure of the substrate. Nevertheless the alveolar system with grass, the alveolar system with 4 cm of substrate, the hexa with 6 cm of substrate and the hexa with sedum already provide high absorption for frequencies as low as 250 or 400 Hz. Thus, these green roofs systems are particularly interesting in urban settings, as traffic noise is usually low frequency noise and is hardly absorbed by smooth surfaces such as pavements or façades.
Fig 2: absorption coefficient of the alveolar samples (left) and hexa samples (rigth).
In the next step of this research is intended to make computational simulations of the noise reduction provided by the hexa and alveolar system in different noisy situations such as near airports or intense urban traffic.
Stephan Paul – firstname.lastname@example.org
Program Acoustical Engineering
Fed. University of Santa Maria
Santa Maria, RS, Brazil
Ricardo Brum – email@example.com
Program Acoustical Engineering
Fed. University of Santa Maria
Santa Maria, RS, Brazil
Andrey Ricardo da Silva – firstname.lastname@example.org
Fed. University of Santa Catarina
Florianópolis, SC, Brazil
Tenile Rieger Piovesan – email@example.com
Graduate program in Civil Engineering
Fed. University of Santa Maria
Santa Maria, RS, Brazil