M. Ercan Altinsoy - ercan.altinsoy@ruhr-uni-bochum.de
Jens Blauert
Institute
of Communication Acoustics
Ruhr University Bochum,
Bochum, GERMANY, D-44780
Richard H.Y. So
Department of Industrial Engineering and Engineering Management,
Hong Kong University of
Science & Technology, HONG KONG
Popular version of paper 2aPP5
Presented Tuesday morning, November, 11, 2003
146th ASA Meeting, Austin, TX
In our daily life, we obtain information from different sensory modalities (visual, auditory, tactile etc.) by interacting with our natural world. Until the end of the1990s, most multimedia applications only utilized visual and auditory feedback. One reason for the lack of virtual tactile information could be that the visual and auditory modalities are more dominant than the sense of touch in humans. The other reason might be the computational complexity required to generate real touch sensations in a computerized environment. Haptic feedback brings the sense of touch (tactile sense) and force-feedback to multi-media applications besides the mostly utilized modalities e.g., auditory and visual. Recently haptic researchers have made significant progress in developing haptic feedback devices for multimedia applications, particularly virtual reality, game, medical surgery, and web applications. Due to the increasing usage of the haptic modality in multimedia applications, the perceptual aspects of the interaction between tactile and other modalities is becoming increasingly important.
This paper describes experiments relating to the effect of loudness on haptic force-feedback perception. In a normal environment, force-feedback would be perceived in our hands as a consequence of beating a drum and combined with the loudness of the drum sound give us the required information about how much force we applied when playing the drum. In this study, psychophysical experiments were conducted to investigate the effect of loudness on haptic force-feedback perception (strongness) by playing a virtual drum.
The Cyber Grasp force feedback system was used to present force feedback information generated from the virtual drum to the subjects (see Fig. 1). The force feedback system consists of a force-reflecting exoskeleton and a hand-tracker glove, and provides force-feedback to each finger of the user relative to the palm of their hand. The auditory stimulus was a drum sound with loudness proportional to the beat-force magnitude and it was presented from a PC. The sound was amplified and delivered to both ears simultaneously through closed-face dynamic headphones which have a very high sound isolation level and therefore masked the background noise generated by the force-feedback system.
Figure 1: a) a natural condition which would result in a perfect integration of auditory and tactile information, b) shows a typical listener receiving information c) shows the force-feedback device
Altogether three different experiments were conducted. The same six subjects (four right-handed men and two right-handed women with self reported normal-hearing ability and normal tactual/motoric capabilities) participated in the experiments. Subjects were presented with: 1) only haptic force-feedback information, 2) only auditory information, and 3) auditory and haptic information together. In the third experiment, some stimulus-pairs were designed to be physically accurate (the drum sound was presented with loudness proportional to the beat-force magnitude) and in some stimulus-pairs, the drum sounds were presented with loudness greater than would be expected by the beat force. In each experiment, subjects were asked how much force they had applied when playing the virtual drum by assigning number to the test simuli. Strength magnitude was estimated using a magnitude estimation with a standard stimuli. In each trial a standard stimuli was presented and the participant was told that the strongness sensation it produces has a certain numerical value (i.e.10). After the standard stimuli, a test stimulus was presented and the participants task was to assign numbers proportional to his/her subjective impression of the strongness related to the standard stimuli.
Summary
results of experiment 3 for three different applied force-feedback conditions:
4 N (blue bars), 6 N (red bars) and 12 N (yellow bars) are shown in Figure 2.
Each bar shows the perceived magnitude of force for one sound pressure level
and one applied force magnitude, N being newtons. For the constant 4 N force-feedback
condition, if sound pressure level increase, it results in an increase
in the perceived force-feedback magnitude. The perceived increase in force magnitude
function of sound pressure level also holds for the 6 N and 12 N applied force
condition.
In
conclusion, these results show that the magnitude of strength increases
with increasing loudness in spite of no change in force-feedback as generated
by the virtual drum and applied to the
subjects hand. Therefore
it appears that participants weight loudness to a greater degree than haptic
information if there
is no-change in force feedback, when
trying to discern information
from two modalities. But also there appears an interaction between applied force
and sound pressure level, with the effect of increasing sound pressure level
having a greater effect upon perceived force, if the standard applied force
is greater. So in summary, humans do not exactly feel what the haptic
sense tells them, but, rather they integrate the two modalities of hearing and
touch and what they feel will be dependent upon the loudness of the stimuli
and the force-feedback which it creates.These results contribute to our knowledge
on how to effectively combine haptic and auditory information in virtual environments.
[This
work was supported by the International
Graduate School for Neuroscience (IGSN) at the Ruhr University Bochum, Germany.
Experiments were conducted in the Virtual Reality Laboratory of the Department
of Industrial Engineering and Engineering Management, Hong Kong University of
Science & Technology.]