Acoustic music communication over long distances has always been part of our cultural life. The African Talking Drum and whistled communication in Europe are two examples of human communication, and we can find similar traditions in the animal world in the form of bird and whale songs. However, only recently have musicians started to explore the possibility of joint music performances over electronic telecommunication networks; new Internet developments have accelerated this movement.
With regard to audio, a number of technological developments have specifically dealt with live music communication. For example, the communication delay, which also characterizes long-distance telephone connections, could be greatly reduced. In addition, large increases in transmission bandwidth led to a much better audio quality of the transmission systems.
The specific video requirements for co-located music performances have received less attention, maybe because music is predominantly considered to be an acoustic event. In this paper, a number of observations will be presented to emphasize the importance of visual cues in music performances. Most of the observations were drawn from numerous projects of the Telematic Circle, a group initiated in Summer 2007 by Pauline Oliveros (Deep Listening Institute & Rensselaer Polytechnic Institute), Chris Chafe (CCRMA, Stanford University) and the author. The main focus of the Telematic Circle is to spend as much time performing music over the telematic channels as was required to maintain, develop, and upgrade the telematic systems. Tele-Colonization (performed at the International Conference on Auditory Displays, ICAD 2007 in Montreal) and Dynamic Spaces (performed at SIGGRAPH 2007 in San Diego) were among the projects of the Telematic Circle.
Within our practical work on improvised music it was often found advantageous to run the audio ahead of the video in networked music performances, instead of choosing audio/visual synchronicity, which is essential for speech applications. This way, the audio delay could be minimized. The audio delay needs to be below a critical value if the remote ensembles perform together rhythmically. Other aspects of speech and music applications are very similar. For example, one similarity is the need to exchange documents, which -- in the case of music -- are predominantly music scores.
Laboratory experiments, conducted together with my graduate students, Daniel Valente, Shane Myrbeck, and Anne Guthrie, show that musicians have a clear expectation of how a televised remote location should sound from the visual imprint alone. The experiments also demonstrated the importance of visual cues in a telematic music performance.
As of today, most telematic environments are established in the virtual reality tradition, where it is intended to create the illusion of a single physical concert venue, so to say making the enabling telematic technology disappear. In contrast, a more abstract approach based on virtual functionality is pursued in the authors current work. Here, the design of the telematic system is optimized for the opportunity it provides to the musicians. An example of this approach is the integration of music scores into the virtual environment.