Volume 7, Number 9, Abstract 874, Page 874a doi:10.1167/7.9.874 http://journalofvision.org/7/9/874/ ISSN 1534-7362
Audiovisual congruence and the processing of synchrony in swing groove drumming
Melanie Russell
Department of Psychology, University of Glasgow
[e-mail]
Karin Petrini
Department of Psychology, University of Glasgow
Phil McAleer
Department of Psychology, University of Glasgow
Davide Rocchesso
Department of Computer Science, University of Verona
Sofia Dahl
Cognitive and Systematic Musicology Lab, School of Music , Ohio State University
Carl Haakon Waadeland
Department of Music, Norwegian University of Science and Technology
Federico Avanzini
Department of Information Engineering, University of Padova
Frank Pollick
Department of Psychology, University of Glasgow
Abstract

Single cell data from macaque suggest special processing of the sights and sounds of biological actions (Kohler, Keysers, et al, Science 2002). Recently Arrighi, Alais & Burr (JOV, 2006) have examined this hypothesis using judgments of perceptual synchrony of audio and visual streams of conga drumming as well as with synthetic audio and visual streams. The perception of audiovisual temporal synchrony provides a window on how these two different sensory modalities are integrated. To further investigate the perception of audiovisual synchrony from human movements we first obtained 3D motion capture data (240 Hz) of a drummer performing swing groove drumming with an accent on the first beat at 60, 90 and 120 BPM. This movement data was converted into visual point light displays with points representing the shoulder, elbow, wrist, hand and 2 points on the drumstick and output as 60 Hz quicktime movies. Matching synthetic sounds were obtained by a simulation of the first 25 modes of a circular membrane that took as input the time and impact velocity of a strike and output the resulting audio signal. In one experimental condition the estimated impact velocity was provided to the algorithm and in the other condition the average impact velocity of all strikes was used. This allowed production of one audiovisual stream with a natural covariation of drummer motion and sound present, and another audiovisual stream with this covariation absent. For both the covariation present and absent conditions we presented observers with audiovisual displays of varying levels of positive and negative asynchrony between audio and visual streams and collected forced-choice judgments of synchrony. Our preliminary results indicate that the novel manipulation of eliminating the natural covariation between physical movement and audio signal influences observers’ integration of sight and sound.
Wellcome Trust, British Academy

History
Received April 27, 2007; published June 30, 2007
Citation
Russell, M., Petrini, K., McAleer, P., Rocchesso, D., Dahl, S., Haakon Waadeland, C., Avanzini, F., & Pollick, F. (2007). Audiovisual congruence and the processing of synchrony in swing groove drumming [Abstract]. Journal of Vision, 7(9):874, 874a, http://journalofvision.org/7/9/874/, doi:10.1167/7.9.874.
Keywords
None
On-Line Presentation
None
for related articles by these authors

for papers that cite this paper
Get citation
Get help with this






jov