The article Feeling the Effort of Classical Musicians – A Pipeline from Electromyography to Smartphone Vibration for Live Music Performance provides an insightful overview of the live-stream MappEMG pipeline project, in which a mobile application was developed to mimic the muscle response of classical music performers for audience members. This project began with the notion that these “gestures”, or invisible muscle movements made by musicians while performing, are integral to the musician’s sense of place within a piece, and would increase the audience’s sense of immersion if they could be shared.
The project utilizes EMG sensor mapping to track muscle movements, “From an artistic perspective, EMG gives a direct access to the performer’s intention in terms of implied musical effort, which is expressed through actual physiological effort”. These movements are then reproduced as vibrations through the mobile device. Sound-based vibrotactile feedback has already been used for collaborative composition, audience interaction, and greater immersion for those with hearing impairments.
I found this case very compelling as I had never heard of a similar project. It is also interesting to see a real world use case involving the tools we are using now, such as Max8. I appreciate the drive to create a more immersive audience experience, and also to explore a new element of the performance in the musicians’ gestures.
References
Verdugo, F., Ceglia, A., Frisson, C., Burton, A., Begon, M., Gibet, S., & Wanderley, M. M. (2022). Feeling the Effort of Classical Musicians – A Pipeline from Electromyography to Smartphone Vibration for Live Music Performance. NIME 2022. https://doi.org/10.21428/92fbeb44.3ce22588
