Exploring the Multi-Layered Affordances of Composing and Performing Interactive Music with Responsive Technologies

Anna Einarsson, Tom Ziemke

The question motivating the work presented here, starting from a view of music as embodied and situated activity, is how can we account for the complexity of interactive music performance situations. These are situations in which human performers interact with responsive technologies, such as sensor-driven technology or sound synthesis affected by analysis of the performed sound signal. This requires investigating in detail the underlying mechanisms, but also providing a more holistic approach that does not lose track of the complex whole constituted by the interactions and relationships of composers, performers, audience, technologies, etc. The concept of affordances has frequently been invoked in musical research, which has seen a "bodily turn" in recent years, similar to the development of the embodied cognition approach in the cognitive sciences. We therefore begin by broadly delineating its usage in the cognitive sciences in general, and in music research in particular. We argue that what is still missing in the discourse on musical affordances is an encompassing theoretical framework incorporating the sociocultural dimensions that are fundamental to the situatedness and embodiment of interactive music performance and composition. We further argue that the cultural affordances framework, proposed by Rietveld and Kiverstein (2014) and recently articulated further by Ramstead et al. (2016) in this journal, although not previously applied to music, constitutes a promising starting point. It captures and elucidates this complex web of relationships in terms of shared landscapes and individual fields of affordances. We illustrate this with examples foremost from the first author's artistic work as composer and performer of interactive music. This sheds new light on musical composition as a process of construction-and embodied mental simulation-of situations, guiding the performers' and audience's attention in shifting fields of affordances. More generally, we believe that the theoretical perspectives and concrete examples discussed in this paper help to elucidate how situations-and with them affordances-are dynamically constructed through the interactions of various mechanisms as people engage in embodied and situated activity.