Abstract
This article argues for a processual and experiential approach to dealing with music. Starting from the theoretical writings of James, Dewey and von Uexküll as well as from empirical evidence from current neurobiological research, it introduces an adaptive model of sense-making, relying heavily on the epistemological paradigms of “embodied” and “experiential cognition”. Central in this approach is an “enactive” conception of music cognition as the outcome of interactions with the sounds, stressing the role of the cogniser as an actor who constructs and organises his/her knowledge. This involves low-level reactive machinery—a kind of lock-and-key—as well as higher-order cognitive mediation that goes beyond mere causality and that allows the music user to “cope” with the sounds.

Is music something out there? A kind of artefact that is reified or objectified, and that can be dealt with in a static way? Or does it rely on processes which call forth interactions with the sounds? Should we conceive of music users besides the music, and think about music as something which is perceived, conceptualised and enacted upon in order to be meaningful? And if so, what is the role of the body and the mind in this process of sense-making? This paper tries to answer these questions by introducing a theoretical framework that leans heavily on the seminal writings of John Dewey, William James and Jakob von Uexküll, together with empirical evidence from current neurobiological research. Its central focus, however, is on the role of musical experience and the way we make sense of it (see Blacking, 1955, Määttänen, 1993, Reybrouck, 2004b, Westerlund, 2002).

The musical experience is multifaceted: it is crucial in the construction of musical knowledge and points in the direction of a processual approach to dealing with music—with “embodied” and “experiential cognition” as major epistemological paradigms. But what exactly is embodied and experiential cognition? And how are both related to the process of dealing with music? In order to answer these questions, I propose to introduce a general adaptive model of sense-making which is grounded in our biology and our cognitive abilities (Reybrouck, 2005a, 2006b) in an attempt to bring together body, mind and music. There are, in fact, current conceptual developments in cognitive science which argue for the inclusion of the body in our understanding of the mind. As such, it is possible to articulate a plausible and grounded theory which is closely related to theories of cognitive organisation which treat cognition as an activity that is structured by the body which is immersed in an environment that shapes its experience. Or put in other terms: cognition depends upon experiences based in having a body—with sensorimotor capacities—that are embedded in an encompassing biological, psychological, and cultural context (Lakoff & Johnson, 1999, and for a musical analogy: Iyer, 2002 and Saslaw, 1996).

This approach to cognition—the embodiment hypothesis—suggests an alternative basis for cognitive processes in general. It understands perception as perceptually guided action (see below), and conceives of sensory and motor processes as being fundamentally inseparable, mutually informative, and structured so as to ground our conceptual systems (Varela, Thompson & Rosch, 1991: 173). It allows the cognisers to explore their environment with their bodies and their senses, correlating multisensory input with bodily experience through elaborate mechanisms of feedback among the sensory and motor apparatus, and with temporary information in the sensory input being matched to motor images of the body in the sensorimotor loop (Todd, 1999).

In this embodied viewpoint, the mind is no longer seen as passively reflective of the outside world, but as an active constructor of its own reality. In particular, cognition and bodily activity intertwine to a high degree in the sense that the fundamental building blocks of cognitive processes are control schemata for motor patterns which arise from perceptual interactions with the environment. This means that the drives for the cognitive system arise from within the system itself, in the form of needs and goals, and not merely from the outer world (Prem, 1996).

1 Introductory claims: dealing with music

The major topic of this paper is a processual approach to dealing with music,stressing the role of sensory-motor experience and interactions with the sounds—both at at the level of manifest and virtual interactions. It is an approach which is somewhat related to the early claims of cognitive musicology:

Goal state analysis [score analysis] of music misses the crucial point that music is above all a human experience, not merely a set of artefacts or “structures' (Laske 1977: 71).

This claim can not be overstated: it has been remolded several times (Cusick 1994, Polednak 1985) with approaches that go beyond analytical conceptions of music as a “mind-mind game”. What matters are not practices of composer’s minds for the sake of informing the practices of other minds, but the real and actually sounding experience. To quote Cusick:

...the score is not the work of the performer; nor is the score-made-sound the work: the work includes the performer’s mobilizing of previously studied skills so as to embody, to make real, to make sounding, a set of relationships that are only partly relationships among sounds (1994: 18).

Music, on this view, is something which is heard and “enacted” upon rather than being merely imagined or represented. It is not merely reducible to symbolic representations of the sounds—with the score as a prototypical example—and not to sounds as artefacts. Sounds, on the contrary, arethe outcome of human actions. Even if they are not self-produced, they can induce a kind of (ideo)motor resonance that prompts the listener to experience the sounds as if they have been involved in their production (Reybrouck 2001b). This is a claim which is somewhat analogous to the “central version” of the motor theory of perception (Scheerer 1984, Viviani 1990, see Reybrouck 2001b for a musical analogy). The key phenomenon in this conception is that motor ”intention”—rather than manifest motor behaviour—is thought to be a largely endogenous phenomenon which is localised in the central nervous system. It has been shown, in fact, that there is a motor aspect in perception and that the same areas in the brain are activated during imagined and executed actions (Berthoz 1997, Decety 1996, Jeannerod 1994).Yet, not all perception is reducible to motor components, but motor components are involved in perception and are an integral part of it. Even if they are not manifest, they operate at virtual levels of imagery and simulation—also called ideomotor simulation—with motor behaviour being manifest only at an ideational level of mental representation.

Each theoretical discourse about music, therefore, should deal to some extent with an understanding of the bodily activities—if only at a subliminal level—that are involved in the production and perception of the sounds (Blacking 1995, Iyer 2002, Mead 1999, Shove & Repp 1995). The problem, however, is rather tedious, as there is a considerable amount of music which obscures the recognition of sounds as the result of sound producing actions. This is the case, e.g., with acousmatic listening(Chion 1983, Schaeffer 1966) where the listener is encouraged to focus on the sonic properties of the sounds without any reference to their visible sources. This is a kind of “reduced listening” where acousmatic sounds are heard through loudspeakers without any clue as to their causes. These acousmatic sounds are presumed to be the basic elements of the vocabulary of “acousmatic arts” and of “musique concrète” in the sense that they go beyond an “explanatory” way of listening: there is no straight connection between the sound-as-heard and any sound-producing action.

There is, however, some natural search for auditory source-images which is typical for listening in general: auditory cognition involves source-knowledge (Bregman 1990, McAdams1984, 1993) and this in a cross-modal way (motor, kinesthetic, haptic and visual besides the purely auditory components). As such, listening can refer toauditory sources, to sound-producing actions and to the associated kinematic images (Godøy 1997b, 2001).

These findings are important: they stress the role of action and perception, and reveal the shortcomings of theories that deal exclusively with symbolic and highly computational aspects of behaviour. Higher-order behaviour, in general, is adaptive to a high degree: it relates sensory input to motor behaviour through mental mappings and correlations and through elaborate mechanisms of feedback.

These “sensory-motor” claims can be easily translated to the realm of music, if we are inclined to adopt a processual approach to music as a temporal and sounding phenomenon. Central in this approach is the concept of dealing with music, which I consider to be a generic term that encompasses traditional musical behaviours—such as listening, performing, improvising and composing—, as well as more general “perceptual” and “behavioural” categories, such as exploring, selecting and focussing of attention on the perceptual side, and actions, interactions and transactions with the (sonic) world on the behavioural side. In order to encompass all these behaviours, it is desirable, further, not to speak of listeners, or performers—as these embrace only some of the possible ways of dealing with music—but of music usersin general as a broad category of subjects that deal with music by means of one or more of these behaviours.

Each of these behaviours can be considered in isolation, but actual musical behaviour, often involves most of them—be it at a manifest or internalised level. Playing a musical instrument, e.g., is a typical example: it involves the actual production of sound as well as listening to this sound, along with mental processes such as rehearsal from memory, anticipation as to what is coming next, and evaluation and control of what is actually sounding. Other musical behaviours such as listening or composing rely on the same mechanisms with either the sensory input or motor output being skipped. They force the music user to rely on simulation and imagery in order to deal with the music at a virtual level of interaction.

The concept of interaction, however, is rather ill-defined. Much depends on the actual definition of the term: what does it mean to interact? Is interaction to be distinguished from action and transaction? And what is the relation with experience proper? Must we conceive of an actual interchange between actor and that which is enacted upon—a kind of “here-and-now-semantics”—, or should we conceive of interaction at a virtual level of imagery and representation? A critical element in this questioning is the actual relation with time and the processing of music “in time” or “out-of-time”. What counts in this distinction is the perceptual bonding with sounding stimuli which are actually present as against the same stimuli allowing imagery to complement the percepts with imaginary projections. Listening, e.g., proceeds in time, and keeps track with the unfolding over time. Composing, on the other hand, mostly proceeds out-of-time, relying on imagery and mental replicas of the sound rather than on the music as it actually sounds. Improvising, in turn, is a hybrid case: the actual process of playing proceeds in time but many of the mental computations proceed out-of-time (anticipating with respect to future playing and recollecting previous playing in memory). Real musical experiences, however, are time-consuming in keeping step with the sonorous articulation over time.All other kinds of dealing with music are secondhand and highly mediated with the mind working with mental replicas rather than with the actual sounding music.

Dealing with music “in time” calls forth ongoing processes of epistemic interactions with the sonic world. This is exemplified in figure 1 which displays two girls who are playing together in a kind of imitative play. The fixation of their gaze illustrates the way they are involved in mutual communication with the music as a mediator. Each executive movement of the fingers is motivated by input through the senses or by the activation of previous schemata in their memory. The produced sounds, in turn, are fed back to the senses allowing them to adjust them if necessary.

Figure 1. Two girls involved in imitative play, exemplifying the role of sensorimotor integration.

The example illustrates a general claim: the musician who is actually playing needs the qualitative media of actual sounding, with the meaning of music being embodied in immediate experience and relying heavily on its sensory qualities.

Sensory processing, however, is only the first stage of sense-making. In real interaction it is to be complemented by the motor part and the cognitive processes which are built on it. The latter, especially, have received considerable attention in traditional music research. There is, however, a danger of nominalism, in providing only conceptual labels for events which are characterised by a sonorous unfolding over time. As such, we should avoid to deal with music in dualistic terms—separating the body from the mind—in favour of a more holistic and integrated way, that brings together sensory experience, motor reactions and cognitive processing. The approach I am proposing, therefore, is an integrated model of “sensorimotor integration” with the music user behaving as “device” that co-ordinates input and output through input-output-mappings.

2 Music and musicological research:
interdisciplinary claims

How does music research relate to these experiential claims? Is the musical experience an established topic of research or is it to be found only in the margins of musicology as a discipline? The answer is not obvious. There are, in fact, prevailing paradigms which run through musicology such as historical research, music analysis and performance studies. On the other hand, new paradigms are evolving which challenge traditional approaches by stressing some weaknesses and shortcomings of traditional approaches. I see five major topics here: (i) the subject matter of much traditional music research is too narrow in focussing primarily on the western canon of art music (the common-practice tradition), (ii) music research is basically object-oriented and deals with second order stimuli, relying on symbolic transcriptions of the music rather than on the music as it sounds, (iii) there is a lack of operational terminology for describing both the music as a temporal art and the process of dealing with the music—what is needed is a descriptive as well as an explanatory vocabulary with terms that refer unambiguously to what they stand for; (iv) there is need of techniques for visualising and recording both the music and the reactions to the sounding music in an way that does justice to the scientific claims of exactness, completeness and repeatability.

Many of these claims are current topics of research. I only mention the contributions from cognitive and computational musicology, from acoustics and computer sciences, besides some recent neurobiological and psychological studies. Many of theses disciplines, however, are working in isolation with only little connections with each other. What I argue for, therefore, is an interdisciplinary approach that brings together contributions from different fields that are all related to the process of dealing with music. This common field is not yet established as an official research community—with institutions, official journals and academic positions—but there are at least emerging research communities which focus on a kind of common paradigm which is articulated by four major claims: (i) music as a sounding art; (ii) the process of dealing with music; (iii) the role of the musical experience and (iv) the process of sense-making while dealing with music.

(i) The first claim is an ontological claim: it states that music is only real music when it sounds. This is an empiricist claim that stresses the firsthand information in perception rather than relying on second-order stimuli. It means that we should conceive of “music-as-listened-to” and “as-perceived”, rather than thinking and conceptualising of music at a mere symbolic level without any actual connection to the music as it sounds. The claim is important as it challenges symbolic approaches which deal with music merely at a mental level. The act of composing is a paradigm case, but even sight-reading and score analysis deal with notes—as symbolic reference to sounding things—rather than with music as it sounds.

(ii) The second claim concerns the role of interaction with the sound, either at the actual level of real sounding music or at the virtual level of imagery and representation.

Real time interaction, e.g., mostly relies on conservative behaviour: it keeps step with the actual unfolding over time in an attempt to hold possible disturbances between fixed limits. It is obvious in many musical applications such as the traditional pedagogy of instrumental teaching—where the apprentice tries to imitate the teacher’s instrumental playing—, the act of playing music from a score and the act of improvising. Most of these interactions combine motor output and sensory processing with the aim to deviate as little as possible from the standard of performance (the target performance which they hear imagine). In all these cases, the mind is functioning as a central processing mechanism that co-ordinates the input-output mappings.

It is possible, however, to go beyond conservative behaviour and to perform mental operations which transcend the inexorable character of time-bound reactivity. As such, the mind operates at a level of virtual simultaneity which is working “out-of-time” through mechanisms of anticipation and memory. This is the level of mental computations and symbolic play (Reybrouck 2002, 2006a) which considers the input/output couplings which can be handled in terms of modelling or predictive computations (Bel & Vecchione 1993) and which entail the basic idea of the homo ludens as a playing automaton.

This is, in fact, a broader conception of computation—computation is considered mainly from a symbol-processing point of view with as basic idea formal symbol manipulation by axiomatic rules—which embraces the whole field of “mental operations” which can be performed on symbolic representations of the sound. They allow the music user to have access simultaneously to events which are stored in memory or which canbe created “de novo” in imagery and to achieve a level of “epistemic autonomy” in the transactions with the sonic world.

(iii) The role of musical experience, thirdly, has been rather marginal in existing musicological research. There are psychological studies (Deutsch 1982, McAdams &Bigand 1993, Tighe & Dowling 1993, Sloboda 1985,1988, and Hodges 1996 for an overview) and music reception and cognition studies (Deliège & Sloboda 1997,Francès 1958, Handel 1989, Swanwick 1994), but musicology as a discipline is still waiting for a comprehensive and theoretically grounded framework that explains the idiosyncrasies and commonalities of listening behaviour. Yet there is a considerable body of older theoretical writings that have dealt extensively with the topic of having an experience (Dewey 1925, 1934/1958, James 1912/1976, 1968). These writings, however, did not yet receive much attention in musicological research (see Määttänen 1993, Reybrouck 2004b, 2005a, Westerlund 2002). Central in their claims is the tension between percept and concept, and between the particularities of the sensory experience and the conceptual labels that are applied to them.

Theoretical writings, however, have the danger of being speculative and intuitive at times. It is interesting, therefore, to look for empirical findings that can support their claims, and much is to be expected from the growing field of music and brain studies. It offers a vast body of empirical grounding for the theoretical framework that is related to having a musical experience (Peretz & Zatorre 2003, Wallin 1991, Wallin, Merker & Brown 2000, Zatorre & Peretz 2001).

(iv) The process of “sense-making”, finally, implies a shift from ontological (what is music?) to epistemological questions (what is music cognition and how can it be acquired?) with as major claim the “construction” of meaning out of the perceptual flux (Reybrouck 2001a, 2002, 2005a). It involves a semiotisation of the sonic world which means that we must conceive of the music user not as a mere passive recipient but as an organism that tries to build up semiotic linkages with the world. In building up these linkages, he or shecan rely on innate and acquired mechanisms of information “pickup”and information “processing”. As such, there is a tension between wired-in reactivity to the (sonic) environment—with reactions that behave like lock-and-key—and mediate reactions which are the outcome of learning processes and cognitive mediation.

The contributions of biosemiotics should be mentioned here: they provide important insights about signification processes which are typical for living organisms in general and which are rooted in their biology (for an overview, see Kull 2001, and for an application to music: Reybrouck 2001a, 2005a). Being an interdisciplinary field of theoretical and empirical research of communication and signification in living systems, it focuses on the study of the behaviour of living systems in their interaction with the environment.

3 Theoretical and empirical grounding:
knowledge construction and the coupling of action and perception

New paradigms are not emerging in a vacuum: they are mostly the outcome of theoretical insights and empirical facts that point to some converging claims. This is true especially for semiotics which has been very programmatic in its initial claims but which is now moving steadily and gradually to a more evidence-based discipline. The program, then, is to bring theory and facts together in a coherent and comprehensive framework. To quote Morris:

There is need both for fact-finders and for systematizers. The former must makeclear the conditions under which semiosis occurs and what precisely takes place in the process, the latter must in the light of available facts develop a precise systematized theoretical structure which future fact-finders can in turn use (1938/1975: 56).

The theoretical framework I want to advocate is centred around the construction of meaning, which, in turn, can be considered as a kind of adaptation to the environment (Reybrouck 2005a, b). Knowledge, on this view, serves an organism to organise its experience—as was maintained already by Piaget (see von Glasersfeld 1991: XVIII): it is not a representation of an independent reality but an organism-dependent way of knowing that is relative to its way of experiencing.

A beautiful example of this claim is von Glasersfeld’s doctrine of radical constructivism (von Glasersfeld 1978, 1982, 1995a, b). Starting from Kant—especially his Critic of Pure Reason—who stated that whatever we call knowledge is necessarily determined to a large extent by the knower’s way of perceiving, and conceiving and restating the epistemological claims of Piaget who tried to describe by what means the human mind goes from a state of less sufficient knowledge to a state of higher knowledge, he introduced a theory of “active knowing” which is the outcome of our own construction:

... knowledge is not passively received but built up by the cognizing subject, and the function of cognition is adaptive and serves the organization of the experiential world, not the discovery of ontological reality (von Glasersfeld 1995b: 18).

Knowledge, on this view, is in the heads of the persons, and is constructed as the outcome of their experience.

These claims may sound somewhat speculative and remote from current musicological research. There is, however, a lot of empirical evidence—especially from the field of neurobiological research and “music and brain” studies—that points in the direction of musical applications. I only mention four topics: (i) studies concerning the plasticity of the brain, (ii) the role of adaptation, (iii) the search for neural correlates of skill acquisition and the (iv) search for anatomical markers of musical abilities and competence.

The search for “anatomical markers” of musical skills, firstly, has resulted in rather interesting findings which point to structural and functional adaptation of brain tissue in response to intense environmental demands during critical periods of brain maturation. Dealing with music (processing and performance), in particular, is a skilled activity that is dependent on several higher functions of the brain: it requires the simultaneous integration of multimodal sensory and motor information with multimodal sensory feedback mechanisms to monitor performance (Gaser & Schlaug2003).

This neural “plasticity” is typical of adaptive behaviour. It favours multiple interactions with the world—both at the sensory and motor level—and is related to the field of sensorimotor learning. To quote Paillard:

The concept of sensorimotor integration is an important notion in biology. It poses the question as to the origin of structural solidarities and functional cohesions that are to be found in the individuation of biological systems and the interdependency of the organism and its environment. The individual memory, for example, is the outcome of its capacity to control its present activities in terms of personal experiences that are the outcome of previous activities. The organism enriches, in a way, its repertory of genetic adaptations in supplementing it with acquired dispositions that are immediately at hand and mobilizable when confronted with a situation that can be foreseen or recognized as a familiar one (Paillard 1994a: 925).

Sensory-motor co-ordinations are important strategies: they are “conservative” in keeping step with the actual perceptual flux and extend the level of interaction with the environment, ranging from mere sensory processing, to active behaviour and co-ordination between them. This is quite obvious in instrumental playing: a musician who is playing a violin “shapes” his sound through the mechanisms of sensory-motor integration and feedback. The same holds true also for listening, if we conceive of it as an internalised simulation of the actual performance of the music. As such there is a continuity between sensory-motor integration and ideomotor simulation, the former dealing with movements that are actually executed and manifest—as in the actual production of musical sounds—and the latter dealing with movements that are simulated at an internal level of imagery (Paillard 1994b, Reybrouck 2001b).

Ideomotor simulation, further, has a major role in “motor learning” and “enactive listening”: there is, in fact, a close relation between motor imagery and motor execution with empirical evidence showing that both processes involve activities of very similar cerebral motor structures at all stages of motor control (Crammond 1997). It means, further, that we can conceive of music in motor terms without actually performing movements. But even more important is the coupling of perception and action, in the sense that motor imagery is involved in perception—sometimes called the motor theory of perception (Berthoz 1997, Jeannerod 1994, Mackay 1992, Scheerer 1984, Viviani & Stucchi 1992, and Reybrouck 2001b for a musical analogy). There is, in fact, a growing body of research that focuses on the relation between action and motor imagery (Annett 1996, Berthoz 1997, Decety 1996, Deecke, 1996, Jeannerod 1994, Paillard 1994a, and for musical applications: Godøy 2001, Gromko & Poorman 1998) and on the concept of perception as simulated action and as active exploration (Berthoz 1997).

The claims are appealing: they point in the direction of “action-oriented” research, in addition to the already existing psychology of “perception”. What really matters here is a shift from rather static categories of perception to functional categories that integrate perceptual attributes and classes of action (Mazet 1991: 100). It prompts us to consider the role of the body with its sensory and motor interfaces and to conceive of music cognition in “embodied” and “experiential” terms.

4 Experiential cognition and cognitive semantics

The transition from “static” categories of music knowledge to a more “dynamic” and “processual” approach is motivated by theoretical grounding, which is related to two major topics: the concept of processand the philosophy of time.

The “processual” approach can hardly be overstated: music, as a temporal art, is characterised by consumption of time. In distinction to, e.g., a geometric figure which is presented at a glance, it relies on the successive presentation of its component parts. As such, there must be some completion—also called point of condensation (Francès 1958)—before the music user can make sense of the music. In order to do so, he or she should bring together the particularities and idiosyncrasies of the sonorous unfolding and the more overarching principles of relational continuity. As such, there is a basic tension between the discreteness and successivity of small temporal windows and the more global synoptic overview (seefigure 2). The latter allows the music user to grasp the music in a simultaneous act of consciousness or comprehension (Godøy 1997a, Reybrouck 2004a) in order to conceive of it at a more global level of representation. Such global processing is an “economic” way of processing: it affords a global overview but at the cost of the richness and fullness of the full perceptual experience.

Figure 2. Music as motion. The top figure depicts the listener as a still spectator who can distance at will in order to recollect a small or bigger portion of the music. The lower part depicts the music in a kind of representational format allowing the listener to navigate through the music and to inspect its unfolding over time through a small temporal window.

The tension between “richness of experience” and “economy of processing”, further, isnot typical of music. It has been elaborated already extensively in the pragmatic philosophy of Dewey (1934/1958) andJames (1912/1976) with as major topic the concept of “having an experience”. As Dewey puts it:

Experience in the degree in which it is experience is heightened vitality. Instead of signifying being shut up within one’s private feelings and sensations, it signifies active and alert commerce with the world; at its height it signifies complete interpenetration of self and the world of objects and events (1934/1958: 19).

This heightened vitality has “adaptive value” as well: it is exemplified in the life of the savage man who is in danger in a threatening environment. Observation, for him, is both “action in preparation” and “foresight for the future” with the senses functioning as sentinels of immediate thought and outposts of action. They are not mere pathways for gathering material that is stored away for a delayed and remote possibility (Dewey 1934/1958: 19).

Having an experience, further, is not unidirectional with the senses as the only interface. According to Dewey, it has pattern and structure because it is doing and undergoing in relationship. It is exemplified most typically in the artistic experience:

...art, in its form, unites the very same relation of doing and undergoing, outgoing and incoming energy, that makes an experience to be an experience.—Man whittles, carves, sings, dances, gestures, molds, draws and paints. The doing or making is artistic when the perceived result is of such a nature that its qualities as perceived have controlled the question of production. The act of producing that is directed by intent to produce something that is enjoyed in the immediate experience of perceiving has qualities that a spontaneous or uncontrolled activity does not have. The artist embodies in himself the attitude of the perceiver while he works” (1934/1958: 48). And further: “Without external embodiment, an experience remains incomplete—It is no linguistic accident that `building', `construction', `work', designate both a process and its finished product. Without the meaning of the verb that of the noun remains blank (Dewey 1934/1958: 51).

The perceptual experience, further, has another quality: it is characterised by full and rich experience:

... the object of—or better in—perception is not one of a kind in general, a sample of a cloud or river, but is this individual thing existing here and now with all the unrepeatable particularities that accompany and mark such existences. In its capacity of object-of-perception, it exists in exactly the same interaction with a living creature that constitutes the activity of perceiving (Dewey 1934/1958: 177).

Most objects of our ordinary perception lack this completeness, being cut short as soon as there is an act of recognition. The full perceptual realisation of just the individual thing we perceive is replaced by the identification of something that acts as an index of a specific and limited kind of conduct. Aesthetic perception, on the other hand, is characterised by full perception.

A musical experience, on this view, is not basically different from an auditory experience at large. It is continuous with the natural experience or experience proper (see Dewey 1934/1958) with a difference in degree rather than in quality. Or put in more general terms: the connection of art and aesthetic perception with experiences at large does not signify a lowering of their significance and dignity. Nor is the attempt to connect the higher and ideal things of experience with basic vital roots to be regarded as betrayal of their nature and denial of their value. It is legitimate, on the contrary,to bring the high achievements of fine art into connection with common life, the life that we share with all the living creatures (Dewey 1934/1958: 20).

A somewhat related approach was advocated by James (1912/1976) who introduced his doctrine of radical empiricism, which is an original epistemology that deals with the tension between “concept” and “percept”. It stresses the role of knowledge-by-acquaintance—as the kind of knowledge we have of a thing by its presentation to the senses—and states that the significance of concepts consists always in their relation to perceptual particulars. What matters is the fullness of reality which we become aware of only in the perceptual flux (1911/1968). Conceptual knowledge is needed only in order to manage information in a more “economical” way. As such, it is related to principles of cognitive economy, or as James himself puts it:

We extend our view when we insert our percepts into our conceptual map. We learn about them, and of some of them we transfigure the value; but the map remains superficial through the abstractness, and false through the discreteness of its elements; and the whole operation, so far from making things appear more rational, becomes the source of quite gratuitous unintelligibilities. Conceptual knowledge is forever inadequate to the fullness of the reality to be known. Reality consists of existential particulars as well as of essences and universals and class-names, and of existential particulars we become aware only in the perceptual flux. The flux can never be superseded (James 1912/1976: 245).

A related position has been advocated by Dewey, who states that

... the object of—or better in—perception is not one of a kind in general, a sample of a cloud or river, but is this individual thing existing here and now with all the unrepeatable particularities that accompany and mark such existences. In its capacity of object-of-perception, it exists in exactly the same interaction with a living creature that constitutes the activity of perceiving (Dewey 1934/1958: 177).

Most objects of our ordinary perception lack this completeness, being short-circuited as soon as there is an act of recognition. The full “perceptual realisation” of just the individual thing we perceive is then replaced by the “identification” of something that acts as an index of a specific and limited kind of conduct.

Aesthetic perception, of course, is characterised by “full perception” that stresses the fullness and richness of perception. The same also holds true for listening to music, which is both an “experiential” and a “conceptual” matter: consisting of sensory realia as well as their symbolic counterparts, it embraces both “perceptual immediacy” and “conceptual abstraction”.

Experience, however, is not only related to the richness of perception. It has a role in the construction of knowledge with meaning being characterised in terms of the “experience” of the human beings who are doing the cognising. As such, it is a basic claim of cognitive semantics (Jackendoff, 1987, Johnson, 1987, Lakoff, 1987, 1988) which accounts for

what meaning is to human beings, rather than trying to replace humanly meaningful thought by reference to a metaphysical account of a reality external to human experience (Lakoff 1987: 120).

The same holds true for conceptual or mentalistic semantics which state their priority over real semantics in stating that

one cannot take for granted the “real world' as the domain of entities to which language refers. Rather, the information that speakers can convey must be about their construal of the external world, where one's construal is the result of an interaction between external input and the means available to internally represent it (Jackendoff 1987: 83).

There are two major claims in this approach: non-objectivism and embodiment. The first (non-objectivism) conceives of meaning as a matter of human understanding—it is called also a semantics of understanding (Fillmore, 1984)—and constitutes our experience of a common world we make sense of (Johnson, 1987). Cognition, on this view, must not be considered as a “recovery” or “projection” that exists out-there (objectivism). Rather, it stresses the role of experience and interaction with the environment:

...cognition is not the representation of a pregiven world by a pregiven mind but is rather the enactment of a world and a mind on the basis of a history of the variety of actions that a being in the world performs(Varela, Thompson & Rosch, 1991: 9).

The embodiment hypothesis, on the other side, takes an epistemological position of “experiential realism”. It grounds our cognitive activity in the embodiment of the actor and the specific context of activity and defines cognition as

embodied action that depends upon the kinds of experience that come from having a body with various sensorimotor capacities which are embedded in a more encompassing biological, psychological, and cultural context(Varela, Thompson & Rosch, 1991: 173).

The epistemological claims of experiential and enactive cognition (Johnson, 1987, Lakoff, 1987, Varela, Thompson & Rosch, 1991) are intertwined with this approach. They function as a typical example of non-objectivist semantics, which defines meaning as a matter of human understanding which is highly dependent upon structures of embodied imagination and which highlights the dynamic, interactive character of meaning and understanding (Johnson, 1987:175). It is a position which has received attention in recent developments in cognitive science, with a move toward the inclusion of the body in the understanding of the mind. As such, it is related also to the “experiential approach” in cognitive linguistics (Fauconnier, 1985, Lakoff, 1987, Lakoff & Johnson, 1980, Langacker, 1986, 1987, Sweetser, 1990) which states that the fundamental conceptual representations in the human cognitive system are schematic perceptual images extracted from all modes of experience.

According to Barsalou et al. (1993), there are three assumptions that underlie this “experiential” approach: (i) the perceptual representations from experience that represent concepts are not holistic analog images but compositional images that are built analytically from smaller component images; (ii) perceptual representations are much sparser than most actual perceptions: they are abstract and schematic; and (iii) perceptual representations do not arise solely from vision but from any aspect of experience, including proprioception and the introspection of representational states, information processing operations and emotions. The representations are not “perceptual” in the traditional sense, but they are more generally “experiential”, arising from any aspect of experience during perception of the external and introspection of the internal world.

Cognitive scientists, further, have begun to infer connections between the structure of mental processes and physical embodiment (Iyer, 2002). It is a viewpoint which is known as embodied or situatedcognition and that treats cognition as an activity that is structured by the body situated in its environment—as embodied action.

This is, in fact, a theory of cognitive organisation,and the writings by Lakoff and Johnson (Johnson, 1987, Lakoff, 1987, 1990, 1993, 1994,Lakoff & Johnson, 1980, and for a musical application: Bowman, 2000, Brower, 2000, Cano, 2003, Cox, 2001, Saslaw ,1996, Zbikowski, 1997-98, 1997, 2002, Walser, 1991, Iyer, 2002, Lidov, 1987, Reybrouck, 2001a, 2005a)can serve as a primary orientation here. They have proposed useful ways of reconceiving the nature of linguistic meaning, stressing the role of metaphor as a basic structure of understanding and the role of the body in providing “cross-domain mappings”. Metaphors, in fact, make it possible to conceptualise an unfamiliar domain—the target domain—in terms of another more familiar domain—the source domain (see Zbikowksy, 1997 for a critical review). The human body, then, can function as a primary source for this mapping.

Central in this approach is the focus on image schemata which are based on direct experience of a “kinesthetic” nature. They are defined by Johnson as “recurring, dynamic pattern[s] of our perceptual interactions and motor programs that give[s] coherence and structure to our experience ” (Johnson, 1987, xiv). As such, they are operating continuously in perception, bodily movement through space and physical manipulation of objects.

Image schemata, further, are not propositional—in the sense of abstract subject-predicate structures—but they exist in a continuous, analog fashion in our understanding (Johnson, 1987: 23-24 & 44). They function as gestalt-like structures that organise our mental representations at a very basic “preconceptual” level of cognitive organisation. As such, they are not rich, concrete images or mental pictures, but structures that are more general and abstract than the level at which we usually form particular mental imagetargets of the world (Johnson, 1987: xxxvii, see also Saslaw, 1996).

Image schemata are embodied—they arise from the human body—and entail metaphorical projections from the kinesthetic source domain. As such they rely on non-arbitrary “cross-domain mappings” that function as agents of conceptual organisation (Lakoff, 1993: 203 and for empirical evidence: Barsalou, 1999, Barsalou et al. 1993, Gibbs & Colston, 1995) and that are motivated by biological factors as well (Edelman, 1989: 295, Edelman 1992: 252, Edelman, 1995: 43).

Several kinds of image schemata have been identified (Lakoff ,1987, Johnson, 1987). According to Saslaw(1996), there are two major types: those that deal with our bodies themselves (container, center-periphery, front-back, part-whole) and those that deal with our orientation in, and relationship to the world (link, force, path, source-path-goal, near-far).

There is no place to go in detail here. I only mention two of them which are likely to be relevant for the process of dealing with music (see Brower, 2000,and Saslaw, 1996 for an overview): the container schema and the source-path-goal schema (see figure 3).

Figure 3. Diagrams of the container schema (left) and the source-path-goal schema (right) (free after Brower 2000).

The container schema, as an example of the first type, derives from the conception that the human body is a container with an interior, an exterior and a boundary between them. It is a pervasive mode of understanding everyday experiences in terms of “in” and “out”, with some of these orientations being kinesthetic and directly experienced, while others are related to mental states which are treated as containers our minds are in.

The source-path-goal schema, as an example of the second type, is an important tool in structuring our conception of music as a temporal art. It is grounded in the bodily experience of moving from a starting place to a kind of destination and is defined by four structural elements: (i) a source or starting point, (ii) a destination or end point (or goal), (iii) a path or sequence of contiguous locations connecting the source and the destination; and (iv) a direction toward the destination. The basic logic of the schema is that in proceeding from source to destination along a path, we must go through all the intermediate points on the path and that the further along the path we are, the more time has passed since starting (Lakoff, 1987: 275-278, see also Saslaw, 1996: 221).

The musical analogies of these image schemata are obvious. They have been belaboured by Brower (2000) who argued convincingly for the role that three of their most important features—containers, pathways and goals—should play in the structure and elaboration of musical plot (see also Saslaw ,1996). The basic metaphor of goal-directed motion, e.g., is supported by mappings of tonic as centre and ground, triads and keys as nested containers, scales and arpeggios as pathways for melodic motion, circles of fifths and thirds as pathways for harmonic motion, and tonal motion as subject to forces of gravity, inertia, and tonal attraction. All these basic-level metaphors, further, may undergo extensive elaboration in the context of a musical work. As such, they can be captured in the form of schemas for musical plot structure (see figure 4).

Figure 4. Schemas for musical plot structure (after Brower 2000, with permission).

5 The role of the subject in interaction:
Jakob von Uexküll and cybernetics

The “enactive” approach to cognition grounds cognitive activity in the embodiment of the actor and stresses the role of the cogniser as a “subject” who constructs and organises his/her knowledge. This claim—already advocated by Kant—was reformulated by Jakob von Uexküll who proposed nothing less than a new research programme:

All reality is subjective appearance. This had to be the great, fundamental understanding also of biology ... Kant has put the subject, man, in opposition to the objects and has discovered the fundamental principles according to which the objects are formed in our mind. The task of biology is to widen the results of Kant’s research in two directions: 1. to take into account the role of our body, especially of our sense-organs and our central nervous system and 2. to explore the relations of other subjects (the animals) to the objects(von Uexküll, 1973: 9-10).

Von Uexküll’s claims have been rediscovered recently as a new paradigm for biology and semiotics (Kull, 2001). They are appealing for at least three reasons: (i) they stress the role of the subject and of subjectivity, (ii) they provide a firm theoretical grounding and a very operational terminology for describing the process of interaction of an organism with its environment, and (iii) they fit in very easily with current conceptions about cybernetics and artificial devices. In what follows, I will elaborate on each of them, in an attempt to provide a viable framework for an operational description of the process of sense-making as related to music.

5.1 Von Uexküll, subjectivity and second-order cybernetics

Jakob von Uexküll was a biologist who developed an original and very operationaltheory of meaning—commonly known as Umweltlehre—, which focuses mainly on the interaction with the environment and the role of sense-making and construction of knowledge as the outcome of this interaction (von Uexküll, 1937, 1934/1957, 1920/1973, 1940/1982).

The basic idea behind this theoretical construction is the assumption that an organism perceives the world through a network of “functional relations” which constitutes its own phenomenal world or Umwelt. This is the world around animals as they themselves perceive it and which has been described by von Uexküll in rather poetic terms:

The best time to set out on such an adventure is on a sunny day. The place, a flower-strewn meadow, humming with insects, fluttering with butterflies. Here we may glimpse the worlds of the lowly dwellers of the meadow. To do so, we must first blow, in fancy, a soap bubble around each creature to represent its own world, filled with the perceptions which it alone knows. When we ourselves then step into one of the bubbles, the familiar meadow is transformed. Many of its colorful features disappear, others no longer belong together but appear in new relationships. A new world comes into being. Through the bubble we see the world of the burrowing worm, of the butterfly, or of the field mouse; the world as it appears to the animals themselves, not as it appears to us. This we may call thephenomenal worldor the self-world of the animal” (von Uexküll 1934/1957: 5). And further: “We no longer regard animals as mere machines, but as subjects whose essential activity consists of perceiving and acting. We thus unlock the gates that lead to other realms, for all that a subject perceives becomes his perceptual world and all that he does, his effector world. Perceptual and effector world together form a closed unit, the Umwelt. These different worlds, which are as manifold as the animals themselves, present to all nature lovers new lands of such wealth and beauty that a walk through them is well worth while, even though they unfold not to the physical but only to the spiritual way (1934/1957: 6).

Each “Umwelt” describes the phenomenal world of an organism: it is a collection of “subjective meanings” which are imprinted upon all objects of a subjective subset of the world at large, and which include all meaningful aspects of the world for a particular organism. Umwelts, on this view, are not “out there”, but are constructed and constrained:

The Umwelt of any animal that we wish to investigate is only a section carved out of the environment which we see spread around it ... The first task of Umwelt research is to identify each animal’s perceptual cues among all the stimuli in its environment and to build up the animal’s specific world with them (von Uexküll 1934/1957: 13).

In order to make his claims more operational, von Uexküll has introduced two additional concepts: the concepts of “functional tone” and “functional cycle”.

The first—functional tone—illustrates the sensitivity of organisms—both animals and human beings—to the functional characteristics of their environment. There simply is no one-to-one relationship between an object in the outer world and its meaning. Its actual meaning is rather dependent on a number of different qualities or tones which are, in turn, dependent on the intentions that the organism confers on it. Von Uexküll mentions the example of a tree: it can function as a shelter for a fox, a support for the oil, a thoroughfare for the squirrel, hunting grounds for the ant, egg-laying facilities for the beetle and a source of valuable raw material for the forester (von Uexküll, 1934/1957). Another example is a barking dog which threatens a walker on a rural path. The walker may find a stone on the ground and throw it to the dog in order to chase him off. The stone, which had a “path-quality” up to this moment, gets a “throwing-quality” on this occasion. What matters, then, are not merely the objective qualities of the stone, but the functional qualities it affords to the user of the stone at a particular occasion.

Umwelt research is closely related to the study of functional tones. The latter, in fact, are decisive for the actual description of an Umwelt. As von Uexküll puts it:

The Umwelt only acquires its admirable surety for animals if we include the functional tones in our contemplation of it. We may say that the number of objects which an animal can distinguish in its own world equals the number of functions it can carry out. If, along with few functions, it possesses few functional images, its world, too, will consist of few objects. As a result its world is indeed poorer, but all the more secure. [...]As the number of an animal’s performances grows, the number of objects that populate its Umwelt increases. It grows within the individual life span of every animal that is able to gather experiences. For each new experience entails a readjustment to new impressions. Thus new perceptual images with new functional tones are created (von Uexküll, 1934/1957: 49).

As such, there is not merely “one” Umwelt, but multiples of them with the organism’s perceptual and effector cues functioning as functional tones.

These perceptual and effector cues define the kinds of interactions with the environment. They function as parts of the functional cycle or functional circle (see figure 5) which provides a basic schema for the interactions between the (human or animal) organisms and the objects of their surrounding worlds:

Figuratively speaking, every animal grasps its object with two arms of a forceps, receptor and effector. With the one it invests the object with a receptor cue or perceptual meaning, with the other, an effector cue or operational meaning. But since all of the traits of an object are structurally interconnected, the traits given operational meaning must affect those bearing perceptual meaning through the object, and so change the object itself (von Uexküll, 1934/1957: 10).

Figure 5. The functional cycle, after von Uexküll (1934/1957).

Interactions, on this view, consist principally of “perception” and “operation”: a neutral object from the environment is “harpooned” as a meaning-carrier by a perceiving organ in order to be modified by an effector organ (as meaning-utiliser) in such a way that it disappears altogether from the surrounding world (von Uexküll 1987: 170). Object and subject are in a way dovetailed into each other in order to constitute a systematic whole:

Every action that consists of perception and operation imprints its meaning on the meaningless object and thereby makes it into a subject-related meaning-carrier in the respective Umwelt (von Uexküll, 1940/1982: 31).

This is an important claim: it brings together the process of “sense-making” and the concept of “circularity”, as well as the role of “subjectivity” and its influence on our reactions to the environment. The emphasis on subjectivity, further, is an important link to second order cybernetics which emphasises the role of the knower and observer rather than the known things or events (Maturana 1978, 1988, Maturana & Varela, 1987,Pask, 1975, 1992, Spencer Brown, 1969, von Foerster, 1974, see also Brier, 2000a, b, Heylighen & Joslyn 2001). As Maturana puts it:

... we are seldom aware that an observation is the realization of a series of operations that entail an observer as a system with properties that allow him or her to perform these operations, and, hence, that the properties of the observer, by specifying the operations that he or she can perform determine the observer’s domain of possible observations...(1978: 28-29).

Arguing on these lines, we should conceive of dealing with music in “epistemological” terms (Reybrouck, 2003b): as music users, we are observers who construct and organise our music knowledge and bring with us our observational tools. For doing so, we can rely on strategies of sense-making and cognitive activities such as exploring, selecting, modifying and focussing of attention. Listening, on this view, is not merely the passive registration of an outer sonic world but an active process of selection and intentionality that involves a whole machinery of semanticity and semiotisation: what we are listening to are not merely “sounding things”—the sensory “realia”—, but “things as signs” which shape our (sonic) world (Reybrouck, 1999).It is up to the listeners, then, to go beyond the sensory material in order to delimit the elements and the relations that they consider to be significant and eligible to function in the larger framework of a piece of music (Reybrouck 1999, 2003a).

5.2 Cybernetics and the concept of circularity

Dealing with music, as I conceive of it, is a process of “sense-making” and of “adaptive control” (see below). It relies on perception, action and the mutual relations and co-ordinations between them. Embracing the major moments of perceptual input, effector output, central processing and feedback, it fits in with the cybernetic concept of a control system (see figure 6).

Figure 6. The major moments of a control system.

This linkage with cybernetics is important: cybernetics, as a unifying discipline, brings together concepts as different as the flow of information, control by feedback, adaptation, learning and self-organisation (see Bateson, 1972, Brier 1999a, b, Cariani, 2003). It allows, further, to consider such important things as input-output correlations, the mappings between sensory input and motor output, the computations at the representational level of the brain and the role of feedback.

The latter, especially, is an interesting extension of the linear stimulus-reaction chain: it substitutes a closed loop for an open loop and challenges the mere reactive approach to sensory stimulation (open loop) in favour of a dynamic concept of circularity (closed loop) which brings together perception and action in a continuous process of sense-making and interaction with the environment. It is a very fruitful approach which has received a lot of theoretical grounding and empirical support in recent contributions (Annett, Arbib, 1981,1996, Berthoz, 1997, Decety, 1996, Deecke, 1996, Jeannerod, 1994, Meystel, 1998, Paillard, 1994a). The basic claims, however, have been advocated already in the seminal contributions of von Uexküll and Piaget.

The role of circularity is obvious in Jakob von Uexküll’s writings—his key concept of “functional cycle” is illustrative at this point (see above). Piaget’s claims about reflexive action (1937, 1945, 1967), however, should be mentioned here as well. Reflexive action, as he conceives of it, essentially consists of three parts: (i) a pattern of sensory signals (the stimulus), (ii) an activity which is triggered by the particular pattern of sensory signals (the response) and (iii) the experience of some change which is registered as the consequence of this activity and which turns out to be beneficial for the actor (see also von Glasersfeld, 1995b: 153). The parts, taken together, build up an action schema which increases the internal organisation of the organism, allowing it to act in the face of perturbation. As such, it supersedes the traditional concept of the reflex arc—as a “linear” stimulus-reaction chain—in favour of a basic principle of sensorimotor learning that goes beyond pure reactivity tosensory stimulation: a situation as it is perceived leads to an activity that is evaluated in terms of its beneficial or expected results (see also von Glasersfeld, 1995b: 65). What matters, therefore, are not merely the actions proper but their results. The “circularity” of stimulus and reaction, therefore, is a central topic in our epistemic interactions with the world. Or, as von Uexküll puts it: phenomena which are neutral in a way receive properties which they do not have independently from the reacting subject:

Without the readiness to react, there can be no stimulus—with the cessation of the readiness to react, the stimulus ceases to be a stimulus—and without a stimulus there can be no reaction (von Uexküll, T. 1986a: 122-123).

6 Levels of processing

Human beings are not cameras that capture the world in a kind of photographic image, but active perceivers that create their universe. They are involved in processes of sense-making that go beyond mere reactivity to sensory stimulation. As such, there is no one-to-one relation between the continuous flow of stimuli in the physical world and the perceptual and cognitive processes of the perceiver. Weshould consider, for short, the role of the way how human perceivers structure this perceptual flow.

The same holds true for listening to music, which is not merely the passive registration of acoustic stimuli. There is, in fact, a critical distinction between hearing and listening (Chion, 1983, Handel, 1989, Schaeffer, 1966): the former is a purely physiological process, the latter leans upon psychological factors such as attention and motivation. But even if there is investment ofattention, there is an additional distinction between acoustical or auditory listening: acoustical listening is listening in terms of the acoustic qualities of the music; auditory listening is a process of sense-making that goes beyond the mere acoustical description of the sound (Handel, 1989).

There is, further, a basic tension between the bottom-up and top-down approach to music cognition. Do we process the huge amount of sensory information which is presented to the senses (bottom-up), or do we rely on cognitive mediation, with the mind weighing and selecting the sensory stimuli in terms of cognitive categories (top-down)? Music cognition, as I conceive of it, is not merely reducible to naive realism which takes the sounding stimuli for granted. It is dependent, on the contrary, on principles of cognitive filtering and mediation (Reybrouck 2004b, 2005a).

An important aspect in this approach is the transition from sensation to perception. Perception, as it was originally conceived, is the experience of objects and events which are actually presented to the senses, with the exclusion of all things that are not sensed directly. In contrast to sensation, it is used for the more “general” aspects of this activity, reserving the term sensation for those facts in our experience which depend upon the activity of our sense organs (Boring, Langfeld & Weld, 1948: 216):“sensation” is merely the conscious response to the stimulation of a sense organ; “perception” involves the whole datum that is presented to our consciousness. It requires selection among sensations, combination, organisation and sometimes even supplementations from imagination (Lee, 1938: 25).

In order to elaborate on this distinction, it is possible to rely on levels of processingwhich are rooted in our biological functioning. Ranging from the level of the reflexes as well as the higher-level cognitive processing of the brain, there is a continuum between lower-level reactivity, over mere information pick-up to higher-level information processing that relies on several higher functions of the brain (Reybrouck 1989, 2001a).

At the lowest level, there is mere “reactivity” to the sound without any cognitive mediation by the mind. This is the level of “causality” with specific stimuli eliciting specific reactions. It involves a reactive machinery that functions as a kind of lock-and-key—with wired-in and closed programs of behaviour that trigger reactions in a quasi-automatic way. As soon, however, as there is some cognitive activity, the organism goes beyond causality by introducing “intermediate variables” between stimulus and reaction (Paillard,1994b, Reybrouck, 2001b). This is cognitive mediation or cognitive penetration—to coin Pylyshyn’s (1985) term—which allows the music user to deal with music at a representational level of virtuality rather than merely reacting to the music as actually sounding stimuli.

The process of “mediation” is a major topic in coping with the sounds: it allows the perceiving organism to carry out adaptive changes in its transactions with the environment and marks off “human” and “animal” existence. As Werner & Kaplan put it:

In animals—particularly in lower animals—organism and environment are closely attuned to each other; one might say that both are elements in a comparatively closed system, within which stimulation and response are tightly interlocked. With ascendence on the evolutionary scale, the closed system begins to open up: the relative rigidity of adaptive responses, the species-species conformity to environments, gives way increasingly to choice responses, to modifiability and plasticity of behavior, and to an increasing trend toward learning through individual experience.” And further: “This human process of becoming familiar with one’s milieu is not simply a mirroring of an external, prefabricated “reality”, but it involves a formation of the world of objects by the human being in terms of his equipment and biopsychological “goals”. The human world, then, cannot claim to effect an independent “reality per se”; it is rather a coherent, man-specific Umwelt, a representation of “what there is” by means available to the human being (Werner & Kaplan, 1963: 12-13).

What really counts in this approach is the “organism-environment interaction” and the role of the “subject” in this interaction. Four major claims can be considered in this process: (i) the speed of processing, (ii) the role of conservative behaviour, (iii) the role of simulation and symbolic play and (iv) the role of sense-making.

As to the speed of processing, there is a whole body of research which is related to the ecological claims of direct perception (Gibson, 1966, 1979, 1982—see below —, Michaels & Carello, 1981), with as basic claim that perception is possible without the mind intervening in this process. It involves presentational immediacy and direct reactivity to the solicitations of the environment, stressing the role of information “pick-up” rather than information “processing”. Such “direct” perception can have adaptive value in case of threatening situations which require quick responses, but the speed of processing is at the cost of the richness of the sensory experience.

This is not the case in conservative behaviour in general, which tries to keep step with the unfolding of the perceptual flux. Proceeding in real-time, it tries to achieve or to maintain a state of equilibrium by “measuring” and “controlling” all possible perturbations. The role of sensory-motor integration is obvious here: it allows us to think of the music user in functional terms as a controller or measuring device and it is most typically exemplified in music performance. Performing, in fact, is a skilled activity that requires the simultaneous integration of multimodal sensory and motor information with multimodal sensory feedback mechanisms to monitor performance (Gaser & Schlaug, 2003).

There is, thirdly, a new paradigm in neuroscience that transcends the conception of the brain as reactive machinery—with automatic responses through co-ordinated mobilisation of preadapted sensorimotor tools—in favour of a conception of the brain as a simulator (Berthoz, 1997). The critical factors are the introduction of mental operations that are interposed between the perception of the stimulus and the triggering of the action (Paillard, 1994b, Reybrouck, 2001b) and the construction of an internal model of the world (Klaus, 1972). It allows the organism to rely heavily on imagery and representation and to deal with the environment at the level of modelling and symbolic play (Reybrouck, 2002, 2006a). The music user, on this view, can be considered as a homo ludens or playing automaton, which is able toperform internal dialogues and to carry out symbolic computations on the mental replicas of the sounding music. In order to do this, however, the “player” must have at his/her disposal a symbolic repertoire for doing the mental arithmetic that is typical of symbolic behaviour in general.

This brings us, finally, to the process of sense-making and the possibility of conceiving of musical stimuli as “signs” rather than as causal stimuli (Wallin, 1991: 231). It allows the music user to perform a way of thinking which is playful, and which can defined in terms of paratelic thinking—a way of “playing around” with ideas, inferences and presuppositions for their own sake and following them wherever they lead, without worrying about their serious implications or practical applications—rather than being goal-directed as in the “telic” mode of cognition (Apter,1984: 424).

7 The music user as an open system:
sense-making and the role of adaptation

Music cognition is not reducible to “naive realism”. It has the mark of our cognising with our minds.As Wallin puts it:

The stimulus as a releaser of a purposive behavior ... is weighed by the organism not only by means of auditory capacity in analyzing an acoustical event, but through a simultaneous evaluation of the event’s significance as well (1991: 231).

This weighing, however, is not arbitrary: it depends upon “innate” and “acquired” mechanisms of sense-making, which, in the case of music, are complemented by continuous interactions with the sounds.

These interactions, especially, are able to change our “semantic”—semantics, in the main, is the study of meaning—relations with the sonic world: they afford meaning to the music user as the outcome of previous perceptual and motor interactions with the sonic environment. As such they are helpful in building up a sonic Umwelt through the mechanisms of functional relations and functional tones (see above).

Musical semantics, however, is a rather tedious topic. There is, firstly, a distinction between self-referential semantics—which conceives of music as referring only to itself—and real semantics—with the music referring to things and events that are outside the music in the “real” world. The distinction, however, is somewhat arbitrary, as music users do not deal with artificial distinctions but with sounds as “signs” that shape their interaction with the sonic world. Music users, on this view, behave as open systems that construct their knowledge as the outcome of interactions with their environment, and which are able to “adapt” themselves to this environment. As such, they are able to build up a perceptual, behavioural and sign repertoire which allow them to cope with the music as environment.

This is an important claim. It emphasises the “construction” of knowledge and relies on the concept of “open system”, as defined already in von Bertalanffy’s theoretical writings:

Every living organism is essentially an open system. It maintains itself in a continuous inflow and outflow, a building up and breaking down of components, never being, so long as it is alive, in a state of chemical and thermodynamic equilibrium but maintained in a so-called steady state which is distinct from the latter. This is the very essence of that fundamental phenomenon of life which is called metabolism, the chemical processes within living cells...(von Bertalanffy, 1968: 39).

Adaptation is fundamental to such a living system or organism. It provides the necessary means for coping with its environment. This latter term, which is used mostly with regard to its physical universe—the physical environment, the prey to catch, the predators to avoid—can, by extension, be used also with regard to its “symbolic” or “cultural universe”. A major question, however, is the tension between wired-in coping mechanisms and those that are the outcome of a learning process. This is, in fact, the “nature/nurture dichotomy”, or the distinction between nativism and empiricism with its corresponding epistemological positions which claim that knowledge is dependent upon innate faculties (the Chomskyan position) as against the construction of knowledge as the result of interaction with the environment (the Piagetian position) (see Hargreaves, 1986, Reybrouck, 1997, 1989, 2005a). This debate, which has coloured decades of discussions about musical competence, is far from being closed, yet, there is a lot to be expected from the neurobiological approach to music research which is providing empirical evidence for theories and intuitions which were intuitive up to now (Peretz & Zatorre, 2003, Wallin, 1991, Wallin,Merker & Bown, 2000, Zatorre & Peretz, 2001).

Rather than joining this debate, I hold a position that conceives of the music user as an adaptive device (Reybrouck, 2005a, b) which is able to change its “semantic relations” to the sonic world. The model I propose is not really innovative: it is the translation of the concept of adaptive devices to the realm of music (Reybrouck, 2005a, b). It takes as a starting point the frequently denounced “robot model” of human behaviour which has been so dominant in psychological research. A leading concept in this context is the stimulus-response scheme, with behaviour being considered as a response to stimuli coming from outside. It is a mechanism which is partly based upon “inherited” neural mechanisms—as in reflexes and instinctive behaviour—but the more important parts are “acquired” or “conditioned” responses (von Bertalanffy, 1968: 189).

The inherited mechanisms are commonly known: most animals and men have neural coding strategies that are used in the representation and the processing of sensory information. To quote Cariani:

While the particular experiential textures of things, their qualia, undoubtedly vary among different vertebrates, the basic body-plans, sensory organs, and neural representations are roughly similar. We see in different colors, hear in different frequency registers, and smell different odors, but the basic relational organizations of our percept-spaces in the end may not be so radically different (Cariani 1998b: 252).

Human beings, in fact, are biological adaptive systems with similar basic life-imperatives as breathing, eating, drinking, sleeping and mating, and higher-level processes of cognitive functioning. As such, it is possible to conceive of “universals of knowing and perception”, and the same holds true for “musical universals” (Brunner, 1998, Kon, 1998,Marconi, 1998, Miereanu & Hascher, 1998, Normet, 1998, Padilla, 1998) which are subclasses of more encompassing levels of cognitive functioning.

According to Murdock, there are three categories of universals, which correspond to (i) primary, genetically coded (instinctive) impulsions, (ii) acquired habits, rooted in fundamental bio-psychological demands and (iii) cultural habits with only very thin links to the conditions of the secondary level (Murdock, 1945). A somewhat analogous distinction has been drawn by Bystrina (1983) who distinguishes between primary, secondary and tertiary codes. Primary codes are of an innate nature (genetic code, perception code and intraorganismic code), secondary codes are the result of a learning process (language code), and tertiary codes operate at the level above the secondary code (cultural codes) (see also Jiranek, 1998). It is possible, however, to transcend the lower levels and to go beyond the mere causal relationships. As Buck puts it:

we enter the craddle as a mixture of inherited tendencies for which we are not in the least responsable, and all our reactions will be in accordance with them unless in later life we learn to modify them through training and education.” This training and education, in the end, is nothing more than “one prolonged endeavour to substitute acquired reactions for native ones (1944/1961: 12).

The distinction between primary, secondary and tertiary codes is very viable. It allows us to rely on primary codes, which are, in a sense, our perceptual primitives. Besides, there is the possibility of knowledge acquisition and learnability, not only at the perceptual, but also at the behavioural level and the level of the mental computations. This means, for short, that the music user behaves as an “adaptive system” with the possibility of adaptation at each level of the control system.

The concept of adaptation, further, is a biological concept: it allows an organism to change itself in order to survive in its environment (Fleagle, 1999). The claim, however, can be translated to the realms of cognition, as advocated already in the seminal contributions ofPiaget (1936, 1937, 1967, and von Glasersfeld, 1978, 1982 for a critical review). Central in this theory are the related principles of “assimilation” and “accommodation” which have furthered the constructivist approach to knowledge acquisition in general (see von Glasersfeld, 1995a, b).

The mind primarily “assimilates”, if it perceives and categorises experience in terms of what it already knows:

... no behaviour, even if it is new to the individual, constitutes an absolute beginning. It is always grafted onto previous schemes and therefore amounts to already constructed structures (innate, as reflexes are, or previously acquired (Piaget, 1976: 17).

Cognitive “assimilation” thus comes about when a cognising organism fits an experience into a conceptual structure it already has. If the result of this process creates a kind of perturbation, then a revision is set up which may lead to accommodation with a corresponding change in existing structures or to the formation of new ones. The principle of accommodation thus provides a mechanism for learning which allows the learner to modify his/her semantic relations with the world. A basic claim in this approach is theepistemological assumption that all information which is acquired from the outer world is matched continuously again internal schemes (Piaget, 1967: 294). If there is no match, the system must accommodate in order to adapt itself to the environment.

Knowledge construction, on this view, has “adaptive value”. It relies on the mechanisms of assimilation and accommodation, which are related to thecentral processing of the control system. Adaptation, however, is not restricted to the mental level of processing: it can be expanded to each level of the control system—perceptual, effector and computational—and can be described in the operational terminology ofadaptive systems or adaptive devices (see Cariani,1991, 1998a, Pattee, 1982, 1985 and Rosen, 1978). Such devices are able to alter their basic functions of sensing, c­o-ordinating and acting with as final result a change in their semantic relations with the world. According to Cariani (1989, 1991, 2001, 2003) there are three possibilities for doing this: (i) to amplify the possibilities of participatory observation by expanding the perceptual and behavioural repertoire, (ii) to adaptively construct sensory and effector tools and (iii) to change the cognitive tools as well.

As to the sensing function, it is possible to modify or augment the sensors, allowing the device to choose its own perceptual categories and to control the types of empirical information it can access. Several strategies are possible here (Cariani 1991, 1998b), but the basic mechanisms are reducible to only two or them: to alter the existing sensing functions or to add new ones. Accordingly, there are four possibilities: (i) prosthesis or adaptive fabrication of new front-ends for existing sensors, (ii) active sensing or using motor actions to alter what is sensed through interaction (poking, pushing, bending), (iii) sensory evolution or adaptive construction of entirely new sensors and (iv) internalised sensing by creating internal, analog representations of the world out of which internal sensors extract newly-relevant properties (1998b: 718).

As to the effector function, there is the possibility of “active measurement” as a process of acting on the world and sensing how this world behaves as a result of actions we perform on it. This is “active sensing” which changes our sensing function without altering the sensor structures, requiring only additional co-ordinative and motor resources to be used.

The co-ordinating function, finally, is related to the central processing of the control system. It enables the construction of better cognitive tools for dealing with the environment and allows the device to handle information processing in terms of a “transfer function” from sensorium to motorium, leaning heavily on innate and wired-in mechanisms of information processing.

“Music users”, also, can be considered as adaptive devices (Reybrouck 2006b): they can change their semantic relations with the world in arbitrarily choosing what kinds of “distinctions” are to be made—perceptual categories, features and primitives—(Reybrouck 2004a), what kinds of “actions” are to be done on the environment—primitive action categories—and what kinds of “co-ordinative mappings” are to be carried out between perception and action. This is, in fact, a whole research program, but the claims fit in very easily with the concept of adaptive devices. To quote Cariani again:

... adaptive systems ... continually modify their internal structure in response to experience. To the extent that an adaptive epistemic system constructs itself and determines the nature of its own informational transactions with its environs, that system achieves a degree of epistemic autonomy relative to its surrounds (Cariani 2001: 60).

Figure 7. Diagram of a perception-cognition-action loop (after Cariani, 1989). The sense organs provide the input (sensorium), the effector organs the output (motorium) and the central processing the co-ordinations between them.

What matters, on this view, is an operational approach to the adaptive control of percept-action loops in artificial devices (Cariani 1989, 2001, 2003, Ziemke & Sharkey, 2001). It is an approach that fits in with the ‘‘ecological conception’’ of interaction with the environment and that presupposes an ‘‘epistemic cut’’ between the organism and its environment. It further draws an operational distinction between the “input” (sensorium), “output” (motorium) and “central processing” as a kind of sensory–motor integration which does the mapping and the co-ordination between sensorium and motorium (Cariani 2001, 2003) (see figure 7).

8 The claims of ecosemiotics and ecological acoustics

The way the music user makes sense out of the perceptual flux is not gratuitous. It is “ecologically” constrained in the sense that it is related to the way how organisms interact with their environment. This ecological claim goes back to the original definition of Haeckel who conceived of ecology as “the science of the relations between the organisms and the environmental outer world” (1988 [1866]: 286). The related field of “ecosemiotics” can be defined accordingly as the study of the semiotic interrelations between an organism and its environment (for a critical definition, see Kull 1998). It means that a full description of perception cannot be given by analysing either the organism or its environment—organism-environment dualism—, but that we need an approach which treats the “environment-as-perceived”. Perception, on this view, is not “organism-neutral”, hence the role of interaction and of mutualism of organism and environment (Ingold, 1992).

Music perception, in turn, can be conceived in terms of organism-environment interaction—with the music user as an organism and the music as environment (Reybrouck, 2006b)—and the related notion of “coping” with the world. Ecological perception, in fact, studies the human cognitive and perceptual apparatus in the service of survival and orientation in the environment (Shepard, 1984). As such, it is related to adaptive behaviour, which fits in with the claims of biosemiotics as an area of knowledge which describes the biological bases of the interaction between an organism and its environment (Hoffmeyer 1997a, b, 1998, Sebeok, 1998, Sebeok & Umiker-Sebeok, 1992). Music, on this view, can be considered as a challenging “environment” and the music user as an organism that must adapt itself in order to cope with this environment. To quote Ingold: “the environment sets the problem, in the form of a challenge; the organisms embodies the solution, in the form of its adaptive response” (Ingold, 1992: 40-41).

There is, however, no tradition of thinking of music in ecological terms (see Gaver 1993a, b, Reybrouck, 2005a, Windsor, 2004, see also Godøy, 1999, Martindale & Moore, 1989, McAdams, 1993; Neisser, 1987). The broader field of ecological perception in general, on the other hand, is an established research domain with considerable theoretical and empirical grounding. Its major focus is on “visual” perception but some of its basic claims are likely to have relevance for the field of audition as well. A major problem, however, is the time-consuming character of music as a temporal art: it challenges the claims of direct perception (see below) in calling forth mechanisms of cognitive processing and conceptual construction of time (Reybrouck 2001b, 2004b). It is interesting, therefore, to explore some basic contributions from the ecological approach which may be translated to the realm of music. I see four major topics: (i) James Gibson’s contributions of direct perception—together with his conception of the senses as perceptual systems and his concept of affordances—, (ii) the principle of reality and cognitive economy, (iii) event perception, and (iv) ecological acoustics.

The concept of ecological perception goes back to Gibson (1966, 1979, 1982). He has provided a wealth of conceptual tools which are valuable instruments for giving an operational description of the process of perception. His key concept of direct perception, e.g. (see also Michaels & Carello, 1981), conceives of perception as occurring immediately without the mind intervening in this process. It involves direct contact with the sensory stimuli, which elicit reactions in a kind of lock-and-key approach. It means further that information is processed in an “all-or-none way” as a “discrete” reaction to stimuli which are continuous. The advantages are obvious: there is the speed of processing and the adaptive value for surviving in case of threatening situations.

Direct perception, however, is a somewhat ill-defined category (Reybrouck, 2005a). It calls forth direct reactivity to the environment, but it is dependenton processes of learning and development as well. A central notion in this approach is Gibson’s concept of information pickup: perceivers ‘‘search out’’ information which then becomes ‘‘obtained’’ information. They pick up information which is already part of the environment and which affords perceptual significance for the organism. In order to do so, they must not lean on ‘‘senses’’—which simply function to arouse sensations—, but on “perceptual systems” which are tuned to the information that is considered to be useful. Perception, on this view, is not to be explained in “mechanistic” terms of stimuli and reactions, but in terms of active systems that search for information. As Gibson puts it:

We shall have to conceive the external senses in a new way, as active rather than passive, as systems rather than channels, and as interrelated rather than mutually exclusive. If they function to pick up information, not simply to arouse sensations, this function should be denoted by a different term. They will be called perceptual systems (Gibson,1966: 47).

There are basically five of them: (i) the orientation system, which is the basic system from which the other systems take their starting point, (ii) the auditive system, (iii) the haptic system, (iv) the smell and taste system and (v) the visual system. All of them are oriented to the direction of attention through adjustments of the bodily posture and exploratory movements. As such, it is possible to conceive of the “eye-head system”, the “ear-head system”, the “hand-body system”, the “nose-head system” and the “mouth-head system”, with the orientation of the head and body being presupposed as a default condition, and the movements of eyes, ears, hands and mouth being in addition. Stimuli, on this view, are not merely stimuli, but stimuli to be “searched for”. It means that perceptual systems are more encompassing systems than mere sensory receptors for exploring the environment: they involve the whole gamut of exploratory movements (sniffing, sensing, groping, directing the head to the stimuli, fixing of the gaze...) which facilitate the process.

What matters in this approach, is a conception of perception as an active process of information pickup. Rather than being the passive registration of sensory stimuli coming from the outer world, it aims at the acquisition of knowledge (Michaels & Carello, 1981). Hence the role of key concepts as attunement, reciprocity and resonance and the corresponding perceptual processes of detection, discrimination, recognition and identification.

The perceivers, on this view, must attune themselves to the information which is available in order to pick it up through mechanisms of attentional strategies. To the extent that this is done more easily and more efficiently, it is possible to consider also the role of perceptual learning (Gibson,1966). The perceiver, then, relies on mechanisms of “information pickup” and “information extraction” by searching actively for useful information.

This brings us to the concept of affordances, which, according to Gibson, are environmental supports for an organism’s intentional activities. Animals—and by extension also human beings—are sensitive to the functional characteristics of their environment. They perceive environmental objects in terms of what they “afford” for the consummation of behaviour rather than in terms of their objective perceptual qualities (Gibson, 1979: 127, see also Gibson, 1966, 1982).

These affordances are “subjective” qualities that render the environment apt for specific activities—supporting locomotion, concealment, manipulation, nutrition and social interaction for the animal—but they are “real” and “objective” as well:

An important fact about the affordances of the environment is that they are in a sense objective, real, and physical, unlike values and meanings, which are often supposed to be subjective, phenomenal, and mental. But, actually, an affordance is neither an objective property nor a subjective property: or it is both if you like. An affordance cuts across the dichotomy of subjective-objective and helps us to understand its inadequacy. It is equally a fact of the environment and a fact of behavior. It is both physical and psychical, yet neither. An affordance points both ways, to the environment and to the observer (Gibson,1979: 129).

It is not difficult to apply this to the realm of music: sounds, e.g., can be objectified as things which can stand off against us, resist us or even harm us. Music users, then, should try to understand music in terms of what it affords to them rather than relying merely on its acoustical qualities. The question, however, is what these “musical affordances” are?

Several options are possible here. One of them is to rely onperceptual-motor interactionswith the sonic environment in an attempt to broaden the cognitive structures from the rather limited linguistic categories to those categories that are the outcome of perceptual-motor interactions with the environment (Mazet, 1991: 92). What matters here is a broadening from mere “perceptual” to “functional categories” of cognition, integrating both perceptual attributes and classes of action (Mazet, 1991: 100). The case of a driver who crosses a rural landscape, a village or a city is illustrative at this point: in trying to describe his crossing, there is a difference between a description in terms of “nouns” or “adjectives”—referring merely to perceptions of the environment (valley, meadow, forest or village elements as church, market, shop)—as against a description in terms of “action verbs”—referring to speedways and busy traffic.

The same holds true for music, which we can compare to a kind of sonic landscapewith places to be labelled in perceptual or actions terms. As to the “perceptual aspect”, many fragments of music can be considered as “sonic biotopes” which resemble the natural biotopes of an interesting landscape. Inspection of the titles of many movements of classical romantic music is revealing at this point. But it possible also to describe things in terms of their activity signature (Beck, 1987). The word “chair”, e.g., means something to sit down or get up. This motor element in applying descriptive categories was already advocated in earlier theories of categorisation (Rosch & Lloyd, 1978), but the translation to the domain of music mostly still has to be done (see Delalande, 1984, Godøy, 1997b, Lidov, 1987, Reybrouck 2001b).

There are, as yet, many possibilities that stress the “action aspect” of dealing with music. I mention five of them: (i) the sound producing actions proper, (ii) the effects of these actions, (iii) the possibility of imagining the sonorous unfolding as a kind of movement through time, (iv) the mental simulation of this movement in terms of bodily based image schemata and (v) the movements which can be possibly induced by the sounds (seeGodøy, 1997b, 2001, Johnson & Larson, 2003, Reybrouck, 2001b).

“Sound producing actions”, obviously, are primary examples to function as musical affordances. According to Godøy (1997b, 1999), there are singular “sound-producing actions”—like hitting, stroking, kicking, blowing etc.—as well as more complex or compound actions—such as drumming a rhythmic pattern or sliding up and down a melodic contour. But even the metaphors used in talking about music refer to sound-producing actions (slow, fast, up and down ...) and the same applies to many musical terms like martellato, leggiero, tenuto, and legato.

The “effects” of sound producing actions provide another approach to musical affordances. Sounds, in fact, mostly are effects of natural forces. They convey meaning which invites the listener to “react” to them or even to “cope” with them. As Dewey puts it:

[Sounds are] effects of the clash, the impact and resistance, of the forces of nature. They express these forces in terms of what they do to one another when they meet; the way they change one another, and change the things that are the theater of their endless conflicts. The lapping of water, the murmur of brooks, the rushing and whistling of wind, the creaking of doors, the thud of fallen objects, the sobs of depression and the shouts of victory—what are these, together with all noises and sounds, but immediate manifestation of changes brought about by the struggle of forces? Every stir of nature is effected by means of vibrations, but an even uninterrupted vibration makes no sound; there must be interruption, impact, and resistance.“ And further: “Sound stimulates directly to immediate exchange because it reports a change. A foot-fall, the breaking of a twig, the rustling of underbrush may signify attack or even death from hostile animal or man. Its import is measured by the care animal and savage take to make no noise as they move. Sound is the conveyer of what is likely to happen. It is fraught much more than vision with the sense of issue (Dewey, 1934/1958: 236 & 238).

There is, however, a distinction between the effects of self-produced sounds and sounds which come from the outer environment, but both can be evaluated as to what they afford to human beings.

The possibility to imagine the sonorous unfolding as movement through time, thirdly, is another way of dealing with music. It leans upon “body-based image schemata”—as advocated in cognitive linguistics (see above)—which claim that conceptual structures are meaningful because they are tied to preconceptual bodily experiences. They use the body as a reference for interactions with and making sense out of the outer world and are basically “egocentric” in describing subjective experiences in terms of bodily metaphors. As such, it is possible to conceive of music as movement over time, and this movement, in turn, can be imagined as movement of our body. Listening, then, involves a kind of motor imagery that projects our bodily movements on the music.

This bodily projection, further, is rather complex, as the music can be conceived as the mover, but the listener can move as well (see above). As such, it is in a way related to the distinction between the objective-exosomatic and subjective-endosomatic realm of cognition (Lidov, 1987) and the distinction between the observer and the observed thing. What I argue for, therefore, is a kind of phenomenal experience which involves the experience of movement but without the action being actual or manifest. It corresponds to the so called internal imagery—or first person perspective—which enables the transition from overt action to internalised forms of action. The whole process calls forth a kind of motor empathy and ideomotor simulation, allowing the listener to experience the music as something that moves over time, while simultaneously experiencing this movement as movement of the own body (Reybrouck, 2001b).

A final interpretation of music in terms of affordances is the possibility to move as a reaction to the music. Movements, in fact, can be induced by sounds. The whole domain of dance music is illustrative at this point, but it is possible to move with other kinds of music as well. Music, then, is a stimulus for movement and is perceived in terms of its motor induction capacities. The movements can be specific and articulate, but they can relate also to more general levels of motor induction, as illustrated in the energetic approaches to music psychology, advanced, e.g., by Kurth (1931) and Mersmann (1926). This level of ideomotor simulation can be basically conceived of as “forces” and “energies” that are inherent in musical structures which, in turn, account for our perception and imagination of “tension”, “resolution” and “movement”.

9 Categories and categorisation:
the principle of reality and cognitive economy

Dealing with music in ecological terms is a challenging approach (Gaver, 1993a, b,Reybrouck 2005a,Windsor, 1995, 2004). It stresses the adaptive value of coping with the sounds in terms of interaction between listener and environment. There is, in fact, no causal relation between the objective qualities of the sounding music and their perception by the listener. Most listeners, in fact do not rely on “auditory listening” which focuses on the acoustical characteristics and musical dimensions of the music. They listen, on the contrary, in ecological terms and in terms of global extramusical categories (Reybrouck, Verschaffel & Lauwerier, submitted).

There is no unambiguous explanation for this phenomenon. Yet, there is an ecological assumption which states that common strategies of sense-making rely on cognition as a schematising process that ecologises the stuff of the world, either to render it more assailable by the organisms or to accommodate the organism to its environment (Shaw & Hazelett, 1986). As such, it is related to the principles of categorisation with their basic principles of cognitive economy and the principle of reality.

Categorisation, in the main, is a cognitive activity that stresses the importance of providing the maximum of information with the least cognitive effort. This is “cognitive economy”: it allows the perceiver “to render discriminably different things equivalent, to group objects and events and people around us into classes, and to respond to them in terms of their class membership rather than their uniqueness”(Bruner, Goodnow & Austin, 1956). It means also that genuinely diverse inputs lead to one single output, without preserving the shape, size, position and other formal characteristics of the stimulus (Neisser, 1967, 1987). As such we use categorisation as a tool for managing complex environments: it is fundamental to any sort of discrimination task and is indispensable in using previous experiences to guide the interpretation of new ones.

Categorisation, further, mostly starts from the assumption of an implicit ontological realism—as advocated already in the early work of Rosch on categorisation (Rosch 1977, Rosch et al., 1976, Rosch & Lloyd,1978, see also Dubois, 1991)—, claiming that the perceived world is not unstructured, but consists of real and natural discontinuities and co-occurrent properties. It takes the categories in the external outer world for granted, as advocated in “objectivist cognition” or “objectivist semantics”. Categorisation, however, does not deal merely with “ontological categories” but with conceptual structures which contain constituents that are differentiated by major ontological category features such as thing, place, direction, action, event, manner and amount, smell and time (Jackendoff, 1988). As such, it brings together the claims of “objectivist“ and “conceptual” or “cognitive semantics” (Reybrouck, 2005a).

The principle of cognitive economy has many implications. An example is the difference between recognising a sounding object or sonic event as a discrete entity and the experience proper of its sonorous articulation over time. It brings us to the related distinction between categoricalvs. auditory perception. Categorical perception is economical: it assigns one discrete meaning to an event that is evolving over time. Auditory perception, on the contrary, relies on auditory or acoustical listening which provides a phenomenological description of the sound in terms of its acoustic qualities. Purely auditory or acoustical listening is quite improbable (Handel, 1989): observers do not perceive the acoustical environment in terms of their acoustic qualities but rather in terms of ecological “events” (Balzano, 1986, Lombardo, 1987). What matters, then, is not really the continuous flow of matter in the physical world—music in its acoustic qualities—, but “music-as-heard” and the way the music user can make sense of it.

There are two possibilities for doing so. The first is the reliance on auditory images (McAdams, 1984) as psychological representations of sound entities which exhibit a coherence in their acoustic behaviour. The second is related to event perception which allows the listener to recover invariant patterns over time. Events, in a more operational definition, can be considered as higher-order variables with time-varying complex acoustic properties which can be described in terms of their invariants.

The concept of “event” is an interesting conceptual tool: it brings together perceptual and conceptual knowledge, as events are continuous in their unfolding but discrete in their labelling. They involve, further, “changes in objects or collections of objects” (Michaels and Carello, 1981), allowing the extraction of invariants which can be structuralor transformational(Bartlett, 1984, Michaels & Carello, 1981, Shaw, Flasher & Mace, 1996): structural invariants refer to features that are not—or only slowly—changing, while transformational invariants refer to styles of change (Shaw & Pittenger, 1987).

The perception of events, on this view, can be defined in intuitive terms as something happening to something, with the “something happening” being specified by “transformational” and the “something” to which something is happening by “structural” invariants (Michaels and Carello, 1981: 26). Transformational invariants specify the change that is occurring in or to the object, structural invariants describe the object by itself. Recognition of the sound of a clarinet, for example, is a structural invariant, the specific articulation of the sound is transformational.

The concepts of structural and transformational invariants and of events are tightly intertwined: the former act as a kind of “glue” that “unitises” sequences of stimulus information into coherent events (Bartlett, 1984). They allow us to describe events both at a glance and in their temporal unfolding, providing a propositional and a continuous description of invariant patterns over time. As such, it is possible to conceive of event description in a propositional way and to specify an event (E) perceptually when both the transformational (TI) and the structural invariant (SI) are available to be detected. An event, then, can be specified when the two-variable function E(TI, SI) can be evaluated (Shaw et al. 1996). To give an example: an event involving a bouncing ball might be denoted as E(TI = bouncing, SI = ball) = bouncing ball.

“Events”, further, behave as basic building blocks. They function as units in perception and memory, calling forth an “ecological approach” to memory phenomena, which is related to the concept ofschemata and the three core ideas of event perception: (i) the units of perception and memory are temporally extended “events”; (ii) the basis of perception and memory is the pick-up of invariants over time; and (iii) perception and memory are essentially veridical.

Event perception, thus, is related to principles of “cognitive economy”. It allows the music user to cope with the sonic world in a way that is less demanding as to processing efforts. There is, e.g, a difference between the recognition of basic level events such as impacts, scraping, and dripping, as well as more complex events such as bouncing, breaking, spilling, and machinery as against the acoustical description of these sounds. The same holds true for complex acoustic events such as e.g. the sound of a propeller airplane (the event) as against a technical description of the same event as “a quasi-harmonic tone lasting approximately three seconds with smooth variations in the fundamental frequency and the overall amplitude” (Gaver 1993a). At this level of description there is a distinction between what we actually hear and what we think we hear—our interpretation of what we hear. There is, in fact, a difference between the experience of hearing events in the world—to hear a sound in terms of its source—and hearing these sounds for themselves. According to Gaver (1993) this is the distinction between everyday listening—which is reducible to the perception of sound-producing events—and musical listening which reduces to the experience of soundsin terms of their sensory and acoustic qualities.

Music theory has hardly ever addressed the topic of everyday listening. The claims, however, are challenging in the sense that may provide means for a better understanding of the process of listening. To quote Gaver:

Studies of everyday listening may serve as the foundation of a new framework for understanding sound and hearing... What should an account of everyday listening be like? It must answer two fundamental questions. The first question is “What do we hear?”. In seeking to expand accounts of sound and hearing beyond the traditional preoccupation with sensations such as pitch and loudness, we must develop a framework for describing ecologically relevant perceptual entities: the dimensions and features of events that we can actually hear. The second question is “How do we hear?” As we expand our account to include dimensions and features of events, we must also develop an ecological acoustics, one which describes the acoustic properties of sounds that convey information about the events we hear. The first question deals with the content of everyday listening, while the second deals with he acoustic structures that allow that content to be heard. By exploring both questions together, then, we can start to build a psychophysics of auditory event perception, in which the relations among perceptual and acoustic dimensions of events are examined (Gaver, 1993a: 287)

10 Conclusions and perspectives

In this paper I have focussed on the process of dealing with music, stressing the role of the musical experience proper. In order to do so, I have tried to give a rather extensive overview of both theoretical grounding and empirical evidence, relying heavily on some seminal works of Dewey, James and von Uexküll as well as on the findings of current neurobiological research.

A major claim of my contribution is the definition of dealing with music as a musical behaviour that is in a way reducible to the basic mechanisms of sensorimotor interactions with the sonic world. These interactions can be actual and manifest—as in “conservative behaviour”—but they can be carried out also at the level of imagery and representation, stressing the role of ideomotor simulation and symbolic play.

Another claim is the aim to provide an operational description of the process of dealing with music. Besides the observable and manifest behaviour I have argued for the introduction of both subjectivity and intentionality in my approach, in order to provide a viable framework for describing the process of sense-making out of the perceptual flux.

An additional claim, finally, is my intention to go beyond a merely descriptive approach to the process of dealing with music, and to argue for the desirability of changing the kinds of common interactions with the sounds. In order to do so, I have introduced the concept of music users as adaptive devices which are able to change their semantic relations with the sonic world. As such, it is possible to conceive of an adaptive model of sense-making with the possibility to change all kinds of interactions, both at the sensory and effector interfaces and at the level of the mental computations. It allows us, finally, to apply these theoretical claims to the whole field of music education and music learning in general.

References

• Annett, J. 1996. “On knowing how to do things: a theory of motor imagery”. Cognitive Brain Research, 3, pp. 65-69.
• Apter, M. 1984. “Reversal theory, cognitive synergy and the arts”. In: Crozier, W. R. & A.J.Chapman (Eds.). Cognitive Processes in the Cognition of Art. North-Holland: Elsevier Science Publishers, pp. 411- 426.
• Arbib, M. 1981. “Perceptual structures and distributed motor control”. In: Brooks, V. (Ed.) Handbook of Physiology. Vol. 2.Motor Control. Bethesda: American Physiological Society, pp. 1449-1480.
• Balzano, G. 1986. “Music Perception as Detection of Pitch-Time Constraints”. In: McCabe, V. & G.Balzano. Event Cognition: An Ecological Perspective. Hillsdale, NJ - London: Lawrence Erlbaum, pp. 217-233.
• Barsalou, L., W.Barbara, K.Olseth, K.Mix & L-L.Wu. 1993. “Concepts and Meaning”. In: Beals, K., G.Cooke, D.Kathman, K.McCullough, S.Kita& D.testen (Eds.). Chicago Linguistics Society 29: Papers from the parasession on conceptual representations. University of Chicago: Chicago Linguistics Society, pp. 23-61.
• Barsalou, L. 1999. “Perceptual Symbols Systems”. Behaviorial & Brain Sciences, 22, pp. 577-660.
• Bartlett, J. 1984. “Cognition of complex events: visual scenes and music”. In: Crozier, W. & A.Chapman (Eds.). Cognitive Processes in the Perception of Art. Amsterdam - New York - Oxford: North-Holland, pp. 225-251.
• Bateson, G. 1972. Steps to an Ecology of Mind. New York: Ballantine.
• Beck, B. 1987. “Metaphors, Cognition, and Artificial Intelligence”. In: Haskell, R. (Ed.). Cognition and Symbolic Structures: The Psychology of Metaphoric Transformation.Norwood: Ablex Publishing Corporation, pp. 9-30.
• Bel, B. & B.Vecchione. 1993. “Computational Musicology”. Computers and the Humanities, 27, p.1.
• Berthoz, A. 1997. Le sens du mouvement. Paris: Odile Jacob.
• Blacking, J. 1995. Music, Culture and Experience. London: University of Chicago Press.
• Boring, E., H.Langfeld & H.Weld. 1948. Foundations of Psychology. New York & London: Wiley & Chapman.
• Bowman, W. 2000. “A somatic “Here and Now” semantic: music, body, and self”. Bulletin of the Council for Research in Music Education, 144, pp. 45-60.
• Bregman, A. 1990. Auditory Scene Analysis: The Perceptual Organization ofSound, CambridgeMA: The MIT Press.
• Brier, S. 1999a. “What is a Possible Ontological and Epistemological Framework for a True Universal ‘Information Science’? The Suggestion of a Cybersemiotics”. In: W. Hofkirchner (Ed.). The Quest for a Unified Theory of Information. Proceedings of the Second International Conference on the Foundations of Information Science. Amsterdam: Gordon and Breach Publishers, pp. 79–99.
• Brier, S. 1999b. “Biosemiotics and the Foundations of Cybersemiotics”. Semiotica, 127 (1/4), pp. 169-198.
• Brier, S. 2000a. “Biosemiotics as a Possible Bridge Between Embodiment in Cognitive Semantics and the Motivation Concept of Animal Cognition in Ethology”. Cybernetics & Human Knowing, 7(1), pp. 57–75.
• Brier, S. 2000b. “Trans-Scientific Frameworks of Knowing: Complementarity Views of the Different Types of Human Kowledge”. Systems Research and BehavioralScience, 17, pp. 433–458.
• Brower, C. 2000. “A Cognitive Theory of Musical Meaning”. Journal of Music Theory, 44(2), pp. 323-379.
• Bruner, J., Goodnow, J. & Austin, G. 1956. A study of thinking. New York: Wiley.
• Brunner, R. 1998. “Univers de genèse, univers de réception“. In: Miereanu, C. and X. Hascher (Eds.). Les Universaux en musique, Actes du 4e Congrès international sur la signification musicale.Paris: Publications de la Sorbonne, pp. 637–648.
• Buck, P. 1944/1961. Psychology for Musicians.London - New York – Toronto: OxfordUniversity Press.
• Bystrina, I. 1983. ”Kodes und Kodewandel”. Zeitschrift für Semiotik, 5, pp. 1-22.
• Cano, R.L. 2003. Setting the body in music. Gesture, Schemata and Stylistic-Cognitive Types. Lecture delivered at “International Conference on Music and Gesture, East Anglia, August 2003”.
• Cariani, P. 1989. On the design of devices with emergent semantic functions. Unpublished Doctoral Dissertation: StateUniversity of New York.
• Cariani,P. 1991. “Some epistemological implications of devices which construct their own sensors and effectors”. In: Varela, F. & P.Bourgine (Eds.). Towards a Practice of Autonomous Systems, Proceedings of the First European Workshop on Artificial Life. Cambridge, MA: MIT Press, pp. 484-493.
• Cariani, P. 1998a. “Epistemic Autonomy through Adaptive Sensing”. In: Proceedings of the 1998IEEE ISIC/CIRA/ISAS Joint Conference, Gathersburg, MD. September 12-17, 1998 pp. 718-723.
• Cariani, P. 1998b. “Life’s journey through the semiosphere”. Semiotica, 120, 3/4, pp. 243-257.
• Cariani, P. 2001. “Symbols and dynamics in the brain”. Biosystems, 60, 1-3, pp. 59-83. Special issue on “Physics and evolution of symbols and codes”.
• Cariani, P. (2003). “Cybernetic systems and the semiotics of translation”. In S.Petrilli (Ed.), Translation Translation. Amsterdam: Rodopi, pp. 349 – 367.
• Chion, M. 1983. Guide des objets sonores. Paris: Editions Buchet/Chastel.
• Cox, A. 2001. “The Mimetic Hypothesis and Embodied Musical Meaning.Musicae Scientiae, 5/2, pp. 195-209.
• Crammond, D. 1997, “Motor imagery: never in your wildest dream”. Trends in Neurosciences, pp. 54-57.
• Cusick, S. 1994. “Feminist Theory, Music Theory, and the Mind/Body Problem”. Perspectives of New Music, 32 (1), pp. 8-27.
• Decety, J. 1996. “Do imagined actions share the same neural substrate?” Cognitive Brain Research, 3, pp. 87-93.
• Deecke, L. 1996. “Planning, preparation, execution, and imagery of volitional action”. Cognitive Brain Research, 3, pp. 59-64.
• Delalande, F. 1984. La musique est un jeu d’enfant. Paris: Buchet/Chastel.
• Deliège, I. & J.Sloboda (Eds.). 1997. Perception and Cognition of Music.Hove: Psychology Press (Taylor & Francis).
• Deutsch, D. (Ed.). 1982. The Psychology of Music. New York - London - Paris - San Diego - San Francisco - Sao Paulo - Sydney - Tokyo - Toronto: Academic Press.
• Dewey, J. 1925. Experience and Nature. Chicago - London: Open Court Publishing Company.
• Dewey, J. 1934/1958. Art as Experience. New York: Capricorn Books.
• Dubois, D. 1991. “Catégorisation et cognition: “10 ans après”, une évaluation des concepts de Rosch”. In: Dubois, D. (Ed.). Sémantique et cognition. Catégories, prototypes, typicalité. Paris: Editions du CNRS, pp. 34-54.
• Edelman, G. 1989. The remembered present. A biological theory of consciousness. New York: Basic Books.
• Edelman, G. 1992. Bright air, brilliant fire: On the matter of mind. New York: Basic Books.
• Edelman, G. 1995. The Wordless Metaphor: Visual Art and the Brain. In: Biennial Exhibition Catalogue. New York: The WhitneyMuseum of Art, pp. 33-47.
• Fauconnier, G. 1985. Mental Spaces: Aspects of Meaning Construction in Natural Language. Cambridge: CambridgeUniversity Press.
• Fillmore, C. 1984. Frames and the Semantics of Understanding. Unpublished manuscript. Berkeley: University of California, Department of Linguistics.
• Fleagle, J. 1999. Primate adaptation and Evolution. San Diego: Academic Press,
• Francès, R. 1958. La perception de la musique. Paris: Vrin. [Translated by W.J.Dowling as: The Perception of Music. Hilssdale: Erlbaum, 1988.]
• Gaser, Ch. & G.Schlaug. 2003. “Brain Structures Differ between Musicians and Non-Musicians”. The Journal of Neuroscience, 23 (27), pp. 9240-9245.
• Gaver, W.W. 1993a. “How do we hear in the world? Explorations of ecological acoustics”. Ecological Psychology, 5, 4, pp. 285-313.
• Gaver, W.W. 1993b. “What in the world do we hear? An ecological approach to auditory event perception”. Ecological Psychology, 5 (1), pp. 1-29.
• Gibbs, R. & H.Colston. 1995. “The Cognitive Psychological Reality of Image Schemas and Their Transformations”. Cognitive Linguistics, 6, 4, pp. 347-78.
• Gibson, J. 1966. The Senses Considered as Perceptual Systems. London, Allen & Unwin.
• Gibson, J. 1979. The Ecological Approach to Visual Perception.Boston: Houghton Mifflin.
• Gibson, J. 1982. Reasons for Realism: Selected Essays of James J.Gibson, E.Reed and R.Jones (Eds.). Hillsdale, NJ: Lawrence Erlbaum.
• Godøy, R. I. 1999. “Cross-Modality and Conceptual Shapes and Spaces in Music Theory”. In: Zannos, I. (Ed.). Music and Signs. Bratislava: ASCO Art & Science, pp. 85-98.
• Godøy, R. I. 1997a. Formalization and Epistemology. Oslo: ScandinavianUniversity Press.
• Godøy, R. I. 1997b. “Knowledge in Music Theory by Shapes of Musical Objects and Sound-Producing Actions”. In: Leman, M. Music, Gestalt and Computing. Studies in Cognitive and Systematic Musicology. Berlin - Heidelberg: Springer Verlag, pp. 89-102.
• Godøy, R.I. 1999. “Cross-modality and conceptual shapes in music theory”. In: Zannos, I.(Ed.), Music and Signs, Semiotic and Cognitive Studies in Music. Bratislava: ASCO Art & Science, pp. 85-98.
• Godøy, R.I. 2001. “Imagined Action, Excitation, and Resonance”. In: Godøy, R.I. & H.Jørgensen (Eds.). Musical Imagery. Lisse - Abingdon - Exton – Tokyo: Swets & Zeitlinger, pp. 237-250.
• Gromko, J. & A. Poorman. 1998. “Does perceptual-motor performance enhance perception of patterned art music?” Musicae Scientiae, 2,2, pp. 157-170.
• Haeckel, E. 1866/1988. Generelle Morphologie des Organismus, Bd. 2: Allgemeine Entwicklungsgeschichte. Berlin: de Gruyter.
• Handel, S. 1989. Listening. An Introduction to the Perception of Auditory Events. Cambridge - London: MIT Press.
• Hargreaves, D. 1986. The Developmental Psychology of Music. Cambridge - London - New York - New Rochelle - Melbourne - Sydney: CambridgeUniversity Press.
• Heylighen, F. & C.Joslyn. 2001. “Cybernetics and Second Order Cybernetics”. In: Meyers , R.A. (Ed.). Encyclopedia of Physical Science & Technology, Vol.4 (3rd ed.). New York: Academic Press, pp. 155-170.
• Hodges, D.A. (Ed.). 1996. Handbook of Music Psychology (2nd. ed). San Antonio: IMR Press, pp. 197-284.
• Hoffmeyer, J. 1997a. “Biosemiotics: Towards a new SynThesis in Biology”. European Journal for Semiotic Studies, 9, 2, pp. 2355-376.
• Hoffmeyer, J. 1997b. Signs of Meaning in the Universe. Bloomington: IndianaUniversity Press.
• Hoffmeyer, J. 1998. “Semiosis and biohistory: A reply”. Semiotica, 120, 3/4, pp. 455-482.
• Ingold, T. 1992. “Culture and the perception of the environment”. In: Croll, E. & D.Partin (Eds.). Bush base: forest farm – culture, environment and development. London: Routledge, pp. 39-56.
• Iyer, V. 2002. “Embodied Mind, Situated Cognition, and Expressive Microtiming in African-American Music”. Music Perception, 19, 3, pp. 387-414.
• Jackendoff, R. 1988. “Conceptual semantics”. In: Eco, U., M.Santambrogio & P.Violi (Eds.). Meaning and mental representations. Bloomington & Indianapolis: IndianaUniversity Press, pp. 81-97.
• Jackendoff, R. 1987. Consciousness and the Computational Mind. Cambridge (Ma) – London: MIT Press.
• James, W. 1911/968. The Writings of William James. A Comprehensive Edition. (Ed. J.McDermott). New York: Random House.
• James, W. 1912/1976. Essays in Radical Empiricism. Cambridge (Ma) – London: HarvardUniversity Press.
• Jeannerod, M. 1994. “The Representing Brain: Neural Correlates of Motor Intention and Imagery”. Behavioural Brain Sciences, 17, pp.187-202.
• Jiránek, J. 1998. “Réflexions sur les constances de sémantique musicale”. In: Miereanu, C. & X.Hascher (Eds.). Les Universaux en musique, Actes du 4e Congrès international sur la signification musicale. Paris: Publications de la Sorbonne, pp. 60-65.
• Johnson, M. 1987. The Body in the Mind. The Bodily Basis of Meaning, Imagination, and Reason.Chicago and London: The University of Chicago Press.
• Johnson, M. & S.Larson. 2003. ““Something in the Way She Moves”—Metaphors of Musical Motion”. Metaphor and Symbol, 18 (2), pp. 63-84.
• Klaus, G. 1972. Kybernetik und Erkenntnistheorie. Berlin: Deutscher Verlag der Wissenschaften.
• Kon, J. 1998. “Einige Bemerkungen über das Problem der musikalischen Universalien”.In: Miereanu, C. & X. Hascher (Eds.). Les Universaux en musique, Actes du 4e Congrès international sur la signification musicale, Paris: Publications de la Sorbonne, pp. 60–65.
• Kull, K. 1998. “Semiotic ecology: different natures in the semiosphere”. Sign System Studies,26, pp. 344-371.
• Kull, K. 2001. “Jakob von Uexküll: A paradigm for biology and semiotics”. Semiotica (Special Issue), 134, 1/4.
• Kurth, E. 1931. Musikpsychologie. Berlin: Max Hesses Verlag.1/4, pp. 599-633.
• Lakoff, G. 1987. Women, Fire, and Dangerous Things: What categories reveal About the Mind. Chicago: University of Chicago Press.
• Lakoff, G. 1988. “Cognitive Semantics”. In: Eco, U., M.Santambrogio & P.Violi (Eds.). Meaning and mental representations. Bloomington & Indianapolis: IndianaUniversity Press, pp. 119-154.
• Lakoff, G. 1990. “The Invariance Hypothesis: Is Abstract Reason Based on Image-Schemata?” Cognitive Linguistics, 1 (1), pp. 39-74.
• Lakoff, G.1993. “The contemporary theory of metaphor”. In: Ortonyi , A. (Ed.). Metaphor and Thought. Cambridge: CambridgeUniversity Press, pp. 202-251.
• Lakoff, G. 1994. What Is a Conceptual System? In: Overton, W. & D.Palermop (Eds.). The Nature and Ontogenesis of Meaning. Hillsdale, NJ: Lawrence Erlbaum, pp. 41-90.
• Lakoff, G. & M.Johnson. 1980. Metaphors We Live By. Chicago: University of Chicago Press.
• Lakoff, G. & Johnson, M. 1999. Philosophy in the Flesh: The Embodied Mind and its Challenge to Western Thought. New York: Basic Books.
• Langacker, R. 1986. “An introduction to cognitive grammar”. Cognitive Science, 10, pp. 1-40.
• Langacker, R. 1987. Foundations of cognitive grammar, Vol. 1. StanfordCA: StanfordUniversity Press.
• Laske, O. 1977. Music, Memory and Thought. Explorations in Cognitive Musicology.Ann Arbor, Profess MI: University Microfilms International.
• Lee, H. 1938. Perception and Aesthetic Value, New York: Prentice-Hall.
• Lidov, D. 1987. “Mind and body in music”. Semiotica, 66, 1/3, pp. 69-97.
• Lombardo, T.J. 1987. The Reciprocity of Perceiver and Environment. The Evolution of James J.Gibson’s Ecological Psychology. Hillsdale, NJ – London: Lawrence Erlbaum.
• Määttänen, P. 1993. Action and Experience. A Naturalistic Approach to Cognition. Helsinki: Academia Scientiarum Fennica.
• MacKay, D. 1992. “Constraints on Theories of Inner Speech”. In: Reisberg, D.(Ed.). 1992. Auditory Imagery. Hillsdale - Hove and London: Lawrence Erlbaum, pp. 121-149.
• Marconi, L. 1998. “Universals in Music and Musical Experiences”. In: Miereanu, C. & X.Hascher (Eds.). Les Universaux en musique, Actes du 4e Congrès international sur la signification musicale. Paris: Publications de la Sorbonne, pp. 657-666.
• Martindale, C. & K.Moore. 1989. “Relationship of Musical Preference to Collative, Ecological, and Psychophysical Variables”. Music Perception, 6, 4, pp. 431-446.
• Maturana, H. 1970. Biology of Cognition. Urbana: University of Illinois.
• Maturana, H. 1978. “Biology of Language: the Epistemology of Reality”. In: Miller, G. & E.Lenneberg (Eds.). Psychology and biology of Language and Thought. New York: Academic Press, pp. 27-46.
• Maturana, H. 1988. Ontology of observing: The biological foundations of self-consciousness and the physical domain of existence. Texts in Cybernetic Theory.
• Maturana, H. & F.Varela. 1987. The Tree of Knowledge: The Biological Roots of Human Understanding.Boston: New Science Library.
• Mazet, C. 1991. "Fonctionnalité dans l’organisation catégorielle”. In: Dubois, D. (Ed.). Sémantique et cognition. Catégories, prototypes, typicalité. Paris: Editions du CNRS, pp. 89-100.
• McAdams, S. 1984. “The auditory image: a metaphor for musical and psychological research on auditory organization”. In: Crozier, W.R. & A.J.Chapman (Eds.). Cognitive Processes in the Cognition of Art. North-Holland: Elsevier Science Publishers, pp. 289-323.
• McAdams, S. 1993. “Recognition of sound sources and events”. In: S.McAdams & E.Bigand (Eds.). Thinking in sound. The cognitive psychology of human audition. Oxford: Clarendon Press, pp. 146-198.
• McAdams, S. & E. Bigand (Eds.). 1993. Thinking in sound. The cognitive psychology of human audition. Oxford: Clarendon Press.
• Mead, A. 1999. “Bodily Hearing: Physiological Metaphors and Musical Understanding”. Journal of Music Theory, 43, 1, pp. 1-19.
• Mersmann, H. 1926. Angewandte Musikästhetik. Berlin: Max Hesses Verlag.
• Meystel, A. 1998. “Multiresolutional Umwelt: Towards a semiotics of neurocontrol”. Semiotica, 120, 3/4, pp. 343-380.
• Michaels, C. & C.Carello. 1981. Direct Perception. Englewood Cliffs, New Jersey: Prentice-Hall.
• Miereanu, C. & X. Hascher (Eds.) 1998. Les Universaux en musique, Actes du 4e Congrès international sur la signification musicale. Paris: Publications de la Sorbonne.
• Morris, C. 1938/1975. Foundations of the Theory of Signs, Vol.1, nr.2. Chicago – London: University of Chicago Press.
• Murdock, G.P. 1945. “The Common Denominator of Cultures”. In: R.Linton (Ed.). The Science of Man in the World Crisis. New York: ColumbiaUniversity Press, pp. 123-142.
• Neisser, U. 1967. Cognitive Psychology. New York: Appleton-Century -Crofts.
• Neisser, U. (Ed.). 1987. Concepts and Conceptual Development: Ecological and Intellectual Factors in Categorization. Cambridge: CambridgeUniversity Press.
• Normet, L. 1998. “Universals and their subdivisions”. In Miereanu, C. & X.Hascher (Eds.). Les Universaux en musique, Actes du 4e Congrès international sur la signification musicale. Paris: Publications de la Sorbonne, pp. 191-195.
• Padilla, A. 1998. “Les universaux en musique et la définition de la musique”. In: Miereanu, C. & X. Hascher (Eds.). Les Universaux en musique, Actes du 4e Congrès international sur la signification musicale. Paris: Publications de la Sorbonne, pp. 219–230.
• Paillard, J. 1994a. “La conscience”. In: Richelle, M., J.Requin & M.Robert (Eds.). Traité de psychologie expérimentale.2. Paris: Presses Universitaires de France, pp. 639-684.
• Paillard, J. 1994b. “L’intégration sensori-motrice et idéo-motrice”. In: Richelle, M., J.Requin & M.Robert (1994). Traité de psychologie expérimentale. 1. Paris: Presses Universitaires de France, pp. 925-961.
• Pask, G. 1975. Conversation, Cognition and Learning. Amsterdam: Elsevier.
• Pask, G. 1992. “Different kinds of Cybernetics”. In: Van de Vijver, G. (Ed.). New Perspectives on Cybernetics. Dordrecht - Boston – London: Kluwer Academic Publishers. pp. 11-31.
• Pattee, H. 1982. “Cell psychology: an evolutionary view of the symbol-matter problem”. Cognition and Brain Theory, 5, pp. 325-341.
• Pattee, H. 1985. “Universal principles of measurement and language functions in evolving systems”. In: Casti, J. & A.Karlqvist (Eds.). Complexity, Language, and Life: Mathematical Approaches. Berlin: Springer Verlag, pp. 268-281.
• Peretz, I. & R.Zatorre (Eds.). 2003. The Cognitive Neuroscience of Music. Oxford: OxfordUniversity Press.
• Piaget, J. 1936. La naissance de l’intelligence chez l’enfant. Geneva: Delachaux et Niestlé.
• Piaget, J. 1937. La construction du réel chez l’enfant. Geneva: Delachaux et Niestlé.
• Piaget, J. 1945. La formation du symbole chez l’enfant. Neuchâtel: Delachaux et Niestlé.
• Piaget, J. 1967. Biologie et connaissance. Essai sur les relations entre les régulations organiques et les processus cognitifs. Paris: Gallimard.
• Piaget, J. 1976. “Piaget’s Theory”. In: Inhelder, B. & H.Chipman (Eds.). Piaget and his School.New York: Springer.
• Polednak, I. 1985. “Zum Problem der Apperzeption der Musik”. IRASM, 16, 1, pp. 43-56.
• Prem, E. 1996. “Elements of an Epistemology of Embodied Artificial Intelligence”. In: M.Mataric (Ed.). Embodied Cognition and Action. (Technical Report FS-96-02.) Menlo Park: AAAI Press, pp. 97-101.
• Pylyshyn, Z. (1985). Computation and Cognition. Toward a Foundation for Cognitive Science.Cambridge - London: MIT Press.
• Reybrouck, M. 1989. “Music and the Higher Functions of the Brain”. Interface, Journal of New Music Research, 18, 1-2, pp. 73-88.
• Reybrouck, M. 1999. “The musical sign between sound and meaning”. In: Zannos, I. (Ed.). Music and Signs. Bratislava: ASCO Art & Science, pp. 39-58.
• Reybrouck, M. 2001a. “Biological roots of musical epistemology: Functional Cycles, Umwelt, and enactive listening. Semiotica, 134, 1-4, pp. 599-63.
• Reybrouck, M. 2001b. Musical Imagery between Sensory Processing and Ideomotor Simulation”. In: Godøy, R.I. & H.Jörgensen (Eds.). Musical Imagery. Lisse: Swets & Zeitlinger, pp. 117-136.
• Reybrouck, M. 2002. Understanding and creating music between measurement, computation and control: symbolic thinking and the experiential approach. Proceedings of the 2nd Conference “Understanding and Creating Music”. Caserta, 21 - 25 november 2002.
• Reybrouck, M. 2003a. “Deixis, Indexicality and Pointing as Heuristic Guides for Enactive Listening. Route Description, Cue Abstraction and Cognitive Maps”. Proceedings of the 5th Triennial ESCOM Conference. Hanover, 8-13 September 2003, pp. 298-301.
• Reybrouck, M. 2003b. “Musical semantics between epistemological assumptions and operational claims”. In: Tarasti, E. (Ed.). Musical Semiotics Revisited. Acta Semiotica Fennica XV. Imatra: International Semiotics Institute, pp. 272-287.
• Reybrouck, M. 2004a. Distinctions and observables: musical semantics between perceptual discrimination and cognitive economy. Lecture delivered at the Eight International Congress on Musical Signification, Paris,3- 8 Octobre 2004.
• Reybrouck, M. 2004b. “Music Cognition, Semiotics and the Experience of Time. Ontosemantical and Epistemological Claims”. Journal of New Music Research, 33, 4, pp. 411-428.
• Reybrouck, M. 2005a. “A Biosemiotic and Ecological Approach to Music Cognition: Event Perception between Auditory Listening and Cognitive Economy”. Axiomathes. An International Journal in Ontology and Cognitive Systems, 15, 2, pp. 229-266.
• Reybrouck, M. 2006a. “Musical Creativity between Symbolic Modelling and Perceptual Constraints: the Role of Adaptive Behaviour and Epistemic Autonomy”. In: I.Deliège & G.Wiggins (Eds.). Musical Creativity: Multidisciplinary Research in Theory and Practice. Oxford: Psychology Press, pp. 42-59.
• Reybrouck, M. (2006b). “The listener as an adaptive device: an ecological and biosemiotical approach to musical semantics”. In: E.Tarasti (Ed.). Music and the Arts. Acta Semiotica Fennica XXIII - Approaches to Musical Semiotics 10. Imatra - Helsinki: International Semiotics Institute - Semiotic Society of Finland, pp. 106-116.
• Reybrouck, M., L.Verschaffel & S.Lauwerier. Children’s drawings as representational tools for musical semantics: an investigation into the assessment of sense-making in a music-listening task (submitted).
• Rosch, E. 1977. “Classification ofreal-world objects: Origins and representations in cognition”. In Johnson-Laird, P. and P. Wason. Thinking. Readings in Cognitive Science.Cambridge - London - New York – Melbourne: CambridgeUniversity Press, pp. 212-222.
• Rosch, E. & B.Lloyd (Eds.) 1978. Cognition and Categorization. Hillsdale - New York: Erlbaum.
• Rosen, R. 1978. Fundamentals of Measurement and Representation of Natural Systems.New York: North-Holland.
• Saslaw, J. 1996. “Forces, Containers and Paths: The Role of Body-Derived Image Schemas in the Conceptualization of Music”. Journal of Music Theory. 40/2, pp. 217-244.
• Schaeffer, P. 1966. Traité des objets musicaux. Paris: Editions du Seuil.
• Scheerer, E. 1984. “Motor theories of cognitive structure: a historical review”. In: Prinz, W. & A.Sanders (Eds.). Cognition and motor processes. Berlin: Springer.
• Sebeok, T. 1998. “The Estonian Connection”. Sign System Studies, 26, pp. 20-41.
• Sebeok, T. & J.Umiker-Sebeok (Eds.). 1992, Biosemiotics. Berlin: Mouton de Gruyter.
• Shaw, R. & W.Hazelett. 1986. “Schemas in Cognition”.In: McCabe, V. & G.Balzano. Event Cognition: An Ecological Perspective. Hillsdale, NJ – London: Erlbaum, pp. 45-58.
• Shaw, R., O.Flascher, & W.Mace. 1996. “Dimensions of Event Perception”. In W.Prinz & B.Bridgeman. Handbook of Perception and Action. Volume 1: Perception. London - San Diego - New York - Boston - Sydney - Tokyo – Toronto: Academic Press, pp. 345-395.
• Shaw, R. & J.Pittenger. 1978. “Perceiving change”. In: Pick , H.L. & E.Salzman (Eds.). Modes of Perceiving and Processing Information. Hillsdale, New Jersey: Erlbaum.
• Shepard, R.N. 1984. “Ecological constraints on internal representation: Resonant kinematics of perceiving, imagining, thinking, and dreaming”
• Shove, P. & B.Repp. 1995. “Musical motion and performance: theoretical and empirical perspectives”. In: Rink, J. (Ed.). The practice of performance. Cambridge, U.K.: CambridgeUniversity Press, pp. 55-83.
• Sloboda, J. 1985. The Musical Mind. The cognitive psychology of music.Oxford: Clarendon Press.
• Sloboda, J. (Ed.). 1988. Generative Processes in Music. The Psychology of Performance, Improvization, and Compostition. Oxford: Clarendon Press.
• Spencer Brown, G. 1969. Laws of Form.London: Allen and Unwin.
• Swanwick, K. 1994. Musical Knowledge: Intuition, Analysis and Music Education. London: Routledge.
• Sweetser, E. 1990. From etymology to pragmatics: Metaphorical and cultural aspects of semantics.Cambridge, England: CambridgeUniversity Press.
• Tighe, T. & W.Dowling. 1993. Psychology and music. The understanding of melody and rhythm. Hillsdale - Hove and London: Hillsdale,
• Todd, P. 1999. “Motion in Music: A Neurobiological Perspective”. Music Perception, 17, 1, pp. 115-126.
• Varela, F., E.Thompson & E. Rosch. 1991. The Embodied Mind. Cognitive science and Human Experience. Cambridge (Ma)-London: MIT Press.
• Viviani, P. 1990. “Moto-perceptual interactions: the evolution of an idea. In: Piatielli Palmarini”, M. (Ed.). Cognitive Science in Europe: Issues and Trends. Golem Monograph Series, 1, pp. 11-39.
• Viviani, P. & Stucchi, N. 1992. “Biological movements look uniform. Evidence of motor-perceptual interactions”. Journal of Experimental Psychology, Human Perception and Performance, 18, pp. 603-623.
• von Bertalanffy, L. 1968. General System Theory. Foundations, Development, Applications.New York: Braziller.
• von Foerster, H. 1974 (Ed.). Cybernetics of Cybernetics. Illinois: University of Illinois.
• von Glasersfeld, E. 1978. “Radical constructivism and Piaget’s concept of knowledge”. In: Murray, F. (Ed.). The Impact of Piagetian Theory. Baltimore: University Park Press, pp. 109-122.
• von Glasersfeld, E. 1982. “An interpretation of Piaget’s constructivism”, Revue Internationale de la Philosophie, 36, 4, pp.612-635.
• von Glasersfeld, E. 1995a. “Einführung in die radikalen Konstruktivismus”. In: P.Watzlawick (Ed.). Die erfundene Wirklichkeit. Wie wissen wir, was wir zu wissen glauben,München: Piper, pp. 16-38.
• von Glasersfeld, E. 1995b. Radical Constructivism: A Way of Knowing and Learning.London - Washington: The Falmer Press.
• von Uexküll, J. 1934/1957. “A Stroll Through the Worlds of Animals and Men. A Picture Book of Invisible Worlds”. In: Schiller, C. (Ed.). Instinctive behavior. The Development of a Modern Concept. New York: International Universities Press, pp. 5-80.
• von Uexküll, J. 1937. “Die neue Umweltlehre. Ein Bindeglied zwischen Natur- und Kulturwissenschaften”. Die Erziehung, 13 5, pp. 185-197.
• von Uexküll, J. 1920/973. Theoretische Biologie. Frankfurt a/M: Suhrkamp.
• von Uexküll, J. 1940/1982. “The Theory of Meaning”. Semiotica, 42, pp. 25-82.
• von Uexküll, T. 1986a. “From Index to Icon. A Semiotic Attempt at Interpreting Piaget’s Developmental Theory”. In: Bouissac, P., M.Herzfeld & R.Posner (Eds.). Iconicity. Essays on the Nature of Culture. Festschrift für Th. A.Sebeok. Tübingen: Stauffenburg Verlag, pp. 119-140.
• Wallin, N. 1991. Biomusicology. Neurophysiological and Evolutionary Perspectives on the Origins and Purposes of Music. New York: Pendragon Press.
• Wallin, N., B.Merker & S.Brown (Eds.). 2000. The Origins of Music. Cambridge, MA - London: The MIT Press.
• Walser, R. 1991. “The Body in the Music: Epistemology and Musical Semiotics”. College Music Symposium. Journal of the College Music Society, 31, pp. 212-126.
• Werner, H. & B.Kaplan. 1963. Symbol Formation.An Organismic-Developmental Approach to Language and the Expression of Thought. New York - London – Sydney: John Wiley.
• Westerlund, H. 2002. Bridging Experience, Action, and Culture in Music Education. Studia Musica 16. Helsinki: SibeliusAcademy.
• Windsor, W.L. 1995. A Perceptual Approach to the Description and Analysis of Acousmatic Music.CityUniversity: Unpublished Doctoral Dissertation. (http://www.personal.leeds.ac.uk/~muswlw/pubs/lwthesis.html)
• Windsor, W.L. 2004. “An Ecological Approach to Semiotics”. Journal of the Theory of Social Behaviour, 34, 2, pp. 179-198.
• Zatorre, R. & I.Peretz (Eds.). 2001. The Biological Foundations of Music. New York: The New YorkAcademy of Sciences.
• Zbikowski, L. 1997, “Conceptual Models and Cross-Domain Mapping: New Perspectives on Theories of Music and Hierarchy”. Journal of Music Theory.41/2: pp. 193-225.
• Zbikowski, L. 1997-98. “Des Herzraums Abschied: Mark Johnson’s Theory of Embodied Knowledge and Music Theory”. Theory and Practice, 22-23, pp. 1-16.
• Zbikowski, L. 2002. Conceptualizing Music: Cognitive Structure, Theory, and Analysis.Oxford: OxfordUniversity Press.
• Ziemke, T. & N.Sharkey. 2001, “A stroll through the world of robots and animals: Applying Jakob von Uexküll’s theory of meaning to adaptive robots and artificial life”. Semiotica,134 (1-4), pp. 701-746.