1. Tuning in

This first chapter explains the words used in the course name (sensing, sound, and music), and why combining perspectives from musicology, psychology, and technology is useful.

Sensing sound and music¶

This course is called “Sensing Sound and Music”. But what does that mean? And what do those three words mean on their own?

Sound and music run through almost everything humans do, both as a way of communicating and as an art form. Sound is a physical phenomenon: vibrations travelling through a medium. Music is sound that has been organised so that it carries emotional, cognitive, and cultural meaning. We sense sound through the auditory system, but, as the course will show, also through the body. Exactly how we sense music is still one of the open questions in music psychology and it also has deep implications for music technology.

Levels of description: from physics to interpretation¶

Throughout the course it helps to keep in mind that we discuss sound and music at several levels. They are connected but they are not the same thing:

• Physical signal (physics) — Pressure waves in air (and in solids or liquids) obey acoustics: frequency–wavelength relationships, reflection, diffraction, and how instruments and rooms shape spectra and timing. This is the world microphones capture and loudspeakers re‑radiate. See Acoustics.
• Digital representation (DSP, digital signal processing) — Once transduced (converted into an electrical signal), sound becomes audio: voltages, then numbers. Sampling, quantisation, filtering, compression, and synthesis all operate here. The representation is always partial, and the choices made when digitising affect what can be recovered or analysed later. See Electroacoustics and Machine listening.
• Perception (psychoacoustics and basic auditory perception) — The ear and early auditory processing turn waveforms into what we distinguish as pitch, loudness, timbre, and location, often in ways that diverge from simple physics (masking, streaming, illusions). See Psychoacoustics and Listening.
• Interpretation (cognition and emotion) — Expectation, memory, attention, culture, and affect colour what sounds mean, whether as speech, music, warning, or ambience. Bodily states and neural circuits studied in psychology and neuroscience link to how we attend to and evaluate sound over time. See The brain, Physiology, and Harmony and melody for structured examples.

These four levels are the “spine” of the course. Each chapter from here on opens by saying which of them it works at, so that you always know whether a claim is about the air, the file, the ear, or the listener. It is easy to mix them up in discussion, that is why you should think about the differences from the start. We can record and measure sound with a microphone at the physical and digital levels. However, questions about experience require arguments that bridge into perception and interpretation. At the end of the course, you will have a much more solid understanding of how the different levels relate to each other and a vocabulary to speak about the differences.

An interdisciplinary approach¶

This course aims to be interdisciplinary. It draws on methods, theories, and perspectives from several fields to ask questions about sound and music. Interdisciplinarity is more than putting different kinds of knowledge side by side; here it means genuinely combining musicology, psychology, and technology. Looking at sound and music from these different angles tells us more about how people perceive, experience, and create music, and about how technology shapes those processes.

Some etymology¶

Etymology is the scientific study of the origin and history of words and how their meanings and forms have changed over time. The word “etymology” is itself derived from the Ancient Greek words étymon, meaning “true sense or sense of a truth”, and the suffix logia, denoting “the study or logic of”. Similarly, we can investigate the three disciplines in question:

• Musicology: The term “musicology” comes from the Greek words mousikē (music) and logia (study). It is the scholarly study of music, covering its history, theory, and cultural context. Musicologists analyse how music is composed, performed, and understood across different societies and eras.

• Psychology: “Psychology” is derived from the Greek psyche (soul, mind) and logia (study). It is the scientific study of the mind and behaviour. Music psychologists explore how people perceive, process, and respond to music emotionally and cognitively.

• Technology: The word “technology” originates from the Greek techne (art, craft, skill) and logia (study). Technology refers to the tools, techniques, and systems humans create to solve problems or enhance capabilities. Music technology includes the development and studies of instruments, recording devices, software, and other innovations that help create, analyse, or experience music.

By the end of the course, you will have learned some of the basic terminology, theories, and methods used in all these directions.

Differences between disciplinarities¶

When approaching different disciplines, it is vital to understand how they interact. Many people call all sorts of collaboration between disciplines “interdisciplinarity”. However, there are some differences, highlighted in this figure:

Intradisciplinary work stays within a single discipline, while crossdisciplinary approaches view one discipline from the perspective of another. Multidisciplinary collaboration involves people from different disciplines working together, each drawing on their own expertise. Interdisciplinary work integrates knowledge and methods from multiple disciplines, synthesising approaches for a deeper understanding. Transdisciplinary approaches go further, creating unified intellectual frameworks that transcend individual disciplinary boundaries.

One approach is not better than another, and many researchers may take on different roles depending on the project type. Still, it is helpful to consider these differences as we approach each discipline and look at how they combine theories and methods.

Concepts, theories, methods¶

A discipline is held together by a set of concepts, theories, and methods, together with shared traditions, specialised terminology, and agreed standards for judging good work. These are usually kept alive through journals, conferences, and professional organisations. What a discipline treats as its territory is the product of its history, its core questions, and the kinds of problems it has taken on.

In our case, each of the three disciplines brings its own theories and methods. Roughly speaking, musicology uses analysis, historiography, and ethnography; psychology runs experiments and surveys; and technology builds on engineering and computational modelling. But as soon as you start reading the literature and following how the fields actually developed, you see that none of them grew in a straight line, and that things get more entangled once they are mixed together.

A closer look: the Mozart effect¶

Critical thinking is a key skill to develop as a student. You need to learn how to read research results and make up your mind about how solid its conclusions are. Let us discuss one famous claim that made headlines some years ago. The Mozart effect is the popular idea that listening to Mozart makes you more intelligent. In the 1990s it sold recordings and toys, and the US state of Georgia even funded classical CDs for newborns based on its claims.

• The claim began modestly. Rauscher et al. (1993) reported that college students performed better on a spatial reasoning task after ten minutes of a Mozart sonata than after relaxation instructions or silence.
• The evidence was thin: 36 participants, one laboratory session, and an advantage that faded within about fifteen minutes. Nothing was measured about general intelligence, about children, or about long-term listening.
• The method was a standard psychology experiment, and it invited replication. Many groups tried, and most found little or nothing.
• The limits became clear when Pietschnig et al. (2010) pooled nearly forty studies with over three thousand participants in a meta-analysis. The remaining effect was small, appeared after any enjoyable stimulus, and was inflated by selective publication of positive results. Short-lived arousal and mood explain the data better than anything specific to Mozart.

The lesson is not that music does nothing, but that a claim, its evidence, and its method have to be read together. The reading checklist in the introduction exists for exactly this situation.

Music psychology and technology¶

Most people are unfamiliar with music psychology and technology as academic disciplines. The same could be said about musicology, although some musicological fundamentals are usually taught in schools, including basic (Western) music theory (and notation) and music history. The chapter harmony and melody picks up that thread after the acoustics and psychoacoustics units, connecting frequency and pitch to the symbols and concepts you may already use in analysis. Let us consider some of the fundamentals of each discipline.

Fundamentals of music psychology¶

Music psychology focuses on understanding how humans perceive, process, and respond to sound and music. This includes exploring topics such as:

• Perception: How the auditory system and brain transform acoustic signals into musical attributes: pitch, loudness, timbre, rhythm, and spatial location.
• Cognition: Higher‑level mental processes for understanding music: memory, expectation, pattern and structure recognition, attention, and segmentation.
• Emotion: How music evokes, communicates, and regulates affective states, including physiological responses, appraisal, and mood modulation.
• Action & Behaviour: Ways music drives movement and interaction, such as motor coordination, entrainment, performance practice, dance, and social bonding.
• Development & Learning: Acquisition and change of musical abilities and preferences across the lifespan, from infant sensitivity to skilled expertise and cultural learning.

In this course, we will primarily investigate perception, and briefly touch on cognition and behaviour.

Music psychology is a thriving field internationally, with numerous communities, conferences, and journals:

Given its interdisciplinary nature, music psychology researchers are typically working in either musicology or psychology departments, which also often “skews” the research in one or the other direction. Researchers employed in musicology departments tend to be more focused on real-world musical experiences, what is often termed “ecological validity,” and using (more) qualitative methods. Researchers in psychology departments typically work (more) on controlled experiments and use quantitative methods.

While many researchers in music psychology may often feel “alone” in their respective departments, there are a few larger, specialised departments or research centres that focus specifically on music psychology. In these institutions, one can often see the width of the field, covering many different theoretical and methodological perspectives. RITMO is one such example.

Fundamentals of music technology¶

Many people think about the products coming out of music technology research. However, it is important to understand that music technology (the “logy” of the techne) is a discipline on its own, involving creating, using, and reflecting on tools and systems for making, analysing, and manipulating sound. Key areas include:

• Sound synthesis and analysis: Techniques for generating and modelling sound (additive, subtractive, FM, wavetable, granular) and for analysing its structure (time-domain, spectral analysis, resynthesis). Applied to instrument and timbre design, sound design, and research into perceptual attributes.

• Digital audio recording and processing: Capture, edit, mix, and transform audio using DAWs and audio toolchains. Typical tasks include multitrack recording, editing, filtering, equalisation, dynamics processing (compression/limiting), time- and frequency-based effects (reverb, delay, pitch‑shift, time‑stretch), and mastering workflows.

• Music Information Retrieval (MIR): Computational extraction of musical information from audio and symbolic data. Often implemented with machine learning and signal‑processing libraries for applications like recommendation systems, analysis, and musicology.

• Interactive systems and new interfaces: Design and implementation of real‑time, responsive systems for performance and interaction. Covers sensors and controllers, expressive mappings, communication protocols (MIDI, OSC), and platforms/frameworks for prototyping. Used in digital instruments, live coding, installations, and adaptive performance systems.

Music technology researchers are typically employed in departments of musicology, engineering, or informatics. Many of them combine creative and artistic exploration with scientific inquiry.

There are also many international journals, communities, and annual conferences in music technology:

Comparing music psychology and technology¶

Even though we actively try to unify perspectives from psychology and technology in this course, there are some essential differences between the two disciplines. Music psychology is primarily a scientific field of study, focused on understanding how humans perceive, process, and respond to music through empirical research and theoretical frameworks. Its methods are rooted in experimental design, data analysis, and psychological theory, aiming to uncover universal principles and individual variations in musical experience.

Music technology, on the other hand, spans several domains: science, art, design, and engineering. It encompasses scientific research into sound and audio processing, artistic exploration through composition and performance, design of musical instruments and interfaces, and engineering of hardware and software systems. Music technologists may work on developing new tools for music creation, analysing audio signals, designing interactive installations, or exploring creative possibilities in digital media. This diversity means that music technology is not limited to scientific inquiry but also includes creative practice, technical innovation, and user-centred design.

At the University of Oslo, we have a long tradition of combining music psychology and technology. The logic behind this is that advanced technologies can help psychological inquiry, and psychological insights can impact the development and use of new technologies. And both disciplines can help us better understand music as a whole.

Listening to the world¶

After getting introduced to some of the (inter)disciplinary foundations for this course, let us go back to some basics: listening. We will spend next week’s class only on listening, but we will start with a little warm-up here. After all, listening is an essential human capacity and central to music, and to psychology and technology. However, most people do not think much about listening in daily life, except for when it is annoying. With a sound level too low, it is hard to hear what is said; with a sound level too loud, it is unpleasant and even dangerous. But why is that, and how do we talk about it in precise terms? That is what we will explore both here and later in class.

Hearing vs listening¶

Let us start with the difference between hearing and listening. We will take hearing to be the passive, physiological business of detecting sound waves through the auditory system, and listening to be the active, cognitive business of interpreting those sounds and making meaning of them.

In the psychology literature, and in Hallam et al. (2014) in particular, you will find lists of different types of hearing and listening:

• Passive hearing: This occurs when sounds are registered by the ears and processed by the brain without conscious attention. For example, background noise in a café or the hum of an air conditioner.
• Selective hearing: The ability to focus on specific sounds or voices while ignoring others, such as following a conversation in a noisy environment (also known as the cocktail party effect).
• Active listening: Engaging attention and intention to understand, analyse, or respond to sounds or speech. This matters in music appreciation, communication, and learning.
• Critical listening: Evaluating and analysing sound quality, musical structure, or meaning, often used in music production, performance, or academic study.
• Empathetic listening: Focusing on the emotional content and intent behind sounds or speech, important in social interactions and therapeutic contexts.

The mechanisms behind these kinds of hearing and listening are still under active investigation, and opinions diverge on how exactly they differ, overlap, or connect. The distinctions are still useful. They give us a vocabulary for the move from automatic sensory processing to deliberate engagement and interpretation.

Embodied music cognition¶

The view of human psychology in this book is not neutral (no book’s is, even when it pretends otherwise). It grows out of embodied music cognition, which stresses the role of the body in musical experience. Marc Leman did not coin the term, but it became widely known after his book of the same name Leman, 2007.

Embodied music cognition looked radical at first and has since become fairly mainstream. Researchers at the University of Oslo, starting with professor (now-emeritus) Rolf Inge Godøy, have helped develop this line of work over the last few decades. It underpins the studies of music-related body motion in the fourMs Lab and much of what goes on at RITMO.

The core idea of embodied music cognition is that producing and perceiving music involve sensory, motor, and emotional dimensions at once. To paraphrase James J. Gibson, the founder of ecological psychology: you explore the world with eyes in a head on a body that moves around. Gibson was mainly interested in vision, but his thinking has shaped how others approach the rest of the senses.

A related idea is that of musicking. The musicologist Christopher Small argued that music is best understood as a verb—to music—rather than a noun Small, 1999. Music is not an object or a finished product, like a song or a score, but something people do. When a band plays a concert, the musicians, the dancers, the audience, the sound engineers, even the people selling tickets and cleaning the hall, are all musicking. The music lives not only in the notes played but in the shared experience and interaction.

It is particularly researchers in music education and therapy that use the term musicking because it lets them treat music as a social activity rather than a thing. You no longer study “the music” in isolation; you study the whole web of musical interaction, from the gestures of a performer to the way listeners synchronise their bodies to a beat. Bringing the body in is what makes embodied music cognition a way of studying how music is actually experienced and understood.

Multimodality¶

The French composer Edgard Varèse famously argued that “music is organised sound”. This was his starting point for moving beyond melody and harmony, exploring oise and timbre as compositional material. Music is any sound deliberately shaped and structured over time.

In this course we widen the idea further. We treat music as inherently multimodal. We—like other animals—perceive and make sense of the world through several senses at once, not through hearing alone.

For psychologists, multimodality is about how the brain combines information from these channels into a single experience. In music, the clearest case is the link between hearing and body movement. A performer’s visible gestures change how we hear their sound, and our own movements feed into how we understand it. But the other senses take part too. Touch gives instrumentalists and singers feedback on what they are doing; proprioception and balance help musicians coordinate posture and motion, especially in dance and performance; and smell or taste can colour the emotional context and memory of a musical event.

Most people would agree that sound is at the heart of music, yet there are striking cases of musicians who have made music with severe hearing loss. Ludwig van Beethoven kept composing major works after becoming profoundly deaf, leaning on vision and touch, and the percussionist Evelyn Glennie performs by feeling vibrations through her hands, feet, and body. How touch and vibration carry musical information is explored in the body.

Cross-modal associations¶

Beyond combining sight and sound, experiments often turn up systematic cross-modal correspondences: many listeners agree that, for example, higher pitches go with brighter lights or smaller shapes. Many such mappings are well-known and often used in film and game scoring. It should be noted such correspondences are not the same as synaesthesia, which are specific and consistent connections experienced by some (few) individuals.

Joint timing between people—such as in ensemble playing, choral breathing, dancing together—returns from an embodied angle in the chapter on the body; auditory–visual laboratory phenomena are treated systematically under Vision.

Action and perception¶

The last concept for this week is the action–perception loop. The general claim of embodied cognition is that we sense through action and act through sensing. In music, you can see this whenever someone plays an instrument. The sound they hear shapes the next thing they do, and what they do shapes the next sound. A pianist adjusts their touch in response to the tone just produced, and round it goes.

Eric Clarke puts the action–perception loop at the centre of his idea of ecological listening, drawing together psychology, musicology, and acoustic ecology to ask how we engage with sound in everyday life. In his book Ways of Listening Clarke, 2005 he sets out three listening modes:

• Direct perception: Listeners perceive sounds in relation to their environment without needing extensive cognitive processing.
• Affordances: Sounds provide cues about actions or interactions possible within a given environment.
• Contextual listening: The meaning of sounds is shaped by their environmental and situational context.

Clarke also points out that we often hear sounds as “the sound of” something: “the sound of a guitar”, “the sound of a scream”. This source-based listening sits neatly within his three modes: identifying the source is direct perception, hearing what it lets you do is affordance, and reading its situational meaning is contextual listening. Attributing sounds to objects or events is one of the most common ways we make sense of what we hear, and it is one of several listening perspectives we pick up next week.

Chapter summary¶

This chapter framed how sensing, sound, and music are defined for the course; why musicology, psychology, and technology each offer partial but complementary views; and how interdisciplinarity, listening modes, embodiment, multimodality, and the action–perception loop connect everyday musical activity to the topics that follow. A closer look at the Mozart effect showed why a claim, its evidence, and its method need to be read together.

Questions¶

1. How do the three pillars of this course—musicology, psychology, and technology—each contribute to understanding sound and music, and what are the risks of staying inside a single discipline?
2. What is the difference between hearing and listening in this course’s sense, and why does that distinction matter for research and practice?
3. How does embodied cognition help explain music perception and performance beyond “purely auditory” accounts?
4. What is the action–perception loop, and how might you observe it in a concrete musical situation you know?
5. In the Mozart effect case, what exactly did the original experiment show, and how did the later meta-analysis change the picture?