So far, we have treated sound and music mainly as something we hear. But we also perceive music through our bodies. This is the point where the four levels of description stop being a tidy ladder: a performer’s gesture is a physical event, a perceptual cue, and an interpretive act at the same time, and the whole argument of embodied cognition is that separating them distorts what is going on. This week looks at how the body shapes music perception and cognition: the body’s part in musical experience, some anatomy and biomechanics, and methods for analysing human movement.
Embodied music cognition¶
You met embodied music cognition at the start of this course. An embodied approach stresses the role of the body in producing, perceiving, and understanding music. Instead of treating musical experience as only an auditory phenomenon—or as something purely mental or abstract—it shows how bodily sensations, movements, and actions carry musical meaning.
Some key ideas in embodied music cognition include:
Sensorimotor coupling: Listening to music often triggers spontaneous bodily responses—tapping a foot, nodding, swaying—showing the close connection between perception and action. This is the action–perception loop: perceiving music can prompt movement, and movement, in turn, shapes how the music is experienced.
Multimodal integration: Musical experience is multimodal. It draws together auditory (sound), visual (sight), and kinaesthetic (movement and bodily sensation) information, and the body coordinates these streams. Watching a performer’s gestures can sharpen our sense of phrasing, while feeling vibrations or movement can deepen our sense of rhythm and timing.
Gesture and imagery: Expressive body movements, from large gestures to small adjustments, help musicians communicate, phrase, and convey emotion. They shape the sound and also shape how performers and listeners represent music in their minds. Even without overt movement, musicians and listeners often engage in motor imagery, mentally simulating the gestures tied to a piece, which supports learning, memory, and interpretation.
The Belgian systematic musicology professor Marc Leman popularised the term embodied music cognition in the early 2000s. In his book Embodied Music Cognition and Mediation Technology Leman, 2007, he traces the different processes at work in the bodies of both performers and perceivers, and describes how musical intentionality rests on sonic and visual communication between them:

An illustration of Marc Leman’s model of embodied music cognition (illustration from Jensenius (2022)).
Embodied music cognition is studied across several disciplines, including musicology, psychology, neuroscience, and human movement science. The literature therefore holds many perspectives and methods. Most of this research is empirical. It collects and analyses data about musical experiences and bodily responses in a systematic way.
4E cognition¶
In recent years, the idea of 4E cognition has gained ground, building on the “embodied turn” in cognitive science and philosophy of mind. The 4E framework—standing for Embodied, Embedded, Enactive, and Extended cognition—challenges the traditional view that cognition is confined to the brain.
Researchers such as Francisco Varela, Evan Thompson, Alva Noë, and Andy Clark argue that understanding cognition means taking the body, the environment, and their interactions into account. On the 4E view, cognitive processes are:
Embodied: Our perception and understanding of music are rooted in the body. How we move, breathe, and physically interact with instruments or our surroundings shapes how we experience and interpret sound. Tapping a foot to a beat or feeling vibrations through the body are embodied ways of sensing music.
Embedded: Musical experience is shaped by its context. Our interactions with sound depend on the physical and social setting: the acoustics of a concert hall, the presence of other listeners, the cultural frame. These factors embed musical cognition in a wider world.
Enactive: We take an active part in making musical meaning. Listening is not passive; we anticipate, move, and respond, co-constructing the experience. A performer’s gestures or a listener’s dance are forms of enactive engagement.
Extended: Tools and technologies can extend our cognitive processes. Instruments, recording devices, and even smartphones become part of the system through which we sense, produce, and understand music. Motion capture systems, for instance, extend our ability to analyse and reflect on musical movement.
The 4E perspective treats music as an interactive process that involves the whole person, situated in a specific context, working with both physical and digital tools.
The framework has also drawn criticism, much of it gathered in the Oxford Handbook of 4E Cognition Newen et al., 2018. One concern is that the definitions of “embodied,” “embedded,” “enactive,” and “extended” can be vague or overlapping, which makes 4E hard to tell apart from traditional cognitive science in practice. Another is whether there is enough empirical evidence for every part of the framework, especially the claim that cognition can genuinely be “extended” into tools or the environment. Critics also warn against overreach, granting cognitive status to objects such as smartphones or instruments that may not really take part in cognition. Some argue that 4E approaches downplay the neural mechanisms that traditional neuroscience emphasises. And there is lasting philosophical disagreement over whether 4E is a radical break or a reframing of older ideas about mind, body, and environment. Even so, the perspective has prompted lively interdisciplinary debate and new research in music cognition and neighbouring fields.
The body in music performance¶
When we talk about music-related body motion, it helps to separate performers from perceivers. Their roles differ, but both depend on the body to engage with music. Let us start with performers.
Music performance is physical. Musicians use coordinated actions—goal-directed, time-limited motion sequences—to produce sound, shape phrases, and communicate with others. The body is both the source of musical ideas and their interpreter, turning intention into audible and visible action. This holds for all music-making: singing, playing instruments, conducting, and dancing.
We can sort music-related body motion in performers into four main types:
Sound-producing actions: Movements that directly generate sound on an instrument or interface. They include selection (choosing notes or sounds), excitation (initiating sound by striking, plucking, or bowing), and modification (altering qualities such as pitch or timbre). Examples: pressing piano keys, bowing a violin, turning a synthesiser knob.
Sound-facilitating actions: Movements that support sound production without making sound themselves, such as holding posture, shaping phrasing, and entraining the body to rhythm. Examples: a pianist’s arm movement for dynamics, a clarinettist’s breath support, tapping a foot to keep time.
Sound-accompanying actions: Motion that reflects or mimics musical features without producing sound, such as tracing sound contours in the air, mimicking instrumental gestures, or air performance. Examples: moving a hand upward with rising pitch, playing air guitar, dancing to music.
Communicative gestures: Gestures that convey meaning, emotion, or instructions to other performers or the audience. They may be expressive, regulatory, or linguistic. Examples: a conductor’s baton movements, a nod to cue an entrance, expressive hand gestures.
These types of action sit along a continuum of connection to musical sound, as shown below:

Relationship between motion and sound (illustration from Jensenius (2022)).
The categories are useful for clarity, but in practice they overlap. Sound-facilitating actions are often inseparable from sound-producing ones, and performers may use communicative gestures while playing. Keeping this interplay in view matters when studying music-related body motion.
The body in music perception¶
Many of the same actions show up in perceivers, people “listening” to music. We put “listen” in quotation marks because we experience music with the whole body, which is the heart of embodied music cognition. Perceivers also make sounds during a performance, whether involuntary (breathing) or voluntary (clapping, singing along).
Perceiver motion can be voluntary or involuntary, and it shapes how we experience and understand music. Examples include:
- Dancing: Engaging the whole body in rhythmic movement, often in response to the beat, melody, or emotional content of the music. Dance can be highly structured (as in ballroom or folk dance) or spontaneous and improvised. It is a direct way for listeners to embody musical structure, rhythm, and emotion, and it is a universal part of musical cultures.
- Air performance: Imitating the actions of playing an instrument or singing: “air guitar”, “air drumming”, or lip-syncing. These gestures reflect an embodied connection to the music and can heighten engagement. Air performance lets listeners simulate being a performer, reinforcing their grasp of musical gestures and techniques.
- Finger-tapping or foot-stamping: Tapping fingers on a surface, or stamping feet to the beat, whether involuntarily or on purpose. These widespread, often automatic responses help listeners synchronise with musical timing and structure. Finger-tapping in particular is widely used in research as a behavioural measure of beat perception and sensorimotor synchronisation, giving insight into how brain and body interact during listening.
- Involuntary swaying or nodding: Subtle motion, such as swaying or head-nodding, that happens without conscious intent. These responses are tied to the brain’s sensorimotor coupling with musical rhythm and can be observed across cultures and age groups. They arise spontaneously and reflect how closely the auditory and motor systems are integrated.
These bodily responses are not mere by-products of listening; they are part of musical perception and cognition. Moving to music can strengthen memory, emotional response, and social connection.
Research shows that moving while listening can aid rhythm and beat perception, helping listeners internalise and predict patterns and follow complex or syncopated music. Movement also heightens emotional engagement, intensifying responses such as joy, excitement, or nostalgia. Gestures and movement support the learning and memory of melodies, lyrics, and rhythms, which is why movement is often built into music education. And group movement—dancing or clapping together—builds social bonds through a shared sense of unity.
Micromotion and standstill¶
The involuntary end of that list is easy to assert and hard to pin down, because the movements are small and people stop making them as soon as they notice they are being watched. One way around this is to invert the task: instead of asking people to move to music, ask them to stand as still as they possibly can, and measure what is left.
This is the design behind the Norwegian Championship of Standstill, run annually at the University of Oslo since 2012, in which participants stand still in groups for six minutes while an infrared motion-capture system records the position of a marker on the head. The competitive framing matters, since it removes any suspicion that participants are moving because they cannot be bothered not to.
What remains is micromotion, the residual sway of a body doing its best to hold a position. In one such study Jensenius et al. (2017), 91 participants stood still for three minutes in silence and then three minutes with music. Nobody managed to be still. The average speed of head motion was around 6.5 mm/s, and it was slightly but consistently higher with music playing than in silence.
Two things are worth taking from this. The first is the scale: the effect is measured in millimetres per second, so it is nothing you would see in a video, and claims about it depend entirely on the measurement techniques described later in this chapter. The second is the direction. Music does not have to be danced to in order to move you, and the coupling between hearing rhythm and moving is persistent enough that instructions to suppress it only reduce it. The involuntary swaying in the list above is not a figure of speech.
Joint action, dance, and tactile engagement¶
Joint action¶
Joint action in music covers situations where two or more people align perception and movement towards a shared goal in time: locking to the same pulse in an ensemble, adjusting attacks to a conductor’s gesture, breathing with a choir, or coupling dance steps to musical metre. The underlying mechanism is the entrainment of coupled oscillators described in time and rhythm; here the oscillators are people. Cognitive scientists sometimes separate shared intentionality (we mean to play “this” phrase together) from mere interpersonal synchrony (our movements happen to coincide); both turn up in rehearsal and performance.
Coordination draws on several channels at once. Auditory information alone can be enough, but visual cues such as conducting, head nods, and instrument motion, and even tactile contact or spatial proximity, all support alignment. Roles matter too. Leader–follower relationships and intentional asynchrony, such as a jazz soloist laying back behind the beat or a band pushing forward, are musically meaningful rather than errors, and musical joint action needs constant error correction, the micro-delays and anticipations that keep parts distinct yet coordinated.
Empirical work links synchrony to social cohesion and affect. Group settings nevertheless introduce confounding factors that solo studies avoid: the mere presence of others (social facilitation), the awareness of being judged (evaluation apprehension), and shared excitement can all change both movement and physiology, so claims about “bonding” or “unity” from synchrony alone need careful design and interpretation.
Dance¶
Dance couples full-body movement with musical time, metre, and phrase structure in culturally specific ways. Foot patterns, spine shape, and use of space are not decorative add-ons; they are part of how rhythm and identity are expressed. Ethnochoreology and dance studies document repertoires where “the beat” lives as much in stride and bounce as in the drums. For a listening-oriented course, the main point is that dance–music relationships are learned and stylistic—compare, for instance, club genres with contrasting upper- versus lower-body emphasis—and that motion capture and video analysis now sit alongside audio when researchers study groove.
Haptics and tactile channels¶
Haptics covers active touch and forces, and it overlaps with passive tactile vibration for musicians: fingering contact on strings or keys, stick rebound, and the resonance of an instrument body against the torso. Sound also reaches the skin directly. At a concert, the low-frequency energy of bass and kick drum radiates through floors and bodies, so you feel the drop in your chest as much as you hear it.
The percussionist Evelyn Glennie, who lost most of her hearing by the age of twelve, has built a world-class career on exactly this channel. She learned to feel vibrations through her hands, feet, and body, often performing barefoot, and uses that information to play and interpret music, as she describes in her TED talk How to truly listen:
(If the video is unavailable, try the archived page.)
Vibrotactile feedback puts the same channel to deliberate use. Digital musical instruments often map sound parameters to compliant controllers so performers “feel” the synthesis change under their hands, and wearable devices can deliver rhythmic cues through the skin. Accessibility-oriented concert designs for Deaf and hard-of-hearing audiences route timing and pulse through vibrating platforms or wearables, and when masking makes hearing ambiguous, touch can hold the pulse steady. Together with hearing—and vision when available—these channels make up multimodal musical engagement.
Anatomy and biomechanics¶
To make sense of motion capture, we first need to know how the body is built and how it moves. This section gives an overview of anatomy and biomechanics as they relate to motion analysis.
If this looks like a detour into sports science, it is worth saying why it is here. Every claim about music-related movement rests on a description of what moved, relative to what. A study of a violinist’s bowing needs to say whether the arm rotates at the shoulder or the elbow, and in which plane; a study of pianists’ injuries needs joint angles and ranges; a comparison of drummers’ strokes needs to separate the motion of the stick from the motion of the wrist that drives it. Without a shared vocabulary these descriptions become “the arm went up a bit”, which no one can replicate or measure. The terms below are that vocabulary. You do not need to memorise them, but you will meet them in every empirical paper on music and movement.
Anatomical position and planes¶
Anatomy studies the structure of the body. To keep descriptions of locations and actions consistent, we use the anatomical position: standing upright, head and eyes forward, arms at the sides with palms facing forward, and feet parallel and pointing ahead.

The human body in the anatomical position, with labelled regions.
The body is divided into regions—head, neck, trunk, upper limbs, and lower limbs—each with further subdivisions. For movement analysis, it helps to distinguish areas such as the arm and forearm, or the thigh and leg.
To describe positions and movements in three dimensions, we use anatomical planes: imaginary divisions of the body that give standard reference points for anatomical terms and motion analysis. The sagittal plane divides the body into left and right sections (the median plane runs exactly down the middle), the frontal (coronal) plane separates front (anterior) from back (posterior), and the transverse plane divides the body horizontally into upper (superior) and lower (inferior) parts. These planes let us describe the direction and type of movement precisely, in both clinical and research settings.

The three main anatomical planes: sagittal (divides left and right), frontal/coronal (divides front and back), and transverse (divides upper and lower parts of the body).
To describe movement directions, the belly button (navel) is often used as a reference point for the whole body, though other landmarks may be chosen for specific segments. Movements are usually described along three primary axes: medial–lateral (side-to-side or left–right), anterior–posterior (front-to-back), and superior–inferior (up–down). These axes correspond to the sagittal, frontal, and transverse planes, and they standardise descriptions of motion in research and clinical work.

Common directions of human body motion, illustrated with arrows.
The muscular system¶
The musculoskeletal system produces and controls movement. It has two main parts: the muscular system and the skeletal system. The muscular system is made up of muscles that act on the skeleton to move or position body parts, while the skeletal system—bones, joints, and cartilage—provides structure and protection. We start with the muscles.
There are three types of muscle tissue: cardiac (heart), smooth (organs), and skeletal (attached to bones). Skeletal muscles handle voluntary movement.
A skeletal muscle has a thick, red muscle belly and narrow, white tendons at each end that anchor it to bones. When the muscle contracts, it pulls on the tendons, and the attached bone moves.

When a skeletal muscle contracts, it pulls on the attached bone via tendons, resulting in movement at the joint.
Muscles can only pull, not push, so movement usually involves several muscles working together. The agonist is the main muscle driving a movement; synergists assist it by adding force or stabilising the origin bone (sometimes called fixators); and the antagonist produces the opposite action, balancing or resisting the agonist so movement stays controlled and smooth.
With over 600 skeletal muscles, only the major superficial ones are shown here:

Major muscle groups of the human body, shown from the front (left) and back (right).
The skeletal system¶
The adult skeleton has about 206 bones, which form the body’s framework. Bones act as levers for movement and give muscles their attachment points. Many bones have distinct landmarks, features that anchor muscles and can often be felt on your own body.

Major bones and bone groups of the human body, shown from the front (left) and back (right).
Joints are the connections between bones that let the skeleton move. The structure of each joint determines its possible directions and range of motion. Knowing how joints move helps us analyse how the body produces complex actions, such as those in music performance.
Joint movements are usually described as pairs of opposite actions, always referenced from the anatomical position: flexion and extension in the sagittal plane; abduction and adduction in the frontal plane; and internal (medial) rotation and external (lateral) rotation in the transverse plane.

Major movement types at the joints, including flexion/extension, abduction/adduction, and internal/external rotation. Movements are always described relative to the anatomical position.
Degrees of freedom (DoF) are the number of independent directions in which a joint can move. Each DoF stands for a specific type of movement (flexion/extension, abduction/adduction, rotation). Range of motion (RoM) describes how far a joint can move within each DoF, usually measured in degrees. Together, DoF and RoM tell us how a joint functions, what it can do, and where its limits lie, in everyday activity and in demanding tasks like music performance.
Biomechanics: principles of human movement¶
Biomechanics studies how mechanical principles apply to living organisms, especially the human body. Its key areas include:
- Statics: Bodies at rest or in equilibrium (e.g., standing, holding a posture).
- Dynamics: Bodies in motion (e.g., walking, playing an instrument). It splits into kinematics and kinetics.
To analyse movement, we use a reference frame, a coordinate system for describing positions and motion. A global reference frame is fixed to the environment, such as the laboratory or stage, and acts as an external standard for measuring movement. A local reference frame is attached to a body segment, such as the hand relative to the forearm, so we can describe motion in relation to other body parts. Combining global and local frames lets us describe human movement both precisely and in context.
Kinematics¶
Kinematics describes motion—how body parts move—without considering the forces behind it. It answers what moves, where, and how fast.
Kinematics is what a motion capture system actually records: a stream of positions over time, from which everything else is derived.
Dig deeper: kinematic terms
- Position: The location of a point or segment in space, usually given as X, Y, Z coordinates.
- Displacement: The straight-line change in position from start to end (a vector quantity).
- Distance: The total length of the path travelled, regardless of direction (a scalar quantity).
- Speed: How fast something moves, without regard to direction (scalar).
- Velocity: Speed in a particular direction (vector), both how fast and which way.
- Acceleration: The rate at which velocity changes over time, describing how quickly something speeds up or slows down.

The figure above shows the difference between displacement (the shortest path from start to end) and distance (the total path travelled).
Kinematic analysis lets us track movement patterns in music performance, such as the trajectory of a violinist’s bow or a pianist’s hand.
Kinetics¶
Kinetics examines the forces and torques that produce or result from movement, focusing on why and how motion happens.
Kinetics asks what caused that motion. Motion capture cannot see forces directly, so kinetics is usually inferred from kinematics plus a model of the body, or measured separately with force plates and sensors.
Dig deeper: kinetic terms
- Force: Any push or pull that can change a body’s motion. Muscles generate internal forces; external forces include gravity, ground reaction, and contact with objects or instruments.
- Torque (moment): A rotational force that turns a body segment around a joint or axis. Muscles create torque to move limbs and control posture.
- Power: The rate at which work is done or energy transferred during movement. In performance, power shows up in actions like striking a drum or bowing a string.
- Balance: Keeping the body’s centre of mass over its base of support, both at rest and in motion. Good balance underpins stable, controlled performance.
- Centre of gravity (CoG): The point at which the body’s mass is evenly distributed in all directions. Its position shifts with posture and movement.
- Base of support: The area beneath the body that provides stability, usually the space between the feet or other points of contact with the ground.

A person remains balanced as long as the line of gravity from their CoG falls within their base of support.
Kinetics helps us analyse how musicians generate, control, and coordinate movement, and it informs both injury prevention and performance.
Motion capture¶
There are many ways to study human body motion. Many people picture “motion capture” as suits, markers, or sensors, but the term can cover any method that systematically records movement, qualitative as well as quantitative. In practice the two are often combined. Researchers may use both video and sensors, and an analysis may mix interpretive and numerical work. For clarity, this course treats qualitative and quantitative methods separately, while recognising that mixed-method approaches are common.
Qualitative approaches¶
Qualitative motion analysis works through observation, reflection, and descriptive frameworks rather than numerical measurement.
- Introspection: Reflecting on one’s own movement experience. Musicians and researchers may assess their own performance, notice sensations of effort or discomfort, or describe how a movement feels during music-making.
- Observation: Systematically watching others move—live or on video—and annotating their motion: posture, gesture, timing, expressiveness. Some may not count observation as “proper” motion capture, but it is a structured, repeatable way to document movement without specialised technology.
Observation-based methods are widely used in clinical, educational, sports, and artistic settings. Video recordings allow repeated viewing, slow-motion analysis, and collaborative review, which makes subtle details and patterns easier to spot.
Music researchers have drawn on the qualitative methods developed by the dancer and choreographer Rudolf Laban (1879–1958), who created two influential systems in the early to mid-20th century:
- Labanotation: A symbolic notation system for recording and analysing human movement, especially in dance. It uses standardised symbols for body parts, directions, levels, and timing, allowing detailed documentation of movement sequences.
- Laban Movement Analysis (LMA): A framework for describing the qualitative side of movement. LMA covers four components: body (what moves), effort (how it moves), shape (the form the body takes), and space (where it moves). The “effort” component is especially relevant to music, describing motion in terms of space (direct/indirect), time (quick/sustained), weight (strong/light), and flow (bound/free).
Qualitative approaches capture the expressive, communicative, and contextual sides of movement. They often complement quantitative methods by reaching aspects that are hard to measure numerically, such as emotion, intention, and style.
Quantitative approaches¶
Quantitative methods rely on numerical representations of motion. Video can serve as a quantitative tool too, if features are extracted and measured rather than just observed. The work often involves plotting measurements and applying statistical or machine learning techniques. Two main types of motion capture stand out: camera-based and sensor-based. Each has strengths and limits, and the choice between them depends on the research context, the precision needed, and practical constraints.
Camera-based motion capture¶
This approach uses cameras—either standard video cameras or specialised systems (infrared, stereo, or depth)—to record and analyse movement. Markers may be placed on the body to track specific points, or markerless systems can use computer vision to estimate body positions. Camera-based systems are widely used in biomechanics, animation, and music research because they can capture detailed, full-body motion in three dimensions. They do, however, often need controlled environments and careful calibration, and they can be sensitive to lighting and occlusion.

An example of an infrared, marker-based motion capture system, allowing for precise measurements of the body.
At the University of Oslo, camera-based motion capture is available at RITMO and the Department of Musicology. The video below, from the University of Oslo’s online course Music Moves, shows one of the labs:
(If the video is unavailable, try the archived page.)
Markerless motion capture¶
Marker-based systems remain the reference standard in many labs, but attaching reflective markers takes time and can interfere with natural movement, especially in dance, ensemble playing, or performances in front of an audience. Markerless (or marker-free) motion capture estimates body pose from ordinary video (or depth images) using computer vision and machine learning. A single camera can recover a 2D skeleton; multi-camera rigs can reconstruct full 3D body models. The result is often less precise than professional marker systems, but markerless tools lower the barrier to recording movement in classrooms, concerts, and everyday settings.
Several open libraries and research pipelines illustrate what is possible today. The list below is not exhaustive, but it shows the main families you are likely to encounter:
Going deeper: markerless motion-capture tools
- MediaPipe — Google’s cross-platform framework for real-time 2D pose, hand, and face tracking from a webcam or phone. Widely used in teaching and prototyping because it runs on consumer hardware with minimal setup.
- OpenPose — A landmark system from Carnegie Mellon for multi-person 2D body, hand, and foot keypoints. Influential in research and still a common baseline for video-based motion analysis.
- FreeMoCap — An open-source, GUI-based pipeline for research-oriented 3D markerless capture using one or more ordinary webcams (or imported video). It calibrates a capture volume, processes recordings, and exports kinematic data for analysis or animation. It is aimed at labs and classrooms that want multi-camera 3D without marker suits (documentation).
- MAMMA — Markerless Accurate Multi-person Motion Acquisition, a research pipeline from the Max Planck Institute for Intelligent Systems that recovers detailed 3D body models (SMPL-X) from multi-view video, including two-person interactions Cuevas Velasquez et al., 2026. Relevant when studying joint musical action—duets, ensemble choreography, or conductor–orchestra coupling—where tracking more than one person in 3D matters.
Other camera-based options include depth sensors (for example Azure Kinect body tracking), smartphone pipelines such as OpenCap, and commercial markerless services, though the tools above are among the most accessible open starting points.
Markerless capture still has practical limits. Occlusion (one limb hiding another), loose clothing, uneven lighting, and limited finger detail can reduce accuracy compared with marker-based or sensor-based systems. For many music studies—gesture timing, gross body sway, dance metre, or air performance—they nonetheless offer a practical entry point before investing in lab infrastructure.
Sensor-based motion capture¶
This method relies on wearable sensors attached directly to the body. Common types include inertial measurement units (IMUs), accelerometers, gyroscopes, magnetometers, and sometimes physiological sensors such as EMG for muscle activity. Sensor-based systems are generally portable and less dependent on the surroundings, which suits field studies or situations where cameras are impractical. They can give precise data on joint angles, acceleration, and orientation, but they need careful placement and calibration and can suffer from sensor drift or interference.

An example of a sensor-based motion capture suit used in a performance with the Stavanger Symphony Orchestra in 2023.
Video visualisation¶
Video visualisation sits between qualitative and quantitative approaches. It starts from ordinary video recordings but aims to extract relevant features from the video stream. Instead of just watching and describing movement, it uses computational tools to analyse and represent motion visually.
For example, software can track the position of specific body parts or objects frame by frame and plot movement trajectories, velocity, or acceleration over time. Other techniques include motion history images, which overlay several frames to highlight areas of frequent movement, and heatmaps that show where most activity occurs. These views can reveal patterns, timing, and coordination in performance or listening that are hard to catch by observation alone. The video below, also from the Music Moves course, introduces video analysis of music-related movement:
(If the video is unavailable, try the archived page.)
Video visualisation is useful for spotting subtle or complex movement features, comparing performances, and communicating findings. It also brings subjective interpretation and objective measurement together, which makes it valuable in interdisciplinary research on music-related movement.
Try it live: VideoViz¶
VideoViz brings these techniques into the browser. It turns your webcam into two videograms (horizontal and vertical collapses of the image over time), a self-similarity matrix that highlights repeated movement, and an optional motion view based on frame differencing. It builds on the Musical Gestures Toolbox for Python, which offers the same analyses for recorded video, and whose documentation is the place to start if you want to run them yourself (pip install musicalgestures).
Demo: periodic vertical acceleration at ~120 BPM¶
The demo below uses a sine wave as a stand-in for the vertical bounce of a body moving to a beat. Its frequency analysis shows a peak near 2 Hz, which corresponds to 120 beats per minute.
Source
import numpy as np
import matplotlib.pyplot as plt
from scipy.fft import rfft, rfftfreq
sr_mocap = 200
dur = 8.0
t = np.linspace(0.0, dur, int(sr_mocap * dur), endpoint=False)
bpm = 120.0
fz = bpm / 60.0
rng = np.random.default_rng(3)
acc_z = np.sin(2 * np.pi * fz * t) + 0.08 * rng.standard_normal(len(t))
fk = rfftfreq(len(t), d=1.0 / sr_mocap)
spec = np.abs(rfft(acc_z))
fig, ax = plt.subplots(figsize=(10, 3))
ax.plot(fk[:120], spec[:120])
ax.axvline(fz, color="r", linestyle="--", label=f"{bpm:.0f} BPM ({fz:.2f} Hz)")
ax.set_xlabel("Frequency (Hz)")
ax.set_ylabel("|FFT|")
ax.set_title("Dominant periodicity in a toy bounce signal")
ax.legend()
plt.tight_layout()
plt.show()

Chapter summary¶
This chapter linked embodied and 4E perspectives to measurable movement. Performer gestures, biomechanics, motion capture, joint action, and tactile channels all show how musical meaning emerges through bodies, in social and technological contexts, not through the ears alone.
Questions¶
- How do sound-producing, sound-facilitating, communicative, and accompanimental motions differ in performance research?
- In what ways does a 4E perspective challenge purely “mental” or “auditory-only” accounts of music cognition?
- How do kinematics and kinetics each contribute to describing music-related movement, and what strengths and limits distinguish optical motion capture from wearable sensors in musical studies?
- How do joint action, interpersonal synchrony, and tactile or vibrotactile channels extend analyses beyond individual listeners?
- What did the standstill studies reveal about micromotion during silence and music, and why do claims about such small effects depend on the measurement techniques used?
- Leman, M. (2007). Embodied Music Cognition and Mediation Technology. The MIT Press. 10.7551/mitpress/7476.001.0001
- Jensenius, A. R. (2022). Sound Actions: Conceptualizing Musical Instruments. The MIT Press. 10.7551/mitpress/14220.001.0001
- Newen, A., De Bruin, L., & Gallagher, S. (Eds.). (2018). The Oxford Handbook of 4E Cognition. Oxford University Press. 10.1093/oxfordhb/9780198735410.001.0001
- Jensenius, A. R., Zelechowska, A., & Gonzalez Sanchez, V. E. (2017). The Musical Influence on People’s Micromotion when Standing Still in Groups. Proceedings of the 14th Sound and Music Computing Conference, 195–200. https://www.duo.uio.no/handle/10852/56047
- Cuevas Velasquez, H., Yiannakidis, A., Shin, S., Becherini, G., Höschle, M., Tesch, J., Obersat, T., Alexiadis, T., Halilaj, E., & Black, M. J. (2026). MAMMA: Markerless Accurate Multi-person Motion Acquisition. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). https://mamma.is.tue.mpg.de/
- Musical Gestures. (2010). Routledge. 10.4324/9780203863411
- Clarke, E., & Cook, N. (2004). Empirical Musicology. Oxford University Press. 10.1093/acprof:oso/9780195167498.001.0001