Co-accentuation¶

Whether motion accents land on sound accents, judged against a chance baseline.

Co-accentuation: do motion accents land on sound accents?

From Serdar and Jensenius, Mixed Method Audio-Video Analyses of Felt Togetherness in a Networked Music-Dance Performance, MOCO '26. Each motion peak is tested for an audio onset within a tolerance; the fraction that coincide is the Global Co-Accentuation Index, and a curve over short windows shows whether synchrony came in bursts or was sustained.

This is the measure for the question underneath a project like this one: not whether motion and sound rise together on average, which a correlation answers, but whether the moments line up.

Why the chance baseline is not optional. The index is a raw fraction, and fractions of coincidence rise with density: sprinkle enough onsets over a recording and every motion peak has one within 150 ms whether or not anything is coordinated. An index of 0.8 means nothing until you know what 0.8 would have been by accident. The null here is a circular shift of the onsets, which preserves how many there are and their internal timing, and destroys only their relationship to the motion. Both the observed index and what the null gives are returned, and a claim rests on the difference rather than on the index alone.

The measure is asymmetric, deliberately. It asks what fraction of MOTION peaks found a sound, not the reverse. A dancer moving twice to one sound is a different thing from a musician playing twice to one movement, and a symmetric measure would hide which happened. Swap the arguments to ask the other question.

CoAccentuation `dataclass` ¶

CoAccentuation(gci, n_peaks, n_matched, expected_gci, z, p, tolerance_s)

The result of a co-accentuation test.

Attributes:

Name	Type	Description
`gci`	`float`	Global Co-Accentuation Index --- the fraction of motion peaks with an onset within the tolerance. NaN when there were no peaks, because "nothing was coordinated" and "nothing was asked" are different answers.
`n_peaks`	`int`	How many motion peaks were tested.
`n_matched`	`int`	How many found an onset.
`expected_gci`	`float`	The mean index over the circular-shift null. What this recording's density buys by accident.
`z`	`float`	How many null standard deviations the observed index sits above the null mean.
`p`	`float`	Fraction of null draws reaching the observed index. This is the number a claim rests on.
`tolerance_s`	`float`	The window used, echoed back so a figure can state it.

co_accentuation ¶

co_accentuation(motion_peaks, audio_onsets, duration_s, tolerance_s=0.15, n_null=200, seed=0)

Test whether motion peaks coincide with audio onsets more than by chance.

Parameters:

Name	Type	Description	Default
`motion_peaks`		Times of motion accents, in seconds.	required
`audio_onsets`		Times of audio onsets, in seconds.	required
`duration_s`	`float`	Length of the recording, needed to wrap the null's shifts.	required
`tolerance_s`	`float`	How close counts as coincident. Defaults to 0.15, the value used in the paper this comes from.	`0.15`
`n_null`	`int`	Circular-shift draws for the null. Defaults to 200.	`200`
`seed`	`int`	For the shifts, so a reported p-value can be reproduced.	`0`

Returns:

Name	Type	Description
`CoAccentuation`	`CoAccentuation`	The index, the null it is judged against, and the p-value.

Source code in musicalgestures/_coaccentuation.py

def co_accentuation(motion_peaks, audio_onsets, duration_s: float,
                    tolerance_s: float = 0.15, n_null: int = 200,
                    seed: int = 0) -> CoAccentuation:
    """Test whether motion peaks coincide with audio onsets more than by chance.

    Args:
        motion_peaks: Times of motion accents, in seconds.
        audio_onsets: Times of audio onsets, in seconds.
        duration_s (float): Length of the recording, needed to wrap the null's shifts.
        tolerance_s (float): How close counts as coincident. Defaults to 0.15, the value
            used in the paper this comes from.
        n_null (int): Circular-shift draws for the null. Defaults to 200.
        seed (int): For the shifts, so a reported p-value can be reproduced.

    Returns:
        CoAccentuation: The index, the null it is judged against, and the p-value.
    """
    peaks = np.sort(np.asarray(motion_peaks, dtype=float).ravel())
    onsets = np.sort(np.asarray(audio_onsets, dtype=float).ravel())
    n = len(peaks)
    if n == 0:
        return CoAccentuation(float("nan"), 0, 0, float("nan"), float("nan"),
                              float("nan"), tolerance_s)

    matched = _matched(peaks, onsets, tolerance_s)
    gci = matched / n

    if len(onsets) == 0 or duration_s <= 0 or n_null <= 0:
        return CoAccentuation(gci, n, matched, float("nan"), float("nan"),
                              float("nan"), tolerance_s)

    rng = np.random.default_rng(seed)
    null = np.empty(n_null)
    for i in range(n_null):
        shifted = np.sort((onsets + rng.uniform(0, duration_s)) % duration_s)
        null[i] = _matched(peaks, shifted, tolerance_s) / n
    mu, sd = float(null.mean()), float(null.std())
    z = (gci - mu) / sd if sd > 0 else float("nan")
    #: (count + 1) / (n + 1): a permutation p is never 0, because one more draw might
    #: have reached it.
    p = float((np.sum(null >= gci) + 1) / (n_null + 1))
    return CoAccentuation(gci, n, matched, mu, z, p, tolerance_s)

co_accentuation_curve ¶

co_accentuation_curve(motion_peaks, audio_onsets, duration_s, window_s=5.0, step_s=1.0, tolerance_s=0.15)

The index over sliding windows, to see whether synchrony was sustained or bursty.