Zoomable page

One self-contained offline page that zooms from a whole session to a single action.

One self-contained page that zooms from a whole session down to a single action.

The stated aim for this toolbox's dance corpus included "the ability to zoom from the whole session down to a single action, and preparation for manual annotation". What existed was three printed scales over a thirteen-level pyramid that could have supported any scale. This closes that gap, and it is the only interactive thing in the toolbox.

Self-contained, or it is not a deliverable. The page has to work from a folder somebody was emailed, with no server and no network. So the videogram is embedded as an image and the motion envelope and the annotations as numbers, and the page reads nothing at runtime.

How much to embed is a decision with a right answer. Too little and the page cannot resolve the gestures it exists to show; too much and it will not open. embed_budget makes that trade explicit and the page states the resolution it actually achieved, so nobody mistakes a smooth curve for a still moment.

Min and max per bucket, never a mean. The same rule as every other figure here: a brief motion is what an overview exists to find, and a mean is what removes it.

decimate_minmax_pairs

decimate_minmax_pairs(x, n_buckets)

The lowest and highest value in each of n_buckets equal slices.

Both extremes, because a single brief spike is exactly what a zoomed-out view must not lose, and any average removes it.

Parameters:

Name	Type	Description	Default
x		The series.	required
n_buckets	int	How many buckets to reduce it to.	required

Returns:

Name	Type	Description
tuple		(lows, highs). A series shorter than n_buckets is returned unchanged in
		both, since asking for more detail than exists must not invent any.

Source code in musicalgestures/_zoomview.py

def decimate_minmax_pairs(x, n_buckets: int):
    """The lowest and highest value in each of `n_buckets` equal slices.

    Both extremes, because a single brief spike is exactly what a zoomed-out view must not
    lose, and any average removes it.

    Args:
        x: The series.
        n_buckets (int): How many buckets to reduce it to.

    Returns:
        tuple: (lows, highs). A series shorter than `n_buckets` is returned unchanged in
        both, since asking for more detail than exists must not invent any.
    """
    a = np.asarray(x, dtype=float).ravel()
    if len(a) == 0:
        return np.zeros(0), np.zeros(0)
    n = max(1, int(n_buckets))
    if len(a) <= n:
        return a.copy(), a.copy()
    edges: np.ndarray = np.linspace(0, len(a), n + 1).astype(int)
    lo = np.empty(n)
    hi = np.empty(n)
    for i in range(n):
        seg = a[edges[i]:max(edges[i] + 1, edges[i + 1])]
        lo[i], hi[i] = seg.min(), seg.max()
    return lo, hi

embed_budget

embed_budget(duration_s, max_points=8000)

How many points to embed, and what resolution that buys.

Parameters:

Name	Type	Description	Default
duration_s	float	Length of the recording.	required
max_points	int	Ceiling on embedded points. Defaults to 8000, which is a few hundred kilobytes of JSON and about 1.2 s per point on a two-hour recording.	8000

Returns:

Name	Type	Description
dict	dict	n_points and seconds_per_point, the second being what the page must
state	dict	a viewer who zooms past it is looking at interpolation, not data.

Source code in musicalgestures/_zoomview.py

def embed_budget(duration_s: float, max_points: int = 8000) -> dict:
    """How many points to embed, and what resolution that buys.

    Args:
        duration_s (float): Length of the recording.
        max_points (int): Ceiling on embedded points. Defaults to 8000, which is a few
            hundred kilobytes of JSON and about 1.2 s per point on a two-hour recording.

    Returns:
        dict: `n_points` and `seconds_per_point`, the second being what the page must
        state: a viewer who zooms past it is looking at interpolation, not data.
    """
    n = max(1, int(max_points))
    return {"n_points": n, "seconds_per_point": float(duration_s) / n}

zoomable_page

zoomable_page(analysis_dir, duration_s, out, hierarchy=None, max_points=8000, videogram_width=3000, title='session', which='videogram_v', video=None, audio=None, player=None, start_s=0.0)

Write a self-contained HTML page that zooms from the whole session to one action.

Parameters:

Name	Type	Description	Default
analysis_dir		Directory holding the cached pyramid and tracks.json.	required
duration_s	float	Length of the recording.	required
out		Path to write. Everything is embedded; nothing else is needed beside it.	required
hierarchy		A Hierarchy whose levels become tier bands, or None.	None
max_points	int	Ceiling on embedded envelope points.	8000
videogram_width	int	Width in pixels of the embedded strips.	3000
title	str	Shown on the page.	'session'
which	str	Which pyramid to embed when video is not given.	'videogram_v'
video	dict	Named video strips, label to a (rows, time) array --- for example a videogram and a motiongram --- embedded in order, with the page offering a switch when there is more than one. None keeps the cached pyramid as the single strip.	None
audio	optional	Path to an audio (or video) file. When given, the page gains an audio band that switches between a waveform and a log-mel spectrogram, on the same clock as everything else.	None
start_s	float	Where in the session the page begins, in seconds. The page then covers start_s to start_s + duration_s: the cached track is sliced, and hierarchy spans --- given on the session clock --- are clipped to the range and shifted onto the page's own clock, so one section of a long recording can be paged and analysed on its own. Strips passed via video and the audio file are the caller's to slice, since only the caller knows their time base. Defaults to 0.	0.0
player	str	RELATIVE name of a video file to play above the strips --- for example the proxy that ships in the same folder as the page. Clicking the timeline seeks the video, and a playhead runs across every band during playback. A relative name on purpose: the page stays serverless and needs only the folder it ships in, and it degrades to the strips alone when the file is not beside it.	None

Returns:

Name	Type	Description
Path		The file written.

Source code in musicalgestures/_zoomview.py

def zoomable_page(analysis_dir, duration_s: float, out, hierarchy=None,
                  max_points: int = 8000, videogram_width: int = 3000,
                  title: str = "session", which: str = "videogram_v",
                  video=None, audio=None, player=None, start_s: float = 0.0):
    """Write a self-contained HTML page that zooms from the whole session to one action.

    Args:
        analysis_dir: Directory holding the cached pyramid and `tracks.json`.
        duration_s (float): Length of the recording.
        out: Path to write. Everything is embedded; nothing else is needed beside it.
        hierarchy: A `Hierarchy` whose levels become tier bands, or None.
        max_points (int): Ceiling on embedded envelope points.
        videogram_width (int): Width in pixels of the embedded strips.
        title (str): Shown on the page.
        which (str): Which pyramid to embed when `video` is not given.
        video (dict, optional): Named video strips, label to a (rows, time) array ---
            for example a videogram and a motiongram --- embedded in order, with the
            page offering a switch when there is more than one. None keeps the cached
            pyramid as the single strip.
        audio (optional): Path to an audio (or video) file. When given, the page gains
            an audio band that switches between a waveform and a log-mel spectrogram,
            on the same clock as everything else.
        start_s (float): Where in the session the page begins, in seconds. The page
            then covers `start_s` to `start_s + duration_s`: the cached track is
            sliced, and `hierarchy` spans --- given on the session clock --- are
            clipped to the range and shifted onto the page's own clock, so one
            section of a long recording can be paged and analysed on its own.
            Strips passed via `video` and the `audio` file are the caller's to
            slice, since only the caller knows their time base. Defaults to 0.
        player (str, optional): RELATIVE name of a video file to play above the
            strips --- for example the proxy that ships in the same folder as the
            page. Clicking the timeline seeks the video, and a playhead runs across
            every band during playback. A relative name on purpose: the page stays
            serverless and needs only the folder it ships in, and it degrades to the
            strips alone when the file is not beside it.

    Returns:
        Path: The file written.
    """
    from musicalgestures._tracks import read_columns

    budget = embed_budget(duration_s, max_points)
    meta = json.loads((Path(analysis_dir) / "tracks.json").read_text())
    fps, n_frames = float(meta["fps"]), int(meta["frames"])
    qom = np.asarray(np.memmap(Path(analysis_dir) / meta["qom"], dtype=np.float32,
                               mode="r", shape=(n_frames,)), dtype=float)
    a = min(n_frames, int(round(start_s * fps)))
    b = min(n_frames, int(round((start_s + duration_s) * fps)))
    qom = qom[a:b]
    lo, hi = decimate_minmax_pairs(qom, budget["n_points"])
    scale = float(hi.max()) or 1.0

    tiers = []
    if hierarchy is not None:
        for name, spans in hierarchy.levels.items():
            kept = []
            for span in spans:
                s = max(float(span.start), start_s)
                e = min(float(span.end), start_s + duration_s)
                if e > s:
                    kept.append([round(s - start_s, 2), round(e - start_s, 2)])
            tiers.append({"name": name, "spans": kept})

    if video:
        strips = [{"name": str(name), "png": _array_png(arr)}
                  for name, arr in video.items()]
    else:
        strips = [{"name": "videogram",
                   "png": _videogram_png(analysis_dir, duration_s,
                                         videogram_width, which, start_s)}]

    payload = {
        "title": title,
        "duration": duration_s,
        "secondsPerPoint": budget["seconds_per_point"],
        "lo": [round(v / scale, 4) for v in lo.tolist()],
        "hi": [round(v / scale, 4) for v in hi.tolist()],
        "tiers": tiers,
        "video": strips,
        "audio": (_audio_strips(audio, budget["n_points"], videogram_width)
                  if audio is not None else None),
        "player": str(player) if player is not None else None,
    }
    player_markup = ('<div id="pwrap"><video id="v" controls '
                     'preload="metadata"></video></div>' if player else "")
    html = (_TEMPLATE.replace("__DATA__", json.dumps(payload))
                     .replace("__TITLE__", str(title))
                     .replace("__PLAYER__", player_markup))
    out = Path(out)
    out.write_text(html, encoding="utf8")
    return out

mg_zoompage

mg_zoompage(self, target_name=None, overwrite=True, max_points=8000, which='videogram_v')

The zoomable page for this recording, in one call, as a method.

Everything derives from the video itself. The motion track and gram come from extract_tracks, computed on first call and cached beside the video like every other analysis; the audio band comes from the video's own soundtrack when it has one; and the player is the video, referenced by its bare name, so the page works from the folder the two share and needs no server.

Parameters:

Name	Type	Description	Default
target_name	str	Output path. Defaults to "_zoom.html" beside the video.	None
overwrite	bool	Overwrite or auto-increment. Defaults to True.	True
max_points	int	Ceiling on embedded envelope points.	8000
which	str	Which cached gram to embed as the strip.	'videogram_v'

Returns:

Name	Type	Description
Path		The file written.

Source code in musicalgestures/_zoomview.py

def mg_zoompage(self, target_name=None, overwrite: bool = True,
                max_points: int = 8000, which: str = "videogram_v"):
    """The zoomable page for this recording, in one call, as a method.

    Everything derives from the video itself. The motion track and gram come from
    `extract_tracks`, computed on first call and cached beside the video like every
    other analysis; the audio band comes from the video's own soundtrack when it has
    one; and the player is the video, referenced by its bare name, so the page works
    from the folder the two share and needs no server.

    Args:
        target_name (str, optional): Output path. Defaults to "_zoom.html" beside
            the video.
        overwrite (bool, optional): Overwrite or auto-increment. Defaults to True.
        max_points (int): Ceiling on embedded envelope points.
        which (str): Which cached gram to embed as the strip.

    Returns:
        Path: The file written.
    """
    import os
    import subprocess

    from musicalgestures._tracks import extract_tracks
    from musicalgestures._utils import resolve_filename

    of, _ = os.path.splitext(self.filename)
    target = resolve_filename(of, "_zoom.html", target_name, overwrite)

    analysis = Path(self.filename).parent / "analysis" / Path(of).name
    if not (analysis / "tracks.json").exists():
        extract_tracks(self.filename, progress=False)
    meta = json.loads((analysis / "tracks.json").read_text())
    duration = int(meta["frames"]) / float(meta["fps"])

    #: The audio band only when the file carries sound; a silent clip gets a page
    #: without one rather than an error.
    probe = subprocess.run(
        ["ffprobe", "-v", "error", "-select_streams", "a", "-show_entries",
         "stream=codec_type", "-of", "csv=p=0", self.filename],
        capture_output=True, text=True)
    has_audio = "audio" in probe.stdout

    return zoomable_page(
        analysis, duration, target, max_points=max_points, which=which,
        audio=self.filename if has_audio else None,
        player=os.path.basename(self.filename),
        title=Path(of).name)