Video visualiser

A video is a stack of still images, and most of what is interesting about a moving body happens between them. This app takes the image from your webcam and turns it into some of the visualisations used in music and movement research. The motion image keeps only what changed from one frame to the next. Videograms and motiongrams squeeze each frame into a single line and lay the lines side by side, so a few seconds of movement can be seen in one picture. The self-similarity matrix shows when the image returns to something it has shown before, and the plot at the bottom reduces all of it to a few numbers over time. Press Start camera and allow access when the browser asks.

Privacy: the camera image is analysed locally in your browser, drawn to the screen, and immediately discarded. Nothing is recorded, stored, or sent anywhere. The only thing that ever leaves this page is a picture you choose to save with one of the Download buttons, and it goes to your own device.

Video and motion image

Video

Motion image

The camera image is scaled down to 160 × 120 pixels before anything is analysed. In the motion image, each pixel is the difference in brightness between this frame and the previous one, so whatever stands still turns black and whatever moves lights up. Only edges show clearly: a plain shirt moving sideways changes little in its middle.
A camera never delivers two identical frames, even of a still room: the sensor adds a little flicker of noise to every pixel. Differences smaller than the threshold (on a scale of 0–255) are set to zero. Raise it until the motion image is black when you hold still. The gain only brightens what is drawn in the motion image and motiongrams; it does not change the measurements below.
Videograms and motiongrams

Vertical videogram

earliernow
Each frame is averaged across its width into a single column of pixels, and the columns are placed side by side with the newest on the right. Up and down in the picture is the same as up and down in the video, so a hand that rises draws a line that climbs.

Horizontal videogram

newest at the topearlier at the bottom
Here each frame is averaged down its height into a single row, and the newest row is added at the top so that older rows move downwards. Left and right in the picture is the same as left and right in the video, so swaying from side to side draws a wavy line.
A videogram is made from the frames themselves and shows everything, including the furniture. A motiongram is made from the motion image, so the still background disappears and only movement leaves a trace. Both kinds are kept while the camera runs, so you can switch back and forth without losing anything.
Self-similarity matrix
earliernow
Every recent frame is compared with every other one. Both axes are time, running from the top left (earlier) to the bottom right (now), and the colour at a point says how alike the two frames are: yellow is very alike, dark blue is very different. The diagonal compares each frame with itself, so it is always bright. Holding still fills the matrix with one even colour; a repeated movement produces a chequered pattern of squares or stripes parallel to the diagonal, spaced by the period of the repetition. The matrix follows the videogram or motiongram switch above.
Quantity and centroid of motion

Quantity of motion: no data yet

The quantity of motion is the share of pixels in the motion image that changed by more than the threshold, given as a percentage of the whole picture. It is a single number for how much is moving in the whole picture, regardless of where. The vertical scale adapts to the largest value shown.

Centroid of motion: no data yet

The centroid of motion is the centre of gravity of the motion image: the average position of everything that changed, weighted by how much it changed. It is given from 0 (left or top) to 1 (right or bottom). When nothing moves there is no centroid, and the lines break.

Things to try: move one arm slowly through a large arc, then quickly through the same arc, and compare the traces in the motiongrams and the height of the quantity of motion. Hold completely still for a few seconds and watch which displays go quiet and which do not (the videograms keep showing you, the motiongrams do not). Clap at a steady rate in front of your chest and look for the evenly spaced marks in the vertical motiongram and the stripes in the self-similarity matrix. Put on some music and sway to it: the horizontal motiongram and the horizontal centroid of motion turn the sway into a wave that you can compare with the beat. Finally, turn the threshold down to 0 and hold still to see the camera noise that the threshold is there to remove.

Based on VideoViz and the Musical Gestures Toolbox.