Word-vector explorer
Before transformers, language models learned a vector for every word. Vectors are how every model in this book represents meaning, and this small English set shows what that captures and what it gets wrong. Everything runs in this page after the one-time load of the bundled vectors: nothing you type leaves your browser.
Nearest neighbours
Word
Find nearest neighbours
Type a word and press "Find nearest neighbours".
The vocabulary holds the 7 000 most frequent words in the training corpus, so rare words, names, and most non-English words will not be found.
Analogy: A minus B plus C
Compute A minus B plus C
Enter three words and press "Compute A minus B plus C".
The five closest words to the result, other than A, B, and C themselves. The classic example is king minus man plus woman, which lands near queen.
Map of the vocabulary
Axis start word
Axis end word
Set horizontal axis from these two words
Reset to random projection
Horizontal axis: a fixed random projection.
Hover over a point, or focus the map and press the left/right arrow keys, to see its word.
The vertical axis always comes from the fixed random projection. Setting the axis words replaces only the horizontal axis with the direction from the first word to the second, so words further along that direction score higher on whatever the two words differ by. Try man to woman, then look at where nurse and engineer land.