sourced story
16 January 2013Primary source · 2 sourcesWell documented

word2vec turns words into vectors that capture meaning

Google's Mikolov team learns 'king minus man plus woman equals queen' from raw text alone

On the timeline · around 16 January 2013 · The Statistical and Machine Learning TurnThe Statistical and Machine Learning TurnTransformers and the Generative AI Waveword2vec turns words into vectors that capture meaning200820102012201420162018

Quick facts

Researchers
Tomas Mikolov, Kai Chen, Greg Corrado, Jeffrey Dean
Institution
Google
Training data
1.6 billion words

What happened

Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean at Google published 'Efficient Estimation of Word Representations in Vector Space' on arXiv, introducing what became known as word2vec. The method trained a shallow neural network on the simple task of predicting a word from its surrounding context, or vice versa, across a text corpus of 1.6 billion words. Once trained, each word was represented as a dense vector of a few hundred numbers, and words used in similar contexts ended up with similar vectors, so that arithmetic on the vectors captured relationships, the canonical example being that the vector for 'king' minus 'man' plus 'woman' lands near the vector for 'queen.' The whole training process took under a day on a single CPU, far cheaper than the more complex neural language models it outperformed.

Why it matters

word2vec showed that meaningful, general-purpose representations of language could be learned automatically from raw text with no hand-built grammar or dictionary, establishing word embeddings as a standard building block for the natural-language systems, including later transformers, that followed.

How we know

The original arXiv paper describes the training method and its computational cost directly, and Google's own follow-up NeurIPS paper on distributed representations, by the same team, documents the vector-arithmetic behavior in further detail.

Sources

See something wrong? . Corrections with a source get fixed fastest.

Part of a timelineArtificial Intelligence30 events · From a wartime theory of neurons to machines that write, paint, and fold proteinsView all →
word2vec turns words into vectors that capture meaning · Artificial Intelligence · SourcedStory