word2vec turns words into vectors that capture meaning
Google's Mikolov team learns 'king minus man plus woman equals queen' from raw text alone
Quick facts
- Researchers
- Tomas Mikolov, Kai Chen, Greg Corrado, Jeffrey Dean
- Institution
- Training data
- 1.6 billion words
What happened
Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean at Google published 'Efficient Estimation of Word Representations in Vector Space' on arXiv, introducing what became known as word2vec. The method trained a shallow neural network on the simple task of predicting a word from its surrounding context, or vice versa, across a text corpus of 1.6 billion words. Once trained, each word was represented as a dense vector of a few hundred numbers, and words used in similar contexts ended up with similar vectors, so that arithmetic on the vectors captured relationships, the canonical example being that the vector for 'king' minus 'man' plus 'woman' lands near the vector for 'queen.' The whole training process took under a day on a single CPU, far cheaper than the more complex neural language models it outperformed.
Why it matters
word2vec showed that meaningful, general-purpose representations of language could be learned automatically from raw text with no hand-built grammar or dictionary, establishing word embeddings as a standard building block for the natural-language systems, including later transformers, that followed.
How we know
The original arXiv paper describes the training method and its computational cost directly, and Google's own follow-up NeurIPS paper on distributed representations, by the same team, documents the vector-arithmetic behavior in further detail.
Sources
- Tomas Mikolov, Kai Chen, Greg Corrado, Jeffrey Dean, Google Inc.. Efficient Estimation of Word Representations in Vector Space · Primary source (author-declared)arxiv.org · Cited as a "primary" source (no stronger domain match). · Link is live and its text matches the event's key terms (Jul 2026)
- Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg Corrado, Jeffrey Dean, NeurIPS 2013. Distributed Representations of Words and Phrases and their Compositionality · Primary source (author-declared)proceedings.neurips.cc · Cited as a "primary" source (no stronger domain match).
See something wrong? . Corrections with a source get fixed fastest.
Part of a timelineArtificial Intelligence30 events · From a wartime theory of neurons to machines that write, paint, and fold proteinsView all →