Embeddings
Also known as: vector embeddings, text embeddings, embedding vectors, embedding
Embeddings are lists of numbers, called vectors, that an AI model produces to represent the meaning of a piece of content such as a word, sentence, image, or document.
Content with similar meaning gets similar numbers, so a computer can find related items by measuring distance between vectors. While embeddings sound abstract, they quietly power semantic search, recommendations, duplicate detection, and the retrieval step behind most AI chatbots that answer from company documents.
Embeddings, explained at three levels
At home
Imagine giving every song, photo, or sentence a spot on a giant map, where similar things sit close together. Embeddings are the coordinates on that map, which is how an app finds 'more like this.'
At work
Embeddings let business tools search by meaning instead of exact keywords, so a search for 'vacation policy' can find a document titled 'time off rules.'
At home
An embedding model reads a piece of text and outputs hundreds or thousands of numbers. Comparing two sets of numbers, often with cosine similarity, tells the computer how alike their meanings are.
At work
Teams embed documents once, store the vectors in a vector database, then embed each user query and retrieve the nearest matches. This is the retrieval step in retrieval-augmented generation (RAG). Model choice, chunk size, and language coverage all affect quality.
At home
Embeddings can absorb biases from their training data, linking certain jobs with certain genders, for example. In my opinion, that's a reminder that 'mathematical' doesn't mean 'neutral.'
At work
Considerations include dimension size versus storage cost, re-embedding everything when you switch models, multilingual and multimodal models, domain-specific fine-tuning, and evaluation with retrieval benchmarks. Embeddings of sensitive text can sometimes be partly inverted back to the original, so they should be protected like the source data.
Real-world examples
At home
Jerome searched his photo app for 'beach at sunset' and it found pictures he had never tagged. The app had created embeddings for every photo and matched them to the meaning of his words.
At work
Sophie runs IT at a Toronto law firm. She set up an internal search tool using embeddings, so lawyers could find past memos by describing the issue rather than guessing exact keywords.
How it compares
| feature | Embeddings | Keyword search |
|---|---|---|
| Matches on | Meaning | Exact words |
| Synonyms | Handled naturally | Must be added manually |
| Exact codes and names | Can miss them | Strong |
| Explainability | Harder | Easy |
Embeddings find meaning; keywords find exact terms. Many systems combine both.
For analysts
When a team proposes AI search, I'd ask which embedding model they're using and how they'll test retrieval quality. Bad retrieval is the most common reason company chatbots give wrong answers.
History & origin
The idea that a word's meaning comes from the company it keeps dates to linguists such as J.R. Firth in the 1950s. Yoshua Bengio's team at the Université de Montréal used learned word vectors in a neural language model in 2003. Tomas Mikolov and colleagues at Google released word2vec in 2013, which famously showed that 'king' minus 'man' plus 'woman' landed near 'queen.' Stanford's GloVe followed in 2014. Transformer models such as Google's BERT in 2018 produced embeddings that changed with context, and OpenAI's CLIP in 2021 placed images and text in the same space. Embedding APIs from OpenAI, Cohere, Google, and others then became standard building blocks. In my view, embeddings are the most important AI idea most people have never heard of.