Embeddings
Also known as: vector embeddings, text embeddings, embedding vectors, embedding
Embeddings are lists of numbers, called vectors, that an AI model produces to represent the meaning of a piece of content such as a word, sentence, image, or document.
Content with similar meaning gets similar numbers, so a computer can find related items by measuring distance between vectors. While embeddings sound abstract, they quietly power semantic search, recommendations, duplicate detection, and the retrieval step behind most AI chatbots that answer from company documents.
Embeddings, explained at three levels
At home
Imagine a giant map where every song, photo, or sentence gets a spot, and things with similar meaning are placed near each other. An embedding is the set of coordinates for one item on that map, except the map has hundreds of directions instead of two. When an app shows you 'more like this,' it looks up your item's spot and returns its nearest neighbours.
At work
An embedding is a list of numbers that an AI model produces to capture what a piece of content means. Because similar meanings get similar numbers, software can search by meaning instead of exact keywords, so a search for 'vacation policy' finds a document titled 'time off rules.' For a business, that means staff and customers find answers even when they don't know the official wording.
At home
An embedding model is a neural network that reads a piece of text, or an image, and outputs a fixed-length list of numbers, often 384 to 3,072 of them. Each number is one direction in a 'meaning space' the model learned during training. To compare two items, software measures the angle between their lists, called cosine similarity. That is how your photo app can match the words 'beach at sunset' to pictures you never tagged.
At work
In a business search system, documents are first split into chunks of a few hundred words, and each chunk is embedded once and stored in a vector database. When a user asks a question, the question is embedded with the same model and the closest chunks are retrieved. This is the retrieval step in retrieval-augmented generation (RAG). Common models include OpenAI's text-embedding-3, Cohere Embed, and open models such as BGE, and chunk size and language coverage strongly affect results.
At home
Embedding models learn meaning from patterns in billions of sentences, so they also absorb the associations in that text. Early word2vec vectors placed 'nurse' closer to 'woman' and 'engineer' closer to 'man,' because that is how the training text used those words. Researchers can measure and reduce such bias, but not remove it entirely. While the numbers look objective, they are a compressed copy of human writing. In my opinion, that's a reminder that 'mathematical' doesn't mean 'neutral.'
At work
Embedding size trades quality against storage: a million chunks at 3,072 dimensions in 32-bit numbers take about 12 GB before indexing. Vectors from different models are incompatible, so switching models means re-embedding the whole collection. Generic models can underperform on legal or medical language, and public benchmarks such as MTEB only approximate your data. Research has also shown embeddings can be partly inverted back to the original text, so I'd protect them with the same access controls as the source documents.
Real-world examples
At home
Jerome searched his photo app for 'beach at sunset' and it found pictures he had never tagged. The app had created embeddings for every photo and matched them to the meaning of his words.
At work
Sophie runs IT at a Toronto law firm. She set up an internal search tool using embeddings, so lawyers could find past memos by describing the issue rather than guessing exact keywords.
How it compares
| feature | Embeddings | Keyword search |
|---|---|---|
| Matches on | Meaning | Exact words |
| Synonyms | Handled naturally | Must be added manually |
| Exact codes and names | Can miss them | Strong |
| Explainability | Harder | Easy |
Embeddings find meaning; keywords find exact terms. Many systems combine both.
For analysts
When a team proposes AI search, I'd ask which embedding model they're using and how they'll test retrieval quality. Bad retrieval is the most common reason company chatbots give wrong answers.
History & origin
The idea that a word's meaning comes from the company it keeps dates to linguists such as J.R. Firth in the 1950s. Yoshua Bengio's team at the Université de Montréal used learned word vectors in a neural language model in 2003. Tomas Mikolov and colleagues at Google released word2vec in 2013, which famously showed that 'king' minus 'man' plus 'woman' landed near 'queen.' Stanford's GloVe followed in 2014. Transformer models such as Google's BERT in 2018 produced embeddings that changed with context, and OpenAI's CLIP in 2021 placed images and text in the same space. Embedding APIs from OpenAI, Cohere, Google, and others then became standard building blocks. In my view, embeddings are the most important AI idea most people have never heard of.