Embeddings Explained: How AI Understands Meaning

The problem: computers don't understand words

To a computer, the words "car" and "automobile" are just different sequences of characters. Traditional keyword search treats them as unrelated: search for "car insurance" and a page about "automobile coverage" won't match, even though it's exactly what you want.

What we need is a way to represent text so that things with similar meaning look similar to the computer. That representation is called an embedding.

The intuition: a map of meaning

Imagine placing every word on a map, so that words with related meanings sit close together. Fruits cluster in one region, vehicles in another, emotions in a third. On this map, "car" and "automobile" would be almost on top of each other, "truck" nearby, and "banana" far away.

Any position on a map can be described by coordinates — say, (latitude, longitude). An embedding is exactly that: a list of coordinates that describes where a piece of text sits on a map of meaning.

The twist is that two dimensions aren't nearly enough to capture all the ways meanings can relate. Real embeddings use hundreds or thousands of dimensions:

"The cat sat on the mat"  →  [0.021, -0.113, 0.087, 0.004, ..., -0.052]
                               (e.g. 384, 768, 1536 or 3072 numbers)

You can't visualize 768 dimensions, but the math works the same as on a 2D map: close points mean similar meaning.

Where the numbers come from

Nobody hand-designs these coordinates. They're learned by a neural network called an embedding model, trained on large amounts of text.

Early approaches like word2vec (2013) learned one vector per word using a simple insight: words that appear in similar contexts have similar meanings. "Coffee" and "tea" both show up near "cup", "drink" and "hot", so they end up with similar vectors.

Modern embedding models are transformers (the same architecture behind large language models) and they embed whole sentences, paragraphs or documents, not just single words. They're commonly trained with contrastive learning: the model is shown pairs of texts that belong together (a question and its answer, a title and its article) and pairs that don't, and it learns to pull matching pairs together and push mismatched ones apart.

Because they read the whole passage, modern models understand context: "bank" in "river bank" and "bank account" gets different representations.

Measuring similarity

Once text is turned into vectors, "how similar are these two texts?" becomes "how close are these two vectors?". There are three common ways to measure it:

Here's cosine similarity in a few lines of Python:

import numpy as np

def cosine_similarity(a, b):
    return np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b))

In practice, most systems normalize vectors to length 1 up front, so similarity is just a fast dot product. Always use the measure the embedding model was trained for — the model's documentation will say.

Try it yourself

With the free, open-source sentence-transformers library you can see embeddings working on your own laptop:

pip install sentence-transformers

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("all-MiniLM-L6-v2")   # small, fast, runs on CPU

sentences = [
    "How do I reset my password?",
    "I forgot my login credentials.",
    "What's the weather like tomorrow?",
]
vectors = model.encode(sentences, normalize_embeddings=True)
print(vectors.shape)          # (3, 384): three sentences, 384 numbers each

similarity = vectors @ vectors.T   # dot products = cosine similarities
print(similarity.round(2))

You'll see that the first two sentences score far higher with each other than either does with the weather question — even though "reset my password" and "forgot my login credentials" share no important words. That's semantic understanding in action.

What embeddings are used for

• Semantic search — find documents by meaning, not exact keywords.
• Retrieval-augmented generation (RAG) — find the passages an LLM needs to answer a question. See Build a RAG App From Scratch.
• Recommendations — "more like this" for articles, products or songs.
• Clustering & topic discovery — group thousands of support tickets or reviews by theme automatically.
• Classification — train a small, cheap classifier on top of embeddings instead of fine-tuning a large model.
• Deduplication — spot near-duplicate questions, records or documents.
• Anomaly detection — flag items far from everything else.

Embeddings aren't limited to text, either. Multimodal models such as CLIP place images and text in the same space, which is how you can search a photo library by typing "dog on a beach".

Choosing an embedding model

There are many embedding models — open-source ones you run yourself and commercial ones available through APIs from the major AI providers. The public MTEB (Massive Text Embedding Benchmark) leaderboard compares them across many tasks and is a good starting point. When choosing, consider:

• Quality on your data — benchmarks are a guide, but test with your own queries and documents.
• Dimensions — more dimensions can capture more nuance but cost more storage and slower search. Some models support shortening vectors with little quality loss.
• Max input length — how much text one embedding can cover.
• Languages — multilingual models can match a question in Hindi to a document in English.
• Hosting — API convenience vs. running locally for privacy and cost.

Common pitfalls

• Mixing models. Vectors from different embedding models live in different, incompatible spaces. If you switch models, you must re-embed everything.
• Embedding text that's too long. One vector for a whole book blurs everything together. Split long documents into chunks.
• Ignoring exact matches. Embeddings can miss precise identifiers like part numbers, error codes or names. Combining them with keyword search ("hybrid search") fixes this.
• Trusting scores as absolute. A similarity of 0.6 means different things for different models. Use scores to rank results, and tune any cut-off thresholds on real data.

Key takeaways

• An embedding is a vector of numbers that represents meaning.
• Similar meanings produce nearby vectors, so meaning can be compared with math.
• Embedding models learn these vectors from data, often with contrastive training.
• Cosine similarity (or dot product on normalized vectors) is the standard comparison.
• Embeddings power semantic search, RAG, recommendations, clustering and more.
• Never mix vectors from different models, and chunk long documents.