Computers don't compare meaning directly, only numbers. An embedding is what makes that possible: a piece of text converted into a list of numbers (a vector) positioned so that texts with similar meaning end up as nearby points, and texts with different meaning end up far apart.
Learning Objectives
- Explain what an embedding actually is and what it's used for.
- Explain why embeddings enable search by meaning instead of exact keyword matching.
- Explain how similarity between two embeddings gets measured.
Text In, Vector Out
Text
"vacation policy"
Embedding Model
Vector
[0.12, -0.48, 0.91, ...]
The vector itself has no human-readable meaning per number, it's a coordinate in a high-dimensional space (commonly hundreds or thousands of dimensions). What matters isn't any individual number, it's the vector's position relative to other vectors.
Why This Enables Meaning-Based Search
"vacation policy" → close to → "annual leave rules" "vacation policy" → far from → "database schema"
A plain keyword search for "vacation policy" would likely miss a document titled "Annual Leave Rules", there's no shared word at all. An embedding-based search recognizes the two phrases occupy nearby positions in the vector space, because an embedding model trained on huge amounts of text learns that these phrases tend to appear in similar contexts.
Measuring Similarity: Cosine Similarity
The most common way to compare two embeddings is cosine similarity, a measure of the angle between the two vectors rather than the distance between them. Two vectors pointing in almost the same direction score close to 1 (highly similar); vectors pointing in unrelated directions score close to 0.
Vector A
Compare Angle
cosine similarity
Vector B
Similarity Score
0 to 1
A Concrete Walkthrough
- A support document titled "How to request a refund" gets embedded once, when it's added to the system.
- A user asks "can I get my money back for a late order?", a completely different set of words.
- That question gets embedded using the same embedding model.
- Cosine similarity between the question's vector and every stored document's vector is computed.
- The refund document scores highest, despite sharing almost no exact words with the question, and gets returned as the match.
Common Beginner Mistakes
- Mixing embeddings from two different embedding models in the same search index, vectors from different models aren't positioned in a comparable space, even if they're the same size.
- Assuming embedding similarity means factual correctness, it measures topical/semantic closeness, not whether either piece of text is actually true.
- Re-embedding a huge document collection casually, changing embedding models means recomputing every vector, a real, sometimes costly, one-time operation.
- Expecting a single embedding to capture a very long document's meaning well, extremely long text often needs to be broken into smaller pieces first, covered in the chunking lesson.
FAQ
Do embeddings only work for English text?
No, many embedding models are multilingual and can even place semantically equivalent phrases in different languages near each other in the vector space, though quality varies by model and language.
Is a bigger embedding vector always better?
Not automatically. More dimensions can capture more nuance but cost more to store and search, and past a certain point additional dimensions add limited practical benefit for a given task.
Can embeddings compare things other than text?
Yes, images, audio, and other data types can be embedded into vector spaces too, using models trained for that specific data type, and some models even place text and images in a shared, comparable space.
Interview Questions
What is an embedding, in one sentence?
A piece of text (or other data) converted into a list of numbers, a vector, positioned so that semantically similar inputs end up close together in that vector space.
Why can an embedding-based search find a relevant document that shares no exact words with the query?
Because the search compares meaning, via vector position, rather than exact keyword overlap. An embedding model learns to place semantically related phrases near each other regardless of the specific words used.
What does cosine similarity actually measure?
The angle between two vectors, not the distance between them. Vectors pointing in nearly the same direction score close to 1, meaning high similarity, regardless of their individual magnitudes.
Why can't you mix embeddings from two different models in the same search index?
Different embedding models position vectors in their own distinct space, shaped by that model's own training. A vector from one model isn't meaningfully comparable to a vector from a different model, even if both happen to be the same length.
Summary
An embedding converts text into a vector positioned by meaning, not by exact wording, which is what lets a search system find a relevant match even when the query and the document share no common words. Cosine similarity is the standard way to measure how close two embeddings actually are.
What's Next?
The next lesson covers vector databases, the systems built specifically to store large numbers of embeddings and search them quickly.