Reading tip: If you’re already familiar with boring math like Vectors and dot products, feel free to skip ahead. Then if you understand nothing, get back here, sit down, take a deep breath and read from the start.
Vectors
A vector is an ordered list of real numbers. Geometrically, it can also be seen as an arrow pointing in some direction in space. It’s usually written as a lowercase letter, like:

A vector with \(n\) components is \(n\)-dimensional and lives in \(\mathbb{R}^n\). For example, \(u, v \in \mathbb{R}^2\). In practice, \(n\) can be much larger: LLM embeddings often live in \(\mathbb{R}^{768}\) or \(\mathbb{R}^{4096}\), depending on the model.
When working with vectors, we use two basic operations:
Addition: add matching components:

Scaling: multiply a vector by a number. This stretches or shrinks it along the same line (a negative number also flips its direction), which is why such numbers are called scalars:

A vector’s length, also called magnitude or norm, follows from the Pythagorean theorem: square each component, sum them, and take the square root:

Dot product and cosine similarity
The dot product multiplies matching components and sums them. The result is a single number (a scalar), not a vector:
In ML papers, it’s often written as \(u^T v\): transposing \(u\) turns it into a row vector, and a row times a column gives the same sum:
Geometrically, the dot product is tied to the angle \(\theta\) between the two vectors:
The sign of the dot product tells us whether two vectors point the same way:
- \(u \cdot v > 0\): angle under \(90^\circ\), roughly the same direction
- \(u \cdot v = 0\): exactly \(90^\circ\), perpendicular
- \(u \cdot v < 0\): angle over \(90^\circ\), roughly opposite directions

For our vectors, \(\cos\theta = \frac{1}{\sqrt{5}\sqrt{10}} = \frac{1}{\sqrt{50}} \approx 0.14\), so \(\theta \approx 82^\circ\), nearly perpendicular.

This is where cosine similarity comes from: comparing two embeddings by \(\cos\theta\) measures how closely their directions match, ignoring their lengths.
A similarity search by hand
In a RAG system, every document is embedded and stored ahead of time. At query time, the question is embedded too, and the retriever compares its vector with each stored vector using cosine similarity, returning the closest matches. Let’s run that search by hand.
Step 0: Creating the data
In practice, embeddings come from a dedicated embedding model. Google’s EmbeddingGemma 2, released in October 2026, turns each piece of text into a 768-dimensional vector. I don’t know about you, but I’m not doing math in \(\mathbb{R}^{768}\) by hand. Instead, let’s make our own vectors in \(\mathbb{R}^{4}\), with hand-picked numbers to keep the arithmetic clean, just enough to see how the examples relate to each other.
The query, “How do I take care of a cat?”, gets an arbitrary vector:
A related document, “Feeding tips for a new kitten”, should land near the query, so we take \(q\) and add a little noise \(\varepsilon\):
An unrelated document, “How to change your car’s oil”, gets an independent vector:
We expect \(d_1\) to rank first. Let’s check.
Step 1: Dot products
We start by comparing the query \(q\) with each document \(d_1\) and \(d_2\) using the dot product:
The kitten document scores 1.36 against the oil change’s 0.16, which is a clear lead. But the dot product also grows with the vector’s lengths, so on its own it doesn’t tell us how closely the directions match. For that, we need the lengths.
Step 2: Norms
First, we find the length of the query vector \(q\):
Then the lengths of the two stored documents:
All three vectors are about the same length, a little over 1. Keep that in mind, we will discuss more about this later.
Step 3: Cosine similarity
Now we have enough to calculate the cosine similarity and find the angle between \(q\) and each document vector. For the kitten document \(d_1\):
A cosine of 0.989 is very close to 1, the score for two vectors pointing in exactly the same direction. Taking the inverse cosine gives the angle between them: \(\theta = \arccos(0.989) \approx 8^\circ\).
For the oil-change document:
This is close to 0, which means the vectors are nearly perpendicular: \(\theta = \arccos(0.136) \approx 82^\circ\). The two vectors have almost nothing in common.
So the retriever returns \(d_1\), the kitten document, just as we designed it to when we built \(d_1\) from \(q\) plus a little noise.
What if the lengths differ?
In Step 2, we saw that all three vectors are about the same length. That’s why we see the dot product and cosine similarity agree so far.
Real embeddings aren’t always that tidy. Vectors from different models or model versions can sit at different scales, and a shortened vector that isn’t normalized again (see EmbeddingGemma 2’s model card) ends up shorter than the rest.
To see what a length difference does to the ranking, let’s exaggerate it: take the oil-change vector and make it 10 times longer, without changing its direction:
Its dot product with the query:
That’s exactly \(10 \times (q \cdot d_2)\), and it now beats the kitten document’s 1.36. If we ranked by the raw dot product alone, a question about cats would return an article about oil changes, purely because its vector is longer.
Cosine similarity doesn’t have this problem. Stretching \(d_2\) by 10 multiplies both its dot product with \(q\) and its length by 10, and since cosine similarity divides one by the other, the 10 cancels:
The same is true for any stretch, because a vector’s length always appears in both the numerator and the denominator. Cosine similarity scores only the direction a vector points, so no amount of stretching can push \(d_3\) past \(d_1\).
Make it faster
Cosine similarity works, but look at what it costs: for every single comparison, we compute two lengths and divide. That’s a lot of math! And the stored document’s length never changes. So why not deal with the lengths once, up front, and be done with them?
The trick is to rescale every vector to length 1. Dividing a vector by its own length gives a unit vector: it points the same way, but its length is exactly 1. Usually written as a lower case letter with a hat on top.
For the query, divide each component by \(\|q\| \approx 1.122\):
Same thing for the kitten document, this time dividing by \(\|d_1\| \approx 1.225\):
Now, remember the denominator of cosine similarity? It’s the two lengths multiplied together, which is now just \(1 \times 1 = 1\). Dividing by 1 does nothing, so the plain dot product is the cosine similarity. Don’t trust me? Let’s check:
There it is: 0.989, the same number we got in Step 3, without computing a single length or dividing anything at search time.
This isn’t just a trick to make hand calculations easier. In fact, you will see it on production everywhere. OpenAI’s embeddings already come out with length 1, so a plain dot product gives you the cosine similarity for free. Qdrant goes the other way around: pick cosine similarity for a collection, and it normalizes every vector the moment you upload it, then just runs dot products at search time.
Results
| Document | Dot product | Cosine similarity | Angle |
|---|---|---|---|
| \(d_1\): Feeding tips for a new kitten | 1.36 | 0.989 | \(\approx 8^\circ\) |
| \(d_2\): How to change your car’s oil | 0.16 | 0.136 | \(\approx 82^\circ\) |
| \(d_3\): same as \(d_2\), but \(10\times\) longer | 1.60 | 0.136 | \(\approx 82^\circ\) |

The dot product mixes direction and length, cosine similarity measures direction alone. That’s why RAG retrieval uses cosine similarity, or equivalently, the dot product on normalized vectors.