Cosine Similarity vs Dot Product for Embeddings
Introduction
Every semantic search and every RAG systemClick to read tutorial: RAG Architecture Explained: Chunking, Embeddings, Retrieval ends with the same question: how similar are these two embeddings? Three measures are in common use - the dot product, cosine similarity and Euclidean distance - and vector stores ask you to choose one when you create an index. Choose wrongly, and a long, rambling document can outrank the short one that answers the question. This post explains cosine similarity vs dot product for embeddings with diagrams: what each measure computes, why vector length makes the difference, how normalization makes all three agree - shown with a Java program - and which one to use.(Note - What embeddings are is explained in LLM tokens, context windows, embeddings and hallucinationsClick to read tutorial: LLM Tokens, Context Windows, Embeddings and Hallucinations.)
Three ways to compare two vectors
For two embeddings a and b, each a list of n numbers -- The dot product multiplies the numbers pairwise and adds them up:
a · b = a1b1 + a2b2 + ... + anbn. Higher means more similar. - Cosine similarity divides the dot product by the lengths of the two vectors:
cos θ = a · b / (|a| |b|). It is the cosine of the angle between them - 1 for the same direction, 0 for unrelated (at right angles) and -1 for opposite. Higher means more similar. - Euclidean distance is the straight-line distance between the two points:
|a - b|. Lower means more similar.
|a| = √(a · a). The diagram shows what each measure looks at -
Direction carries the meaning; length can distort the dot product.
a · b = |a| |b| cos θ: the dot product is cosine similarity multiplied by both lengths. So the only question is whether vector length should count. (The dot product is also the core of attention in the TransformerClick to read tutorial: Attention Is All You Need: The Transformer Paper Explained, where it scores how well a query matches each key.)
Length is the difference
In an embedding, the direction carries the meaning. The length may carry something else - with some models it grows with the length of the text or with how common its words are - or nothing useful at all. The program below compares a query with three documents using all three measures, first with the raw vectors and then with every vector normalized - divided by its length, so that all lengths are 1.Dot product, cosine similarity and Euclidean distance, before and after normalization
//SimilarityMetrics.java
import java.util.*;
public class SimilarityMetrics {
static double dot(double[] a, double[] b) {
double s = 0;
for (int i = 0; i < a.length; i++) s += a[i] * b[i];
return s;
}
static double length(double[] a) {
return Math.sqrt(dot(a, a));
}
static double cosine(double[] a, double[] b) {
return dot(a, b) / (length(a) * length(b));
}
static double euclidean(double[] a, double[] b) {
double s = 0;
for (int i = 0; i < a.length; i++) s += (a[i] - b[i]) * (a[i] - b[i]);
return Math.sqrt(s);
}
static double[] normalize(double[] a) {
double len = length(a);
return Arrays.stream(a).map(x -> x / len).toArray();
}
static void rank(String title, double[] q, Map<String, double[]> docs) {
System.out.println(title);
System.out.printf(" %-34s %8s %8s %10s%n", "document", "dot", "cosine", "euclidean");
docs.forEach((name, d) -> System.out.printf(" %-34s %8.3f %8.3f %10.3f%n", name, dot(q, d), cosine(q, d), euclidean(q, d)));
System.out.println(" best by dot: " + best(docs, d -> dot(q, d), true)
+ " | by cosine: " + best(docs, d -> cosine(q, d), true)
+ " | by euclidean: " + best(docs, d -> euclidean(q, d), false));
System.out.println();
}
static String best(Map<String, double[]> docs, java.util.function.ToDoubleFunction<double[]> score, boolean highest) {
return docs.entrySet().stream()
.max(Comparator.comparingDouble(e -> highest ? score.applyAsDouble(e.getValue()) : -score.applyAsDouble(e.getValue())))
.get().getKey().split(" ")[0];
}
public static void main(String[] args) {
// 3-number "embeddings" so the arithmetic can be followed; real ones have hundreds or thousands of numbers
double[] query = {1.0, 1.0, 0.0};
Map<String, double[]> docs = new LinkedHashMap<>();
docs.put("A (short, on topic)", new double[] {0.9, 1.0, 0.1});
docs.put("B (long, partly on topic)", new double[] {3.0, 1.0, 2.5});
docs.put("C (short, off topic)", new double[] {0.2, 0.1, 1.0});
docs.forEach((n, d) -> System.out.printf("length of %-26s %.3f%n", n, length(d)));
System.out.println();
rank("Raw vectors:", query, docs);
Map<String, double[]> unit = new LinkedHashMap<>();
docs.forEach((n, d) -> unit.put(n, normalize(d)));
rank("Normalized to length 1:", normalize(query), unit);
}
}
length of A (short, on topic) 1.349 length of B (long, partly on topic) 4.031 length of C (short, off topic) 1.025 Raw vectors: document dot cosine euclidean A (short, on topic) 1.900 0.996 0.141 B (long, partly on topic) 4.000 0.702 3.202 C (short, off topic) 0.300 0.207 1.565 best by dot: B | by cosine: A | by euclidean: A Normalized to length 1: document dot cosine euclidean A (short, on topic) 0.996 0.996 0.091 B (long, partly on topic) 0.702 0.702 0.772 C (short, off topic) 0.207 0.207 1.259 best by dot: A | by cosine: A | by euclidean: A
Explanation of the code
- Document A points almost exactly like the query, with a cosine of 0.996. Document B points quite differently (cosine 0.702) but is three times as long as A.
- With raw vectors, the dot product ranked B first, with 4.000 against A's 1.900 - only because of B's length. Cosine and Euclidean distance both chose A.
- After normalization, the dot product and the cosine are identical for every document, and all three measures choose A. For vectors of length 1,
|a - b|² = 2 - 2 cos θ, so a higher cosine always means a smaller distance: the three measures give the same ranking. - The off-topic document C was last under every measure, with and without normalization.
After normalization, the dot product can no longer be swayed by length.
Which measure to use
Use the measure the embedding model was trained with - its documentation states it. The model learned to place similar texts close by that measure, so another one can rank results less well. In practice -- Many embedding models - descendants of BERTClick to read tutorial: BERT Paper Explained: Bidirectional Transformers such as Sentence-BERT, and the embedding models of the API providers - return normalized vectors - OpenAI's embedding models, for example, return vectors of length 1. Then all three measures give the same ranking, and the dot product is the best choice, because it skips the division and is the cheapest to compute.
- If the vectors are not normalized and the model was trained with cosine similarity, use cosine - or normalize the vectors once, when they are stored, and use the dot product from then on. Normalizing at storage time is the common trick for speed.
- Some models are trained for the raw dot product, so that the length carries information on purpose. Dense Passage Retrieval, the retriever of the RAG paperClick to read tutorial: RAG Paper Explained: Retrieval-Augmented Generation, scores passages by the inner product. For such models, do not normalize.
- Euclidean distance is the natural choice for some clustering algorithms such as k-means; for normalized vectors it ranks exactly like cosine.
<=> (cosine distance), <#> (negative inner product) and <-> (Euclidean distance), or a metric setting chosen when a collection or index is created, as in Qdrant and Milvus. The index is built for that measure, so it must match the queries - see the vector store comparisonClick to read tutorial: pgvector vs Chroma vs Qdrant vs Milvus vs Redis Compared.Two pitfalls to avoid: cosine distance in many libraries is
1 - cosine similarity, so lower is better, the opposite of similarity; and a similarity threshold such as "at least 0.8" means something different for every embedding model, so set thresholds by testing with your own model and data.
Summary
In this post we compared the three measures used to compare embeddings. We saw that the dot product multiplies and adds the numbers of two vectors, cosine similarity divides that by both lengths and so compares only directions, and Euclidean distance measures the gap between the points. In Java, a long document won under the dot product with raw vectors although it pointed in a different direction, while after normalization all three measures agreed. Lastly, we saw how to choose: follow the embedding model, prefer the dot product on normalized vectors, and match the vector store's metric to the model. In the next post, we will look at the maths used to train the models that produce these vectors, in softmax, cross-entropy and KL divergenceClick to read tutorial: Softmax, Cross-Entropy and KL Divergence Explained.Continue Reading: AI and LLM Papers Explained
Softmax, Cross-Entropy and KL Divergence ExplainedClick to read tutorial: Softmax, Cross-Entropy and KL Divergence Explained Tokenization Explained: Byte Pair Encoding in JavaClick to read tutorial: Tokenization Explained: Byte Pair Encoding in Java Attention Is All You Need: The Transformer Paper ExplainedClick to read tutorial: Attention Is All You Need: The Transformer Paper Explained BERT Paper Explained: Bidirectional TransformersClick to read tutorial: BERT Paper Explained: Bidirectional Transformers Scaling Laws for Language Models (and Chinchilla) ExplainedClick to read tutorial: Scaling Laws for Language Models (and Chinchilla) Explained GPT-3 Paper Explained: Language Models Are Few-Shot LearnersClick to read tutorial: GPT-3 Paper Explained: Language Models Are Few-Shot Learners RAG Paper Explained: Retrieval-Augmented GenerationClick to read tutorial: RAG Paper Explained: Retrieval-Augmented Generation LoRA Paper Explained: Low-Rank Adaptation of LLMsClick to read tutorial: LoRA Paper Explained: Low-Rank Adaptation of LLMs Chain-of-Thought Prompting Paper ExplainedClick to read tutorial: Chain-of-Thought Prompting Paper Explained
Softmax, Cross-Entropy and KL Divergence ExplainedClick to read tutorial: Softmax, Cross-Entropy and KL Divergence Explained Tokenization Explained: Byte Pair Encoding in JavaClick to read tutorial: Tokenization Explained: Byte Pair Encoding in Java Attention Is All You Need: The Transformer Paper ExplainedClick to read tutorial: Attention Is All You Need: The Transformer Paper Explained BERT Paper Explained: Bidirectional TransformersClick to read tutorial: BERT Paper Explained: Bidirectional Transformers Scaling Laws for Language Models (and Chinchilla) ExplainedClick to read tutorial: Scaling Laws for Language Models (and Chinchilla) Explained GPT-3 Paper Explained: Language Models Are Few-Shot LearnersClick to read tutorial: GPT-3 Paper Explained: Language Models Are Few-Shot Learners RAG Paper Explained: Retrieval-Augmented GenerationClick to read tutorial: RAG Paper Explained: Retrieval-Augmented Generation LoRA Paper Explained: Low-Rank Adaptation of LLMsClick to read tutorial: LoRA Paper Explained: Low-Rank Adaptation of LLMs Chain-of-Thought Prompting Paper ExplainedClick to read tutorial: Chain-of-Thought Prompting Paper Explained