Cosine Similarity vs Dot Product for Embeddings

Introduction

Every semantic search and every RAG systemClick to read tutorial: RAG Architecture Explained: Chunking, Embeddings, Retrieval ends with the same question: how similar are these two embeddings? Three measures are in common use - the dot product, cosine similarity and Euclidean distance - and vector stores ask you to choose one when you create an index. Choose wrongly, and a long, rambling document can outrank the short one that answers the question. This post explains cosine similarity vs dot product for embeddings with diagrams: what each measure computes, why vector length makes the difference, how normalization makes all three agree - shown with a Java program - and which one to use.

(Note - What embeddings are is explained in LLM tokens, context windows, embeddings and hallucinationsClick to read tutorial: LLM Tokens, Context Windows, Embeddings and Hallucinations.)

Three ways to compare two vectors

For two embeddings a and b, each a list of n numbers -
  • The dot product multiplies the numbers pairwise and adds them up: a · b = a1b1 + a2b2 + ... + anbn. Higher means more similar.
  • Cosine similarity divides the dot product by the lengths of the two vectors: cos θ = a · b / (|a| |b|). It is the cosine of the angle between them - 1 for the same direction, 0 for unrelated (at right angles) and -1 for opposite. Higher means more similar.
  • Euclidean distance is the straight-line distance between the two points: |a - b|. Lower means more similar.
The length of a vector is |a| = √(a · a). The diagram shows what each measure looks at -
The query and document A point in almost the same direction; document B is much longer but points in a different direction. Cosine compares only the angle; the dot product also grows with length, so B scores higher; Euclidean distance measures the gap between the tips angle to B query A B (long) drawn in 2 dimensions; real embeddings have hundreds or thousands What each measure sees Cosine only the angle: A points almost exactly like the query, B does not → A wins Dot product angle and both lengths: B is so long that it wins despite its angle Euclidean distance gap between the tips: A's tip is close to the query's, B's is far → A wins
Direction carries the meaning; length can distort the dot product.
The relationship between the first two is a · b = |a| |b| cos θ: the dot product is cosine similarity multiplied by both lengths. So the only question is whether vector length should count. (The dot product is also the core of attention in the TransformerClick to read tutorial: Attention Is All You Need: The Transformer Paper Explained, where it scores how well a query matches each key.)

Length is the difference

In an embedding, the direction carries the meaning. The length may carry something else - with some models it grows with the length of the text or with how common its words are - or nothing useful at all. The program below compares a query with three documents using all three measures, first with the raw vectors and then with every vector normalized - divided by its length, so that all lengths are 1.
Dot product, cosine similarity and Euclidean distance, before and after normalization
//SimilarityMetrics.java
import java.util.*;

public class SimilarityMetrics {

  static double dot(double[] a, double[] b) {
    double s = 0;
    for (int i = 0; i < a.length; i++) s += a[i] * b[i];
    return s;
  }

  static double length(double[] a) {
    return Math.sqrt(dot(a, a));
  }

  static double cosine(double[] a, double[] b) {
    return dot(a, b) / (length(a) * length(b));
  }

  static double euclidean(double[] a, double[] b) {
    double s = 0;
    for (int i = 0; i < a.length; i++) s += (a[i] - b[i]) * (a[i] - b[i]);
    return Math.sqrt(s);
  }

  static double[] normalize(double[] a) {
    double len = length(a);
    return Arrays.stream(a).map(x -> x / len).toArray();
  }

  static void rank(String title, double[] q, Map<String, double[]> docs) {
    System.out.println(title);
    System.out.printf("  %-34s %8s %8s %10s%n", "document", "dot", "cosine", "euclidean");
    docs.forEach((name, d) -> System.out.printf("  %-34s %8.3f %8.3f %10.3f%n", name, dot(q, d), cosine(q, d), euclidean(q, d)));
    System.out.println("  best by dot: " + best(docs, d -> dot(q, d), true)
        + " | by cosine: " + best(docs, d -> cosine(q, d), true)
        + " | by euclidean: " + best(docs, d -> euclidean(q, d), false));
    System.out.println();
  }

  static String best(Map<String, double[]> docs, java.util.function.ToDoubleFunction<double[]> score, boolean highest) {
    return docs.entrySet().stream()
        .max(Comparator.comparingDouble(e -> highest ? score.applyAsDouble(e.getValue()) : -score.applyAsDouble(e.getValue())))
        .get().getKey().split(" ")[0];
  }

  public static void main(String[] args) {
    // 3-number "embeddings" so the arithmetic can be followed; real ones have hundreds or thousands of numbers
    double[] query = {1.0, 1.0, 0.0};
    Map<String, double[]> docs = new LinkedHashMap<>();
    docs.put("A (short, on topic)", new double[] {0.9, 1.0, 0.1});
    docs.put("B (long, partly on topic)", new double[] {3.0, 1.0, 2.5});
    docs.put("C (short, off topic)", new double[] {0.2, 0.1, 1.0});
    docs.forEach((n, d) -> System.out.printf("length of %-26s %.3f%n", n, length(d)));
    System.out.println();

    rank("Raw vectors:", query, docs);

    Map<String, double[]> unit = new LinkedHashMap<>();
    docs.forEach((n, d) -> unit.put(n, normalize(d)));
    rank("Normalized to length 1:", normalize(query), unit);
  }
}
 OUTPUT of the above code
length of A (short, on topic)        1.349
length of B (long, partly on topic)  4.031
length of C (short, off topic)       1.025

Raw vectors:
  document                                dot   cosine  euclidean
  A (short, on topic)                   1.900    0.996      0.141
  B (long, partly on topic)             4.000    0.702      3.202
  C (short, off topic)                  0.300    0.207      1.565
  best by dot: B | by cosine: A | by euclidean: A

Normalized to length 1:
  document                                dot   cosine  euclidean
  A (short, on topic)                   0.996    0.996      0.091
  B (long, partly on topic)             0.702    0.702      0.772
  C (short, off topic)                  0.207    0.207      1.259
  best by dot: A | by cosine: A | by euclidean: A

Explanation of the code

  • Document A points almost exactly like the query, with a cosine of 0.996. Document B points quite differently (cosine 0.702) but is three times as long as A.
  • With raw vectors, the dot product ranked B first, with 4.000 against A's 1.900 - only because of B's length. Cosine and Euclidean distance both chose A.
  • After normalization, the dot product and the cosine are identical for every document, and all three measures choose A. For vectors of length 1, |a - b|² = 2 - 2 cos θ, so a higher cosine always means a smaller distance: the three measures give the same ranking.
  • The off-topic document C was last under every measure, with and without normalization.
The diagram shows why normalization makes the measures agree -
Normalizing divides each vector by its length, so every vector ends on the unit circle; only the angles remain, and dot product, cosine and Euclidean distance then rank documents the same way Raw vectors BAq ÷ length Normalized: all length 1 qAB tips on theunit circle:only anglesremain For unit vectors: dot product = cosine, and |a − b|² = 2 − 2·cosine - all three rank the same way.
After normalization, the dot product can no longer be swayed by length.

Which measure to use

Use the measure the embedding model was trained with - its documentation states it. The model learned to place similar texts close by that measure, so another one can rank results less well. In practice -
  • Many embedding models - descendants of BERTClick to read tutorial: BERT Paper Explained: Bidirectional Transformers such as Sentence-BERT, and the embedding models of the API providers - return normalized vectors - OpenAI's embedding models, for example, return vectors of length 1. Then all three measures give the same ranking, and the dot product is the best choice, because it skips the division and is the cheapest to compute.
  • If the vectors are not normalized and the model was trained with cosine similarity, use cosine - or normalize the vectors once, when they are stored, and use the dot product from then on. Normalizing at storage time is the common trick for speed.
  • Some models are trained for the raw dot product, so that the length carries information on purpose. Dense Passage Retrieval, the retriever of the RAG paperClick to read tutorial: RAG Paper Explained: Retrieval-Augmented Generation, scores passages by the inner product. For such models, do not normalize.
  • Euclidean distance is the natural choice for some clustering algorithms such as k-means; for normalized vectors it ranks exactly like cosine.
Vector stores name the measures in their own way - for example pgvector's operators <=> (cosine distance), <#> (negative inner product) and <-> (Euclidean distance), or a metric setting chosen when a collection or index is created, as in Qdrant and Milvus. The index is built for that measure, so it must match the queries - see the vector store comparisonClick to read tutorial: pgvector vs Chroma vs Qdrant vs Milvus vs Redis Compared.

Two pitfalls to avoid: cosine distance in many libraries is 1 - cosine similarity, so lower is better, the opposite of similarity; and a similarity threshold such as "at least 0.8" means something different for every embedding model, so set thresholds by testing with your own model and data.

Summary

In this post we compared the three measures used to compare embeddings. We saw that the dot product multiplies and adds the numbers of two vectors, cosine similarity divides that by both lengths and so compares only directions, and Euclidean distance measures the gap between the points. In Java, a long document won under the dot product with raw vectors although it pointed in a different direction, while after normalization all three measures agreed. Lastly, we saw how to choose: follow the embedding model, prefer the dot product on normalized vectors, and match the vector store's metric to the model. In the next post, we will look at the maths used to train the models that produce these vectors, in softmax, cross-entropy and KL divergenceClick to read tutorial: Softmax, Cross-Entropy and KL Divergence Explained.