Is Cosine-Similarity of Embeddings Really About Similarity?

This paper investigates whether cosine similarity accurately reflects the semantic similarity of learned embeddings, particularly in linear matrix factorization models. The authors demonstrate that the metric can produce arbitrary or non-unique results because certain training objectives allow for the random rescaling of latent dimensions. While some regularization methods yield a unique solution, others leave the final similarity scores dependent on opaque modeling choices rather than the underlying data. These findings suggest that the common practice of applying cosine similarity to high-dimensional vectors may lead to misleading conclusions. Consequently, the researchers advise against the untested use of this metric and suggest alternative normalization or projection techniques to ensure more reliable measurements.