Cosine similarity: measuring the angle between vectors
Cosine similarity measures the angle between two vectors, ignoring how long they are. Two vectors pointing in exactly the same direction have cosine similarity of 1 (perfectly similar). Two vectors pointing in perpendicular directions score 0 (unrelated). Two pointing in opposite directions score -1 (opposite meaning).
Because it ignores the length of the vectors and only looks at their direction, cosine similarity works well for text. A short tweet and a long article about the same topic should be equally similar to a search query, regardless of how many words they contain. Most text embedding models are designed with cosine similarity in mind, and using a different metric with these models typically produces worse search results.
L2 (Euclidean) distance: the straight-line distance
L2 distance, also called Euclidean distance, measures the straight-line distance between two points in the vector space. The closer two vectors are in this straight-line sense, the more similar they are.
Unlike cosine similarity, L2 is sensitive to the length of the vectors. Two vectors pointing in the same direction but one being much longer than the other will have a large L2 distance, even if they represent similar content. L2 distance is appropriate when both the direction and the scale of the vector carry meaningful information, such as in some image embedding spaces. For L2-normalized vectors (all vectors scaled to the same length), L2 distance and cosine similarity produce identical rankings.
Inner product: rewarding both direction and magnitude
Inner product, also called dot product, multiplies the corresponding numbers in two vectors and adds them all up. For vectors that have all been scaled to the same length, inner product gives the same result as cosine similarity. For vectors with different lengths, inner product naturally favors items whose vectors are both pointing in the right direction and are large in magnitude.
This makes inner product useful for recommendation systems, where an item's popularity or quality might be encoded in the length of its vector. More popular or highly-rated items would have larger vectors and naturally score higher. Always check which metric an embedding model was designed for and use the same metric in the database to get the best results.