Distance Metrics And Similarity Measures - Similarity Measures

Similarity Measures

Cosine Similarity

Measures the cosine of the angle between two vectors, focusing on orientation rather than magnitude.

Formula:

$$\text{sim}(a,b)=\frac{a\cdot b}{\Vert a\Vert \Vert b\Vert }=\frac{{\sum }_{i=1}^n{a}_i{b}_i}{\sqrt{{\sum }_{i=1}^n{a}_i^2}\sqrt{{\sum }_{i=1}^n{b}_i^2}}$$

Range: -1 to 1

• 1: Vectors point in the same direction (most similar)
• 0: Vectors are orthogonal (independent)
• -1: Vectors point in opposite directions (most dissimilar)

When to use:

• Text analysis and NLP (document similarity)
• Recommendation systems
• When magnitude doesn’t matter, only direction
• High-dimensional sparse data

Cosine Similarity Vs Dot Product

Cosine Similarity:

• Measures the angle between two vectors
• Independent of magnitude
• Normalized (range: -1 to 1)
• Better for comparing relative orientation

Dot Product:

• Measures both angle and magnitude
• Depends on vector lengths
• Not normalized
• Better when magnitude matters

Example:

Vector A: [1, 2, 3]
Vector B: [2, 4, 6]  (2× Vector A)

Dot Product(A, B) = 1×2 + 2×4 + 3×6 = 28
Cosine Similarity(A, B) = 1.0 (same direction)

Even though B is twice as long, they point in the same direction, so cosine similarity is 1.0.

References:

• Cosine similarity vs dot product as distance metrics - Data Science Stack Exchange
• Vector Similarity Explained

Jaccard Similarity

Measures similarity between sets, commonly used for binary or categorical data.

Formula:

$$J(A,B)=\frac{|A\cap B|}{|A\cup B|}$$

Jaccard Distance:

$$d(A,B)=1-J(A,B)$$

Range: 0 to 1

• 0: Sets are completely different
• 1: Sets are identical

When to use:

• Comparing sets of items (user preferences, tags, keywords)
• Binary features (presence/absence)
• Collaborative filtering
• Text similarity (comparing word sets)

Example:

Set A: {apple, banana, orange}
Set B: {banana, orange, grape, kiwi}

Intersection: {banana, orange} → |A ∩ B| = 2
Union: {apple, banana, orange, grape, kiwi} → |A ∪ B| = 5

Jaccard Similarity = 2/5 = 0.4