Loss Functions
Contrastive loss trains an embedding model so similar pairs are close together and dissimilar pairs are separated by at least a margin.
Intuition
The loss is not trying to predict a class directly. It shapes the geometry of the representation space. In a Siamese network, two inputs pass through the same encoder, and contrastive loss teaches the encoder whether their embeddings should be near or far.
Common Formula
For a pair of embeddings with distance and label , where means similar and means dissimilar:
The first term pulls similar pairs together. The second term pushes dissimilar pairs apart until they are at least margin away.
Why It Is Needed
Contrastive loss is useful when identity or similarity matters more than fixed class labels. Examples include face verification, duplicate detection, image-text matching, and retrieval systems.
It connects to Contrastive Learning, where the model learns useful representations by comparing positive and negative examples.
Failure Modes
- Bad pair construction gives weak supervision.
- Too few hard negatives makes the model learn trivial separation.
- A margin that is too small gives poor separation; a margin that is too large can make training unstable.
One-line Interview Answer
Contrastive loss learns an embedding space by pulling similar examples together and pushing dissimilar examples apart by a margin.