Average Precision

Average Precision (AP) is a metric that summarizes the precision-recall curve as a single number. It’s particularly useful for:

• Imbalanced classification problems
• Information retrieval systems
• Object detection and localization
• Anomaly Detection

Average Precision is the area under the Precision-Recall curve. It provides a single value that summarizes the model’s performance across all classification thresholds.

$$\text{AP}=\sum _{n=1}^N(R_n-R_{n-1})\times P_n$$

Where:

• $P_n$ = Precision at threshold n
• $R_n$ = Recall at threshold n
• N = Number of thresholds

How It Works

1. Generate predictions with confidence scores
2. Sort predictions by confidence (highest first)
3. For each threshold:
   • Calculate precision and recall
   • Plot the point on Precision-Recall curve
4. Calculate area under the curve

Precision-Recall Curve

The PR curve shows the trade-off between precision and recall at different thresholds:

• X-axis: Recall (how many positives we found)
• Y-axis: Precision (how accurate our positive predictions are)

Ideal curve: Stays at top-right corner (high precision, high recall)

Interpretation

AP Score	Interpretation
1.0	Perfect model (100% precision at all recall levels)
~0.9-1.0	Excellent performance
~0.7-0.9	Good performance
~0.5-0.7	Moderate performance
< 0.5	Poor performance
Baseline	Fraction of positive samples in dataset

Example: If 1% of your data is anomalous, a random model would have AP ≈ 0.01.

AP Vs AUC-ROC

See Average Precision - AP Vs AUC-ROC.

Average Precision in Different Contexts

See Average Precision - Different Contexts.

Example Calculation

Dataset: 10 samples (3 positive, 7 negative)

Model predictions (sorted by confidence):

Rank	True Label	Prediction	TP	FP	Precision	Recall
1	1	1	1	0	1.00	0.33
2	1	1	2	0	1.00	0.67
3	0	1	2	1	0.67	0.67
4	1	1	3	1	0.75	1.00
5	0	1	3	2	0.60	1.00

Calculate AP:

AP = (1.00 - 0) × 1.00 + (0.67 - 0.33) × 1.00 + (1.00 - 0.67) × 0.75
   = 0.33 + 0.34 + 0.25
   = 0.92

Advantages

• Robust to class imbalance: Focuses on positive class performance
• Threshold-independent: Evaluates across all thresholds
• Single metric: Easy to compare models
• Intuitive: Based on familiar precision and recall concepts

Disadvantages

• Less intuitive than individual metrics for non-technical stakeholders
• Doesn’t directly show cost trade-offs between FP and FN
• Sensitive to ranking of predictions (order matters)

Practical Considerations

1. Use with imbalanced data: AP is more informative than accuracy for skewed datasets
2. Compare with baseline: Know the random model AP (fraction of positives)
3. Examine the curve: Don’t just look at the score, visualize the PR curve
4. Consider costs: If FP and FN have different costs, adjust threshold accordingly
5. Use with other metrics: Combine with ML Metrics for complete picture

Implementation Example

from sklearn.metrics import average_precision_score, precision_recall_curve
import matplotlib.pyplot as plt
 
# Calculate AP
ap = average_precision_score(y_true, y_scores)
print(f"Average Precision: {ap:.3f}")
 
# Plot Precision-Recall curve
precision, recall, thresholds = precision_recall_curve(y_true, y_scores)
 
plt.figure(figsize=(8, 6))
plt.plot(recall, precision, marker='.', label=f'AP = {ap:.3f}')
plt.xlabel('Recall')
plt.ylabel('Precision')
plt.title('Precision-Recall Curve')
plt.legend()
plt.grid(True)
plt.show()

References

• What is Average Precision in Object Detection & Localization Algorithms | Medium
• Scikit-learn documentation
• COCO detection evaluation metrics