Adversarial Training

Adversarial training exposes the model to adversarially perturbed inputs during training so it learns to classify both clean and perturbed versions of the same input correctly. The objective becomes a min-max problem: minimize loss with respect to model parameters $\theta$, while maximizing loss with respect to a bounded perturbation $\delta$.

$$\underset{\theta }{\min }{\mathbb{E}}_{(x,y)}\left[\underset{\Vert \delta {\Vert }_p\le ϵ}{\max }\mathcal{L}(f_{\theta }(x+\delta ),y)\right]$$

Plain language: “even if the input is slightly messed with, you should still make the right prediction.”

Intuition

Standard training fits the data distribution. Adversarial training fits an $ϵ$-ball around every training point, so any input within that ball maps to the correct label. The norm $\Vert \cdot {\Vert }_p$ defines the threat model: ${\ell }_{\infty }$ for per-pixel perturbations, ${\ell }_2$ for total energy, ${\ell }_0$ for sparse pixel changes.

How The Inner Maximization Is Solved

See Adversarial Training - Inner Maximization.

Connection To Gradient-Based Vulnerabilities

The reason FGSM and PGD work at all is that small input changes can cause large output changes when the loss surface in input space is steep. This is the input-space analogue of the parameter-space issues discussed in Vanishing Gradient — the gradient is fine, but the model’s local linearization extrapolates badly.

Benefits

• Improves robustness to bounded input perturbations within the chosen norm.
• Acts as a strong regularizer; can improve generalization in low-data regimes.
• Forces representations to be locally invariant, which sometimes helps interpretability.

Drawbacks

• Training cost grows roughly with the number of inner-attack steps.
• Often a small drop in clean accuracy in exchange for adversarial accuracy.
• Robustness to one threat model (e.g. ${\ell }_{\infty }$, $ϵ=8/255$) does not transfer to others.
• Doesn’t help against distributional shift or semantic perturbations not captured by the $ϵ$-ball.

One-line Interview Answer

Adversarial training solves a min-max objective where the inner max produces worst-case inputs (typically via PGD) and the outer min trains the model to classify them correctly, trading clean accuracy and compute for bounded-perturbation robustness.

Common Questions

• Why is FGSM weaker than PGD? FGSM is a one-step linear approximation of the inner max; PGD takes many steps and is closer to the true worst case.
• Does adversarial training defend against all attacks? No — it defends within the chosen norm and budget; transfer attacks and unforeseen perturbations can still break it.
• Why does it sometimes hurt clean accuracy? The constraint to be locally flat around training points can underfit fine-grained features that aren’t robust but were predictive on clean data.