Target encoding replaces a categorical value with a statistic of the target variable for that category. For classification, this is often the mean target rate. For regression, it is often the mean target value.

Example: if users from category A have a 30% conversion rate, the category A may be encoded as 0.30.

Intuition

One-hot encoding tells the model that a category exists. Target encoding tells the model how that category has behaved with respect to the prediction target.

This can be powerful for high-cardinality categorical features such as user IDs, product IDs, cities, or domains.

How It Works

For a category $c$ , the basic encoded value is:

e n c (c) = \frac{1}{n _{c}} i : x_{i} = c \sum y_{i}

where $n_{c}$ is the number of training examples in category $c$ .

Leakage Risk

Target encoding can leak target information if it is computed using the same row being encoded. This makes training performance look better than it really is.

To reduce leakage, compute encodings with cross-validation or out-of-fold statistics. Each training row should be encoded using target statistics computed from other rows, not itself.

Smoothing

Rare categories are noisy. A category with only one positive example should not automatically receive an encoding of 1.0.

Smoothing blends the category mean with the global mean:

e n co d e d = w \cdot m e a n_{c} + (1 - w) \cdot m e a n_{g l o ba l}

where $w$ increases as the category has more observations.

When To Use

Target encoding is useful when categorical variables have many unique values and Machine Learning models need compact numeric features. It is often useful for tree-based models and tabular data.

Avoid it or use it carefully when the dataset is small, categories are rare, or leakage controls are not implemented.

Reference

TargetEncoder - scikit-learn documentation

Johnny's Knowledge

Explorer

Target Encoding