Johnny's Knowledge

Isolation Forest

Properties1
tagsmachine-learning, anomaly-detection

multivariate anomalies

  • have two or more attributes
  • attributes are not necessarily anomalous
  • only anomalous when all attributes are considered
    e.g a 12-year old kid weighs 190 pounds, 160cm

split randomly
outlier:

  • require fewer splits
  • close to the root node

Most Important Hyperparameters

  • contamination
    • when IForest classifies data points, raw anomaly scores generated, set a threshold called contamination (accept a value from 0-0.5)
    • For example, a 10% contamination means we are choosing the observations with the top 10% of anomaly scores as outliers.
    • Low contamination results in undetected anomalies, while high contamination can lead to inliers being marked as anomalies.
  • n_estimators
    • We use more trees for high-dimensional datasets to have enough predictive power to learn all the relevant patterns in the data.
  • max_samples and max_features
    • Each of these trees train on a sub-sample of the dataset and a sub-sample of features, controlled by max_samples and max_features parameters, which accept values (0-1)
    • e.g max_samples = 0.6, max_features =0.9 meaning, for every iTree, a different of 60% of the rows and a different 90% of features will be selecte. This frequent sub-sampling reduces the risk of overfitting
      can choose hyperparams by combining the outlier classifier with a supervised learning model to quantify

TBD

Isolation Forest Guide: Explanation and Python Implementation | DataCamp
Isolation Forest Anomaly Detection

in scikit-learn, IsolationForest, the .score_samples() method returns a score where higher values mean more normal. It automatically does the preprocessing that defined before, lets say

pipe = Pipeline([("prep", preprocessor), ("clf", model)])
pipe.fit(X_train)
pipe.score_samples(X_valid)
# same as pipe["clf"].score_samples(pipe["prep"].transform(X_valid))

Backlinks