Principle:Scikit learn Scikit learn Anomaly Detection

Knowledge Sources	Scikit_learn Scikit-learn Docs
Domains	Unsupervised Learning, Outlier Detection
Last Updated	2026-02-08 15:00 GMT

Overview

Anomaly detection identifies observations that deviate significantly from the majority of the data, flagging them as outliers or novelties.

Description

Anomaly detection methods learn what "normal" data looks like and then identify instances that do not conform to this learned pattern. The distinction between outlier detection (the training data contains outliers) and novelty detection (the training data is clean and we detect new anomalies at test time) defines two important sub-problems. These techniques solve the problem of finding unusual patterns in data without requiring labeled examples of anomalies, which are typically rare and difficult to collect. Anomaly detection is critical in fraud detection, network intrusion detection, manufacturing quality control, and medical diagnostics.

Usage

Use Isolation Forest when the dataset is high-dimensional and anomalies are expected to be isolated from normal data -- it is fast and scales well. Use Local Outlier Factor (LOF) when the data has clusters of varying density and local context is important for determining what constitutes an anomaly. Use Elliptic Envelope when the data is approximately Gaussian and you want to detect outliers based on a robust covariance estimate. Isolation Forest and Elliptic Envelope support both outlier detection and novelty detection modes, while LOF is primarily designed for outlier detection (though it has a novelty detection mode).

Theoretical Basis

Isolation Forest is based on the principle that anomalies are easier to isolate than normal points. The algorithm:

1. Construct an ensemble of isolation trees, where each tree recursively partitions data by randomly selecting a feature and a random split value between the feature's min and max.
2. The path length $h(x)$ from the root to the node isolating point $x$ measures how easy it is to isolate.
3. The anomaly score for a point is:

$s(x,n)=2^{-E[h(x)]/c(n)}$

where $E[h(x)]$ is the average path length across trees and $c(n)$ is the average path length of an unsuccessful search in a binary search tree of $n$ elements (normalization factor). Scores close to 1 indicate anomalies; scores close to 0.5 indicate normal points.

Local Outlier Factor (LOF) measures the local density deviation of a point relative to its neighbors:

1. Compute the reachability distance of $x$ from $y$: ${\text{reach-dist}}_k(x,y)=\max (d_k(y),d(x,y))$, where $d_k(y)$ is the distance to $y$'s $k$-th nearest neighbor.
2. Compute the local reachability density: ${\text{lrd}}_k(x)=1/(\frac{1}{k}\sum _{y\in N_k(x)}{\text{reach-dist}}_k(x,y))$
3. The LOF score is:

${\text{LOF}}_k(x)=\frac{1}{k}\sum _{y\in N_k(x)}\frac{{\text{lrd}}_k(y)}{{\text{lrd}}_k(x)}$

LOF values significantly greater than 1 indicate that the point is in a region of lower density than its neighbors, marking it as a local outlier.

Elliptic Envelope assumes data follows a multivariate Gaussian distribution and fits a robust covariance estimate (using the Minimum Covariance Determinant, MCD). The Mahalanobis distance is then used to identify outliers:

$d_M(x)=\sqrt{(x-\hat{\mu }{)}^T{\hat{\mathrm{\Sigma }}}^{-1}(x-\hat{\mu })}$

Points with Mahalanobis distance exceeding a threshold (based on the ${\chi }^2$ distribution) are flagged as outliers.

Related Pages

• Implementation:Scikit_learn_Scikit_learn_IsolationForest
• Implementation:Scikit_learn_Scikit_learn_LocalOutlierFactor
• Implementation:Scikit_learn_Scikit_learn_EllipticEnvelope