DFM¶

This is the implementation of DFM paper.

Model Type: Classification

Description¶

Fast anomaly classification algorithm that consists of a deep feature extraction stage followed by anomaly classification stage consisting of PCA and class-conditional Gaussian Density Estimation.

Feature Extraction¶

Features are extracted by feeding the images through a ResNet18 backbone, which was pre-trained on ImageNet. The output of the penultimate layer (average pooling layer) of the network is used to obtain a semantic feature vector with a fixed length of 2048.

Anomaly Detection¶

In the anomaly classification stage, class-conditional PCA transformations and Gaussian Density models are learned. Two types of scores are calculated (i) Feature-reconstruction scores (norm of the difference between the high-dimensional pre-image of a reduced dimension feature and the original high-dimensional feature), and (ii) Negative log-likelihood under the learnt density models. Either of these scores can be used for anomaly detection.

Usage¶

$ python tools/train.py --model dfm

PyTorch model for DFM model implementation.

class anomalib.models.dfm.torch_model.DFMModel(backbone: str, layer: str, input_size: tuple[int, int], pre_trained: bool = True, pooling_kernel_size: int = 4, n_comps: float = 0.97, score_type: str = 'fre')[source]¶

Bases: Module

Model for the DFM algorithm.

Parameters:

backbone (str) – Pre-trained model backbone.
layer (str) – Layer from which to extract features.
input_size (tuple[int, int]) – Input size for the model.
pre_trained (bool, optional) – Boolean to check whether to use a pre_trained backbone.
pooling_kernel_size (int, optional) – Kernel size to pool features extracted from the CNN.
n_comps (float, optional) – Ratio from which number of components for PCA are calculated. Defaults to 0.97.
score_type (str, optional) – Scoring type. Options are fre and nll. Defaults to “fre”. Anomaly
nll (segmentation is supported with fre only. If using) –
classification (set task in config.yaml to) –

Initializes internal Module state, shared by both nn.Module and ScriptModule.

fit(dataset: Tensor) → None[source]¶

Fit a pca transformation and a Gaussian model to dataset.

Parameters:: dataset (Tensor) – Input dataset to fit the model.

forward(batch: Tensor) → Tensor[source]¶

Computer score from input images.

Parameters:: batch (Tensor) – Input images
Returns:: Scores
Return type:: Tensor

get_features(batch: Tensor) → Tensor[source]¶

Extract features from the pretrained network.

Parameters:: batch (Tensor) – Image batch.
Returns:: Tensor containing extracted features.
Return type:: Tensor

score(features: Tensor, feature_shapes: tuple) → Tensor[source]¶

Compute scores.

Scores are either PCA-based feature reconstruction error (FRE) scores or the Gaussian density-based NLL scores

Parameters:

features (torch.Tensor) – semantic features on which PCA and density modeling is performed.
feature_shapes (tuple) – shape of features tensor. Used to generate anomaly map of correct shape.

Returns:

numpy array of scores

Return type:

score (Tensor)

training: bool¶

class anomalib.models.dfm.torch_model.SingleClassGaussian[source]¶

Bases: DynamicBufferModule

Model Gaussian distribution over a set of points.

Initializes internal Module state, shared by both nn.Module and ScriptModule.

fit(dataset: Tensor) → None[source]¶

Fit a Gaussian model to dataset X.

Covariance matrix is not calculated directly using: C = X.X^T Instead, it is represented in terms of the Singular Value Decomposition of X: X = U.S.V^T Hence, C = U.S^2.U^T This simplifies the calculation of the log-likelihood without requiring full matrix inversion.

Parameters:: dataset (Tensor) – Input dataset to fit the model.

forward(dataset: Tensor) → None[source]¶

Provides the same functionality as fit.

Transforms the input dataset based on singular values calculated earlier.

Parameters:: dataset (Tensor) – Input dataset

score_samples(features: Tensor) → Tensor[source]¶

Compute the NLL (negative log likelihood) scores.

Parameters:: features (Tensor) – semantic features on which density modeling is performed.
Returns:: Torch tensor of scores
Return type:: nll (Tensor)

training: bool¶

DFM: Deep Feature Modeling.

class anomalib.models.dfm.lightning_model.Dfm(backbone: str, layer: str, input_size: tuple[int, int], pre_trained: bool = True, pooling_kernel_size: int = 4, pca_level: float = 0.97, score_type: str = 'fre')[source]¶

Bases: AnomalyModule

DFM: Deep Featured Kernel Density Estimation.

Parameters:

backbone (str) – Backbone CNN network
layer (str) – Layer to extract features from the backbone CNN
input_size (tuple[int, int]) – Input size for the model.
pre_trained (bool, optional) – Boolean to check whether to use a pre_trained backbone.
pooling_kernel_size (int, optional) – Kernel size to pool features extracted from the CNN. Defaults to 4.
pca_level (float, optional) – Ratio from which number of components for PCA are calculated. Defaults to 0.97.
score_type (str, optional) – Scoring type. Options are fre and nll. Defaults to “fre”.
nll (supported with fre only. If using) – for Gaussian modeling, fre: pca feature-reconstruction error. Anomaly segmentation is
nll –
classification (set task in config.yaml to) –

static configure_optimizers() → None[source]¶: DFM doesn’t require optimization, therefore returns no optimizers.

on_validation_start() → None[source]¶: Fit a PCA transformation and a Gaussian model to dataset.

training_step(batch: dict[str, str | Tensor], *args, **kwargs) → None[source]¶

Training Step of DFM.

For each batch, features are extracted from the CNN.

Parameters:

batch (dict[str, str | Tensor]) – Input batch
_ – Index of the batch.

Returns:

Deep CNN features.

validation_step(batch: dict[str, str | Tensor], *args, **kwargs) → STEP_OUTPUT[source]¶

Validation Step of DFM.

Similar to the training step, features are extracted from the CNN for each batch.

Parameters:: batch (dict[str, str | Tensor]) – Input batch
Returns:: Dictionary containing FRE anomaly scores and anomaly maps.

class anomalib.models.dfm.lightning_model.DfmLightning(hparams: DictConfig | ListConfig)[source]¶

Bases: Dfm

DFM: Deep Featured Kernel Density Estimation.

Parameters:: hparams (DictConfig | ListConfig) – Model params