DFM¶
This is the implementation of DFM paper.
Model Type: Classification
Description¶
Fast anomaly classification algorithm that consists of a deep feature extraction stage followed by anomaly classification stage consisting of PCA and class-conditional Gaussian Density Estimation.
Feature Extraction¶
Features are extracted by feeding the images through a ResNet18 backbone, which was pre-trained on ImageNet. The output of the penultimate layer (average pooling layer) of the network is used to obtain a semantic feature vector with a fixed length of 2048.
Anomaly Detection¶
In the anomaly classification stage, class-conditional PCA transformations and Gaussian Density models are learned. Two types of scores are calculated (i) Feature-reconstruction scores (norm of the difference between the high-dimensional pre-image of a reduced dimension feature and the original high-dimensional feature), and (ii) Negative log-likelihood under the learnt density models. Either of these scores can be used for anomaly detection.
Usage¶
$ python tools/train.py --model dfm
PyTorch model for DFM model implementation.
- class anomalib.models.dfm.torch_model.DFMModel(backbone: str, layer: str, input_size: tuple[int, int], pre_trained: bool = True, pooling_kernel_size: int = 4, n_comps: float = 0.97, score_type: str = 'fre')[source]¶
Bases:
Module
Model for the DFM algorithm.
- Parameters:
backbone (str) – Pre-trained model backbone.
layer (str) – Layer from which to extract features.
input_size (tuple[int, int]) – Input size for the model.
pre_trained (bool, optional) – Boolean to check whether to use a pre_trained backbone.
pooling_kernel_size (int, optional) – Kernel size to pool features extracted from the CNN.
n_comps (float, optional) – Ratio from which number of components for PCA are calculated. Defaults to 0.97.
score_type (str, optional) – Scoring type. Options are fre and nll. Defaults to “fre”. Anomaly
nll (segmentation is supported with fre only. If using) –
classification (set task in config.yaml to) –
Initializes internal Module state, shared by both nn.Module and ScriptModule.
- fit(dataset: Tensor) None [source]¶
Fit a pca transformation and a Gaussian model to dataset.
- Parameters:
dataset (Tensor) – Input dataset to fit the model.
- forward(batch: Tensor) Tensor [source]¶
Computer score from input images.
- Parameters:
batch (Tensor) – Input images
- Returns:
Scores
- Return type:
Tensor
- get_features(batch: Tensor) Tensor [source]¶
Extract features from the pretrained network.
- Parameters:
batch (Tensor) – Image batch.
- Returns:
Tensor containing extracted features.
- Return type:
Tensor
- score(features: Tensor, feature_shapes: tuple) Tensor [source]¶
Compute scores.
Scores are either PCA-based feature reconstruction error (FRE) scores or the Gaussian density-based NLL scores
- Parameters:
features (torch.Tensor) – semantic features on which PCA and density modeling is performed.
feature_shapes (tuple) – shape of features tensor. Used to generate anomaly map of correct shape.
- Returns:
numpy array of scores
- Return type:
score (Tensor)
- training: bool¶
- class anomalib.models.dfm.torch_model.SingleClassGaussian[source]¶
Bases:
DynamicBufferModule
Model Gaussian distribution over a set of points.
Initializes internal Module state, shared by both nn.Module and ScriptModule.
- fit(dataset: Tensor) None [source]¶
Fit a Gaussian model to dataset X.
Covariance matrix is not calculated directly using:
C = X.X^T
Instead, it is represented in terms of the Singular Value Decomposition of X:X = U.S.V^T
Hence,C = U.S^2.U^T
This simplifies the calculation of the log-likelihood without requiring full matrix inversion.- Parameters:
dataset (Tensor) – Input dataset to fit the model.
- forward(dataset: Tensor) None [source]¶
Provides the same functionality as fit.
Transforms the input dataset based on singular values calculated earlier.
- Parameters:
dataset (Tensor) – Input dataset
- score_samples(features: Tensor) Tensor [source]¶
Compute the NLL (negative log likelihood) scores.
- Parameters:
features (Tensor) – semantic features on which density modeling is performed.
- Returns:
Torch tensor of scores
- Return type:
nll (Tensor)
- training: bool¶
DFM: Deep Feature Modeling.
- class anomalib.models.dfm.lightning_model.Dfm(backbone: str, layer: str, input_size: tuple[int, int], pre_trained: bool = True, pooling_kernel_size: int = 4, pca_level: float = 0.97, score_type: str = 'fre')[source]¶
Bases:
AnomalyModule
DFM: Deep Featured Kernel Density Estimation.
- Parameters:
backbone (str) – Backbone CNN network
layer (str) – Layer to extract features from the backbone CNN
input_size (tuple[int, int]) – Input size for the model.
pre_trained (bool, optional) – Boolean to check whether to use a pre_trained backbone.
pooling_kernel_size (int, optional) – Kernel size to pool features extracted from the CNN. Defaults to 4.
pca_level (float, optional) – Ratio from which number of components for PCA are calculated. Defaults to 0.97.
score_type (str, optional) – Scoring type. Options are fre and nll. Defaults to “fre”.
nll (supported with fre only. If using) – for Gaussian modeling, fre: pca feature-reconstruction error. Anomaly segmentation is
nll –
classification (set task in config.yaml to) –
- static configure_optimizers() None [source]¶
DFM doesn’t require optimization, therefore returns no optimizers.
- training_step(batch: dict[str, str | Tensor], *args, **kwargs) None [source]¶
Training Step of DFM.
For each batch, features are extracted from the CNN.
- Parameters:
batch (dict[str, str | Tensor]) – Input batch
_ – Index of the batch.
- Returns:
Deep CNN features.
- validation_step(batch: dict[str, str | Tensor], *args, **kwargs) STEP_OUTPUT [source]¶
Validation Step of DFM.
Similar to the training step, features are extracted from the CNN for each batch.
- Parameters:
batch (dict[str, str | Tensor]) – Input batch
- Returns:
Dictionary containing FRE anomaly scores and anomaly maps.