datumaro.plugins.missing_annotation_detection#

Classes

MissingAnnotationDetection(extractor, launcher)

This class is used to find annotations that are missing from the ground truth annotations.

class datumaro.plugins.missing_annotation_detection.MissingAnnotationDetection(extractor: IDataset, launcher: Launcher, batch_size: int = 1, pairwise_dist: float = 0.75, score_threshold: float | None = None, label_agnostic_matching: bool = True)[source]#

Bases: ModelTransform

This class is used to find annotations that are missing from the ground truth annotations.

To accomplish this, it generates model predictions for the dataset using the given launcher. However, not all of these model predictions can be considered missing annotations since the dataset already contains ground truth annotations and some of the model predictions can be duplicated with them. To identify the missing annotations from the model predictions, this class filters out the predictions that spatially overlap with the ground truth annotations.

For example, the follwing example will produce [Bbox(1, 1, 1, 1, label=1, attributes={"score": 0.5})] as the missing annotations since Bbox(0, 0, 1, 1, label=1, attributes={"score": 1.0}) is overlapped with the ground-truth annotation, Bbox(0, 0, 1, 1, label=0) (label_agnostic_matching=True)

ground_truth_annotations = [
    Bbox(0, 0, 1, 1, label=0),
    Bbox(1, 0, 1, 1, label=1),
    Bbox(0, 1, 1, 1, label=2),
]
model_predictions = [
    Bbox(0, 0, 1, 1, label=1, attributes={"score": 1.0}),
    Bbox(1, 1, 1, 1, label=1, attributes={"score": 0.5}),
]

Parameters:

extractor – The dataset used to find missing labeled annotations.
launcher – The launcher used to generate model predictions from the dataset.
batch_size – The size of the batches used during processing.
pairwise_dist – The distance metric used to measure the distance between two annotations. Typically, the distance metric is Intersection over Union (IoU), which is bounded between 0 and 1.
score_threshold – The minimum score required for an annotation to be considered a candidate for missing annotations.
label_agnostic_matching – If set to false, annotations with different labels are not matched to determine their spatial overlap. In the above example, label_agnostic_matching=False will produce model_predictions as is since Bbox(0, 0, 1, 1, label=1, attributes={"score": 1.0}) has different label with Bbox(0, 0, 1, 1, label=0).