Noisy Label Detection

OpenVINO™ Training Extensions provide a feature for detecting noisy labels during model training. With this feature, you can identify noisy labeled samples in your training dataset. Our algorithm accumulates the training loss dynamics during the model training and exports it to Datumaro. The training loss dynamics are then post-processed by exponential moving average (EMA), a strong criterion for detecting noisy label samples [1]. Finally, Datumaro ranks the top-k samples, which can be considered as noisy labeled candidates. We provide an end-to-end example written in a Jupyter notebook, which you can find at the link in the note below.

In OpenVINO™ Training Extensions CLI, you can enable this feature by adding the argument --algo_backend.enable_noisy_label_detection true as follows.

$ otx train ... params --algo_backend.enable_noisy_label_detection true

Note

Currently, it supports both multi-class classification task and detection task with single GPU training.

Note

Important! The post-processing step to analyze the training loss dynamics requires Datumaro. Please see the following Jupyter notebook examples:

1. Multi-class classification task.

2. Detection task.

[1]

Zhou, Tianyi, Shengjie Wang, and Jeff Bilmes. “Robust curriculum learning: from clean label detection to noisy label self-correction.” International Conference on Learning Representations. 2021.