Multiple Object Tracking and Segmentation (MOTS)

Format specification

The Multiple Object Tracking and Segmentation (MOTS) challenge dataset provides a pixel-level segmentation masks for multiple objects within image sequences. There are two format specifications according to the representation of segmentation masks: 1) PNG format and 2) TXT format. The PNG format represents a segmentation mask as a PNG file with a 16-bits single color channel. On the other hand, the TXT format uses run-length encoding (RLE) for a segmentation mask. Datumaro currently only supports the PNG format, not the TXT format.

Supported annotation types:

• Mask (segmentation)

Supported annotation attributes:

• track_id (int) - Unique ID assigned to an object within a trajectory

Import MOTS dataset

You can download the PNG format of MOTS challange dataset here.

A Datumaro project with the MOTS challange source can be created in the following way:

datum project create
datum project import --format mots <path/to/dataset>

It is possible to specify project name and project directory. Run datum project create --help for more information.

The MOTS challange dataset directory should have the following structure:

└─ Dataset/
  ├── train
  │   ├── images
  │   │   ├── <name_1>.<img_ext>
  │   │   ├── <name_2>.<img_ext>
  │   │   └── ...
  │   └── instances
  │       ├── <name_1>.png
  │       ├── <name_2>.png
  │       ├── ...
  │       └── labels.txt
  └── val
      ├── images
      │   └── ...     # Same as above
      └── instances
          └── ...     # Same as above

To make sure that the selected dataset has been added to the project, you can run datum project info, which will display the project information.

Export to other formats

Datumaro can convert the MOTS challange dataset into any other format Datumaro supports.

Such conversion will only be successful if the output format can represent the type of dataset you want to convert, e.g. segmentation annotations can be saved in Cityscapes format, but not as COCO keypoints.

There are several ways to convert a MOTS dataset to other dataset formats:

datum project create
datum project import -f mots <path/to/mots>
datum project export -f coco_instances -o <output/dir>

or

datum convert -if mots -i <path/to/mots> -f coco_instances -o <output/dir>

Or, using Python API:

import datumaro as dm

dataset = dm.Dataset.import_from('<path/to/dataset>', 'mots')
dataset.export('save_dir', 'cityscapes', save_media=True)

Export to MOTS

There are several ways to convert a dataset to MOTS format:

# export dataset into MOTS format from existing project
datum project export -p <path/to/project> -f mots -o <output/dir> \
    -- --save-media

# converting to MOTS format from other format
datum convert -if cityscapes -i <path/to/dataset> \
    -f mots -o <output/dir> -- --save-media

Extra options for exporting to MOTS format:

• --save-media allow to export dataset with saving media files (by default False)

• --image-ext IMAGE_EXT allow to specify image extension for exporting dataset (by default - keep original or use .png, if none)

• --save-dataset-meta - allow to export dataset with saving dataset meta file (by default False)

Examples

Examples of using this format from the code can be found in the format tests