mmsegmentation

OTX Adapters - mmseg.

class otx.algorithms.segmentation.adapters.mmseg.ClassIncrEncoderDecoder(*args, task_adapt=None, **kwargs)

Encoder-decoder for incremental learning.

forward_train(img, img_metas, gt_semantic_seg, aux_img=None, **kwargs)

Forward function for training.

Args:

img (Tensor): Input images. img_metas (list[dict]): List of image info dict where each dict

has: ‘img_shape’, ‘scale_factor’, ‘flip’, and may also contain ‘filename’, ‘ori_shape’, ‘pad_shape’, and ‘img_norm_cfg’. For details on the values of these keys see mmseg/datasets/pipelines/formatting.py:Collect.

gt_semantic_seg (Tensor): Semantic segmentation masks

used if the architecture supports semantic segmentation task.

aux_img (Tensor): Auxiliary images.

Returns:

dict[str, Tensor]: a dictionary of loss components

static load_state_dict_pre_hook(model, model_classes, chkpt_classes, chkpt_dict, prefix, *args, **kwargs)

Modify input state_dict according to class name matching before weight loading.

training: bool

class otx.algorithms.segmentation.adapters.mmseg.ConstantScalarScheduler(scale: float = 30.0)

The learning rate remains constant over time.

The learning rate equals the scale.

Args:

scale (float): The learning rate scale.

class otx.algorithms.segmentation.adapters.mmseg.CrossEntropyLossWithIgnore(reduction='mean', loss_weight=None, **kwargs)

CrossEntropyLossWithIgnore with Ignore Mode Support for Class Incremental Learning.

Args:

model_classes (list[str]): Model classes bg_aware (bool, optional): Whether to enable BG-aware loss

‘background’ class would be added the start of model classes/label schema

reduction (str, optional): . Defaults to ‘mean’.

Options are “none”, “mean” and “sum”.

loss_weight (float, optional): Weight of the loss. Defaults to 1.0.

property name

name.

training: bool

class otx.algorithms.segmentation.adapters.mmseg.CustomFCNHead(*args, **kwargs)

Custom Fully Convolution Networks for Semantic Segmentation.

training: bool

class otx.algorithms.segmentation.adapters.mmseg.DetConB(backbone: Dict[str, Any], neck: Optional[Dict[str, Any]] = None, head: Optional[Dict[str, Any]] = None, pretrained: Optional[str] = None, base_momentum: float = 0.996, num_classes: int = 256, num_samples: int = 16, downsample: int = 32, input_transform: str = 'resize_concat', in_index: Union[List[int], int] = [0], align_corners: bool = False, loss_cfg: Optional[Dict[str, Any]] = None, **kwargs)

Implementation of ‘Efficient Visual Pretraining with Contrastive Detection’ (https://arxiv.org/abs/2103.10957).

Args:

backbone (dict): Config dict for module of backbone ConvNet. neck (dict, optional): Config dict for module of deep features to compact feature vectors.

Default: None.

head (dict, optional): Config dict for module of loss functions. Default: None. pretrained (str, optional): Path to pre-trained weights. Default: None. base_momentum (float): The base momentum coefficient for the target network.

Default: 0.996.

num_classes (int): The number of classes to be considered as pseudo classes. Default: 256. num_samples (int): The number of samples to be sampled. Default: 16. downsample (int): The ratio of the mask size to the feature size. Default: 32. input_transform (str): Input transform of features from backbone. Default: “resize_concat”. in_index (list): Feature index to be used for DetCon if the backbone outputs

multi-scale features wrapped by list or tuple. Default: [0].

align_corners (bool): Whether apply align_corners during resize. Default: False. loss_cfg (dict): DetCon loss configuration.

extract_feat(img: Tensor)

Extract features from images.

Args:

img (Tensor): Input image.

Return:

Tensor: Features from the online_backbone.

forward(img, img_metas, return_loss=True, **kwargs)

Calls either forward_train() or forward_test() depending on whether return_loss is True.

Note this setting will change the expected inputs. When return_loss=True, img and img_meta are single-nested (i.e. Tensor and List[dict]), and when resturn_loss=False, img and img_meta should be double nested (i.e. List[Tensor], List[List[dict]]), with the outer list indicating test time augmentations.

forward_train(img: Tensor, img_metas: List[Dict], gt_semantic_seg: Tensor, return_embedding: bool = False)

Forward function for training.

Args:

img (Tensor): Input images. img_metas (list[dict]): Input information. gt_semantic_seg (Tensor): Pseudo masks.

It is used to organize features among the same classes.

return_embedding (bool): Whether returning embeddings from the online backbone.

It can be used for SupCon. Default: False.

Returns:

dict[str, Tensor]: A dictionary of loss components.

init_weights(pretrained: Optional[str] = None)

Initialize the weights of model.

Args:

pretrained (str, optional): Path to pre-trained weights.

Default: None.

sample_masked_feats(feats: Union[Tensor, List, Tuple], masks: Tensor, projector: Module)

Sampled features from mask.

Args:

feats (list, tuple, Tensor): Features from the backbone. masks (Tensor): Ground truth masks to be sampled and to be used to filter feats. projector (nn.Module): Projector MLP.

Returns:

tuple[Tensor, Tensor]: (proj, sampled_mask_ids), features from the projector and ids used to sample masks.

set_step_params(init_iter, epoch_size)

set_step_params to be skipped.

static state_dict_hook(module, state_dict, *args, **kwargs)

Save only online backbone as output state_dict.

train_step(data_batch: Dict[str, Any], optimizer: Union[Optimizer, Dict], **kwargs)

The iteration step during training.

This method defines an iteration step during training, except for the back propagation and optimizer updating, which are done in an optimizer hook. Note that in some complicated cases or models, the whole process including back propagation and optimizer updating is also defined in this method, such as GAN.

Args:

data_batch (dict): The output of dataloader. optimizer (torch.optim.Optimizer | dict): The optimizer of

runner is passed to train_step(). This argument is unused and reserved.

Returns:

dict: It should contain at least 3 keys: loss, log_vars,

num_samples. loss is a tensor for back propagation, which can be a weighted sum of multiple losses. log_vars contains all the variables to be sent to the logger. num_samples indicates the batch size (when the model is DDP, it means the batch size on each GPU), which is used for averaging the logs.

training: bool

transform_inputs(inputs: Union[List, Tuple])

Transform inputs for decoder.

Args:

inputs (list, tuple): List (or tuple) of multi-level img features.

Returns:

Tensor: The transformed inputs.

val_step(**kwargs)

Disenable validation step during self-supervised learning.

class otx.algorithms.segmentation.adapters.mmseg.DetConLoss(temperature: float = 0.1, use_replicator_loss: bool = True, ignore_index: int = 255)

Modified from deepmind/detcon.

Compute the NCE scores from pairs of predictions and targets. This implements the batched form of the loss described in Section 3.1, Equation 3 in https://arxiv.org/pdf/2103.10957.pdf.

Args:

temperature: (float) the temperature to use for the NCE loss. use_replicator_loss (bool): use cross-replica samples.

forward(pred1, pred2, target1, target2, pind1, pind2, tind1, tind2, local_negatives=True)

Forward loss.

Args:

pred1 (Tensor): (b, num_samples, d) the prediction from first view. pred2 (Tensor): (b, num_samples, d) the prediction from second view. target1 (Tensor): (b, num_samples, d) the projection from first view. target2 (Tensor): (b, num_samples, d) the projection from second view. pind1 (Tensor): (b, num_samples) mask indices for first view’s prediction. pind2 (Tensor): (b, num_samples) mask indices for second view’s prediction. tind1 (Tensor): (b, num_samples) mask indices for first view’s projection. tind2 (Tensor): (b, num_samples) mask indices for second view’s projection. local_negatives (bool): whether to include local negatives.

Returns:

dict[str, Tensor]: A single scalar loss for the XT-NCE objective.

get_distributed_tensors(target1, target2, batch_size, num_samples, num_features, device)

Grab tensors across replicas during distributed training.

training: bool

class otx.algorithms.segmentation.adapters.mmseg.LiteHRNet(extra, in_channels=3, conv_cfg=None, norm_cfg=None, norm_eval=False, with_cp=False, zero_init_residual=False, dropout=None, init_cfg=None)

Lite-HRNet backbone.

High-Resolution Representations for Labeling Pixels and Regions

Args:

extra (dict): detailed configuration for each stage of HRNet. in_channels (int): Number of input image channels. Default: 3. conv_cfg (dict): dictionary to construct and config conv layer. norm_cfg (dict): dictionary to construct and config norm layer. norm_eval (bool): Whether to set norm layers to eval mode, namely,

freeze running stats (mean and var). Note: Effect on Batch Norm and its variants only. Default: False

with_cp (bool): Use checkpoint or not. Using checkpoint will save some

memory while slowing down the training speed.

zero_init_residual (bool): whether to use zero init for last norm layer

in resblocks to let them behave as identity.

forward(x)

Forward function.

init_weights(pretrained=None)

Initialize the weights in backbone.

Args:

pretrained (str, optional): Path to pre-trained weights.

Defaults to None.

train(mode=True)

Convert the model into training mode.

training: bool

class otx.algorithms.segmentation.adapters.mmseg.MMOVBackbone(*args, **kwargs)

MMOVBackbone.

forward(*args, **kwargs)

Forward.

init_weights(pretrained=None)

Initialize the weights.

training: bool

class otx.algorithms.segmentation.adapters.mmseg.MMOVDecodeHead(model_path_or_model: Optional[Union[str, Model]] = None, weight_path: Optional[str] = None, inputs: Optional[Dict[str, Union[str, List[str]]]] = None, outputs: Optional[Dict[str, Union[str, List[str]]]] = None, init_weight: bool = False, verify_shape: bool = True, *args, **kwargs)

MMOVDecodeHead.

forward(inputs)

Forward.

init_weights()

Init weights.

training: bool

class otx.algorithms.segmentation.adapters.mmseg.MPASegDataset(**kwargs)

Wrapper dataset that allows using a OTX dataset to train models.

class otx.algorithms.segmentation.adapters.mmseg.MeanTeacherSegmentor(orig_type=None, unsup_weight=0.1, semisl_start_iter=30, **kwargs)

Mean teacher segmentor for semi-supervised learning.

It creates two models and ema from one to the other for consistency loss.

aug_test(imgs, img_metas, **kwargs)

Aug test.

encode_decode(img, img_metas)

Encode and decode images.

extract_feat(imgs)

Extract feature.

forward_dummy(img, **kwargs)

Forward dummy.

forward_train(img, img_metas, gt_semantic_seg, **kwargs)

Forward train.

static load_state_dict_pre_hook(module, state_dict, *args, **kwargs)

Redirect input state_dict to teacher model.

simple_test(img, img_meta, **kwargs)

Simple test.

static state_dict_hook(module, state_dict, prefix, *args, **kwargs)

Redirect student model as output state_dict (teacher as auxilliary).

training: bool

class otx.algorithms.segmentation.adapters.mmseg.PolyScalarScheduler(start_scale: float, end_scale: float, num_iters: int, power: float = 1.2, by_epoch: bool = False)

The learning rate changes over time according to a polynomial schedule.

Args:

start_scale (float): The initial learning rate scale. end_scale (float): The final learning rate scale. num_iters (int): The number of iterations to reach the final learning rate. power (float): The power of the polynomial schedule. by_epoch (bool): Whether to use epoch as the unit of iteration.

class otx.algorithms.segmentation.adapters.mmseg.SelfSLMLP(in_channels: int, hid_channels: int, out_channels: int, norm_cfg: Dict[str, Any] = {'type': 'BN1d'}, use_conv: bool = False, with_avg_pool: bool = True)

The SelfSLMLP neck: fc/conv-bn-relu-fc/conv.

Args:

in_channels (int): The number of feature output channels from backbone. hid_channels (int): The number of channels for a hidden layer. out_channels (int): The number of output channels of SelfSLMLP. norm_cfg (dict): Normalize configuration. Default: dict(type=”BN1d”). use_conv (bool): Whether using conv instead of fc. Default: False. with_avg_pool (bool): Whether using average pooling before passing MLP.

Default: True.

forward(x)

Forward SelfSLMLP.

Args:

x (Tensor, tuple, list): Inputs to pass MLP.

If a type of the inputs is tuple or list, just use the last index.

Return:

Tensor: Features passed SelfSLMLP.

init_weights(init_linear: str = 'normal', std: float = 0.01, bias: float = 0.0)

Initialize SelfSLMLP weights.

Args:

init_linear (str): Option to initialize weights. Default: “normal”. std (float): Standard deviation for normal initialization. Default: 0.01. bias (float): Bias for normal initialization. Default: 0.

training: bool

class otx.algorithms.segmentation.adapters.mmseg.StepScalarScheduler(scales: List[float], num_iters: List[int], by_epoch: bool = False)

Step learning rate scheduler.

Example:

>>> scheduler = StepScalarScheduler(scales=[1.0, 0.1, 0.01], num_iters=[100, 200])
This means that the learning rate will be 1.0 for the first 100 iterations,
0.1 for the next 200 iterations, and 0.01 for the rest of the iterations.

Args:

scales (List[int]): List of learning rate scales. num_iters (List[int]): A list specifying the count of iterations at each scale. by_epoch (bool): Whether to use epoch as the unit of iteration.

class otx.algorithms.segmentation.adapters.mmseg.SupConDetConB(backbone: Dict[str, Any], decode_head: Optional[Dict[str, Any]] = None, neck: Optional[Dict[str, Any]] = None, head: Optional[Dict[str, Any]] = None, pretrained: Optional[str] = None, base_momentum: float = 0.996, num_classes: int = 256, num_samples: int = 16, downsample: int = 32, input_transform: str = 'resize_concat', in_index: Union[List[int], int] = [0], align_corners: bool = False, loss_cfg: Optional[Dict[str, Any]] = None, train_cfg: Optional[Dict[str, Any]] = None, test_cfg: Optional[Dict[str, Any]] = None, **kwargs)

Apply DetConB as a contrastive part of Supervised Contrastive Learning (https://arxiv.org/abs/2004.11362).

SupCon with DetConB uses ground truth masks instead of pseudo masks to organize features among the same classes.

Args:

decode_head (dict, optional): Config dict for module of decode head. Default: None. train_cfg (dict, optional): Config dict for training. Default: None.

forward_train(img, img_metas, gt_semantic_seg, pixel_weights=None, **kwargs)

Forward function for training.

Args:

img (Tensor): Input images. img_metas (list[dict]): Input information. gt_semantic_seg (Tensor): Ground truth masks.

It is used to organize features among the same classes.

pixel_weights (Tensor): Pixels weights.

Returns:

dict[str, Tensor]: A dictionary of loss components.

training: bool

Initialize OTX Segmentation with MMSEG.

Export task for OTX Segmentation with MMSEG.

class otx.algorithms.segmentation.adapters.mmseg.tasks.exporter.SegExporter(name, mode, config, common_cfg=None, index=0, **kwargs)

Class for segmentation model export.

static naive_export(output_dir, model_builder, precision, cfg, model_name='model')

Export using pytorch backend.

run(model_cfg, model_ckpt, data_cfg, **kwargs)

Run exporter stage.

OTX segmentation inference with MMSEG.

class otx.algorithms.segmentation.adapters.mmseg.tasks.inferrer.SegInferrer(*args, **kwargs)

Inference class with MMSEG.

infer(cfg, model_builder=None, dump_features=False)

Main inference function.

run(model_cfg, model_ckpt, data_cfg, **kwargs)

Run inference stage for segmentation.

• Configuration

• Environment setup

• Run inference via MMSegmentation -> MMCV

otx.algorithms.segmentation.adapters.mmseg.tasks.inferrer.replace_ImageToTensor(pipelines)

Change ImageToTensor pipeline to DefaultFormatBundle.

Base stage for OTX Segmentation.

class otx.algorithms.segmentation.adapters.mmseg.tasks.stage.SegStage(name, mode, config, common_cfg=None, index=0, **kwargs)

Class for configuration of segmentation recipe.

MODEL_BUILDER(train_cfg: Optional[Union[Config, ConfigDict]] = None, test_cfg: Optional[Union[Config, ConfigDict]] = None, checkpoint: Optional[str] = None, device: Union[str, device] = 'cpu', cfg_options: Optional[Union[Config, ConfigDict]] = None, from_scratch: bool = False) → Module

A builder function for mmseg model.

Creates a model, based on the configuration in config. Note that this function updates ‘load_from’ attribute of ‘config’.

configure(model_cfg, model_ckpt, data_cfg, training=True, **kwargs)

Create MMCV-consumable config from given inputs.

configure_classes(cfg, task_adapt_op)

Patch model_classes and data_classes.

configure_data(cfg, training, data_cfg)

Patch data_cfg.

configure_ignore(cfg)

Change to incremental loss (ignore mode).

configure_model(cfg, model_cfg, **kwargs)

Patch model_cfg.

configure_task(cfg, training)

Adjust settings for task adaptation.

Base Trainer for OTX segmentation with MMSEG.

class otx.algorithms.segmentation.adapters.mmseg.tasks.trainer.SegTrainer(name, mode, config, common_cfg=None, index=0, **kwargs)

Class for OTX segmentation train.

run(model_cfg, model_ckpt, data_cfg, **kwargs)

Run training stage for segmentation.

• Configuration

• Environment setup

• Run training via MMSegmentation -> MMCV

Initialize OTX Segmentation with MMSEG.

class otx.algorithms.segmentation.adapters.mmseg.tasks.incremental.IncrSegInferrer(**kwargs)

Inference class for incremental learning.

class otx.algorithms.segmentation.adapters.mmseg.tasks.incremental.IncrSegTrainer(**kwargs)

Trainer for incremental segmentation.

Inference for OTX segmentation model with Incremental learning.

class otx.algorithms.segmentation.adapters.mmseg.tasks.incremental.inferrer.IncrSegInferrer(**kwargs)

Inference class for incremental learning.

Stage for Incremental learning OTX segmentation with MMSEG.

class otx.algorithms.segmentation.adapters.mmseg.tasks.incremental.stage.IncrSegStage(**kwargs)

Calss for incremental learning for segmentation.

configure_task(cfg, training)

Adjust settings for task adaptation.

Trainer for Incremental OTX Segmentation with MMSEG.

class otx.algorithms.segmentation.adapters.mmseg.tasks.incremental.trainer.IncrSegTrainer(**kwargs)

Trainer for incremental segmentation.

Initialize Semi-SL tasks for OTX segmentation.

class otx.algorithms.segmentation.adapters.mmseg.tasks.semisl.SemiSLSegExporter(**kwargs)

Exporter for semi-sl segmentation.

configure(model_cfg, model_ckpt, data_cfg, training=False, **kwargs)

Patch config for semi-sl segmentation.

class otx.algorithms.segmentation.adapters.mmseg.tasks.semisl.SemiSLSegInferrer(**kwargs)

Inference class for Semi-SL.

configure(model_cfg, model_ckpt, data_cfg, training=False, **kwargs)

Patch config for semi-sl classification.

class otx.algorithms.segmentation.adapters.mmseg.tasks.semisl.SemiSLSegTrainer(**kwargs)

Class for semi-sl segmentation model train.

Inferenc for Semi-SL OTX classification with MMCLS.

class otx.algorithms.segmentation.adapters.mmseg.tasks.semisl.inferrer.SemiSLSegInferrer(**kwargs)

Inference class for Semi-SL.

configure(model_cfg, model_ckpt, data_cfg, training=False, **kwargs)

Patch config for semi-sl classification.

Stage for Semi-SL OTX Segmentation with MMSEG.

class otx.algorithms.segmentation.adapters.mmseg.tasks.semisl.stage.SemiSLSegStage(**kwargs)

Semi-SL stage for segmentation.

configure_data(cfg, training, data_cfg)

Patch cfg.data.

configure_task(cfg, training, **kwargs)

Adjust settings for task adaptation.

Train Semi-SL OTX Segmentation model with MMSEG.

class otx.algorithms.segmentation.adapters.mmseg.tasks.semisl.trainer.SemiSLSegTrainer(**kwargs)

Class for semi-sl segmentation model train.