otx.core.model.instance_segmentation#

Class definition for instance segmentation model entity used in OTX.

Classes

OTXInstanceSegModel(label_info, ...[, ...])

Base class for the Instance Segmentation models used in OTX.

OVInstanceSegmentationModel(model_name, ...)

Instance segmentation model compatible for OpenVINO IR inference.
class otx.core.model.instance_segmentation.OTXInstanceSegModel(label_info: LabelInfoTypes, data_input_params: DataInputParams, model_name: str = 'inst_segm_model', optimizer: OptimizerCallable = <function _default_optimizer_callable>, scheduler: LRSchedulerCallable | LRSchedulerListCallable = <function _default_scheduler_callable>, metric: MetricCallable = <function _rle_mean_ap_f_measure_callable>, torch_compile: bool = False, tile_config: TileConfig = TileConfig(enable_tiler=False, enable_adaptive_tiling=True, tile_size=(400, 400), overlap=0.2, iou_threshold=0.45, max_num_instances=1500, object_tile_ratio=0.03, sampling_ratio=1.0, with_full_img=False))[source]#

Bases: OTXModel

Base class for the Instance Segmentation models used in OTX.

Parameters:

  • label_info (LabelInfoTypes) – Information about the labels used in the model.

  • data_input_params (DataInputParams) – Parameters for the data input.

  • model_name (str, optional) – Name of the model. Defaults to “inst_segm_model”.

  • optimizer (OptimizerCallable, optional) – Optimizer for the model. Defaults to DefaultOptimizerCallable.

  • scheduler (LRSchedulerCallable | LRSchedulerListCallable, optional) – Scheduler for the model. Defaults to DefaultSchedulerCallable.

  • metric (MetricCallable, optional) – Metric for evaluating the model. Defaults to MaskRLEMeanAPFMeasureCallable.

  • torch_compile (bool, optional) – Whether to use torch compile. Defaults to False.

  • tile_config (TileConfig, optional) – Configuration for tiling. Defaults to TileConfig(enable_tiler=False).

  • explain_mode (bool, optional) – Whether to enable explainable AI mode. Defaults to False.

Initialize the base model with the given parameters.

Parameters:

  • label_info (LabelInfoTypes) – Information about the labels used in the model.

  • data_input_params (DataInputParams) – Parameters of the input data such as input size, mean, and std.

  • model_name (str, optional) – Name of the model. Defaults to “OTXModel”.

  • optimizer (OptimizerCallable, optional) – Callable for the optimizer. Defaults to DefaultOptimizerCallable.

  • scheduler (LRSchedulerCallable | LRSchedulerListCallable) – Callable for the learning rate scheduler. Defaults to DefaultSchedulerCallable.

  • metric (MetricCallable, optional) – Callable for the metric. Defaults to NullMetricCallable.

  • torch_compile (bool, optional) – Flag to indicate if torch.compile should be used. Defaults to False.

  • tile_config (TileConfig, optional) – Configuration for tiling. Defaults to TileConfig(enable_tiler=False).

Returns:

None

export_model_forward_context() → Iterator[None][source]#

A context manager for managing the model’s forward function during model exportation.

It temporarily modifies the model’s forward function to generate output sinks for explain results during the model graph tracing.

forward_explain(inputs: InstanceSegBatchDataEntity) → InstanceSegBatchPredEntity[source]#

Model forward function.

forward_for_tracing(inputs: Tensor) → tuple[Tensor, ...][source]#

Forward function for export.

forward_tiles(inputs: OTXTileBatchDataEntity) → InstanceSegBatchPredEntity[source]#

Unpack instance segmentation tiles.

Parameters:

inputs (OTXTileBatchDataEntity) – Tile batch data entity.

Returns:

Merged instance segmentation prediction.

Return type:

InstanceSegBatchPredEntity

get_dummy_input(batch_size: int = 1) → InstanceSegBatchDataEntity[source]#

Returns a dummy input for instance segmentation model.

get_explain_fn() → Callable[source]#

Returns explain function.

get_results_from_head(x: tuple[Tensor], entity: InstanceSegBatchDataEntity) → tuple[Tensor, Tensor, Tensor] | list[InstanceData] | list[dict[str, Tensor]][source]#

Get the results from the head of the instance segmentation model.

Parameters:

  • x (tuple[Tensor]) – The features from backbone and neck.

  • data_samples (OptSampleList | None) – A list of data samples.

Returns:

The predicted results from the head of the model. Tuple for the Export case, list for the Predict case.

Return type:

tuple[Tensor, Tensor, Tensor] | list[InstanceData]

on_load_checkpoint(ckpt: dict[str, Any]) → None[source]#

Load state_dict from checkpoint.

For detection, it is need to update confidence threshold information when the metric is FMeasure.

class otx.core.model.instance_segmentation.OVInstanceSegmentationModel(model_name: str, model_type: str = 'MaskRCNN', async_inference: bool = True, max_num_requests: int | None = None, use_throughput_mode: bool = True, model_api_configuration: dict[str, Any] | None = None, metric: MetricCallable = <function _rle_mean_ap_f_measure_callable>, **kwargs)[source]#

Bases: OVModel

Instance segmentation model compatible for OpenVINO IR inference.

It can consume OpenVINO IR model path or model name from Intel OMZ repository and create the OTX detection model compatible for OTX testing pipeline.

Initialize the base model with the given parameters.

Parameters:

  • label_info (LabelInfoTypes) – Information about the labels used in the model.

  • data_input_params (DataInputParams) – Parameters of the input data such as input size, mean, and std.

  • model_name (str, optional) – Name of the model. Defaults to “OTXModel”.

  • optimizer (OptimizerCallable, optional) – Callable for the optimizer. Defaults to DefaultOptimizerCallable.

  • scheduler (LRSchedulerCallable | LRSchedulerListCallable) – Callable for the learning rate scheduler. Defaults to DefaultSchedulerCallable.

  • metric (MetricCallable, optional) – Callable for the metric. Defaults to NullMetricCallable.

  • torch_compile (bool, optional) – Flag to indicate if torch.compile should be used. Defaults to False.

  • tile_config (TileConfig, optional) – Configuration for tiling. Defaults to TileConfig(enable_tiler=False).

Returns:

None