otx.algorithms.visual_prompting.tasks.inference#

Visual Prompting Task.

Classes

InferenceTask(task_environment[, output_path])

Base Visual Prompting Task.

ZeroShotTask(task_environment[, output_path])

Learn task for Zero-shot learning.
class otx.algorithms.visual_prompting.tasks.inference.InferenceTask(task_environment: TaskEnvironment, output_path: str | None = None)[source]#

Bases: IInferenceTask, IEvaluationTask, IExportTask, IUnload

Base Visual Prompting Task.

Train, Infer, and Export an Visual Prompting Task.

Parameters:

  • task_environment (TaskEnvironment) – OTX Task environment.

  • output_path (Optional[str]) – output path where task output are saved.

cleanup() → None[source]#

Clean up work directory.

evaluate(output_resultset: ResultSetEntity, evaluation_metric: str | None = None) → None[source]#

Evaluate the performance on a result set.

Parameters:

  • output_resultset (ResultSetEntity) – Result Set from which the performance is evaluated.

  • evaluation_metric (Optional[str], optional) – Evaluation metric. Defaults to None. Instead, metric is chosen depending on the task type.

export(export_type: ExportType, output_model: ModelEntity, precision: ModelPrecision = ModelPrecision.FP32, dump_features: bool = False) → None[source]#

Export model to OpenVINO IR.

When SAM gets an image for inference, image encoder runs just once to get image embedding. After that, prompt encoder + mask decoder runs repeatedly to get mask prediction. For this case, SAM should be divided into two parts, image encoder and prompt encoder + mask decoder.

Parameters:

  • export_type (ExportType) – Export type should be ExportType.OPENVINO

  • output_model (ModelEntity) – The model entity in which to write the OpenVINO IR data

  • precision (bool) – Output model weights and inference precision

  • dump_features (bool) – Flag to return “feature_vector” and “saliency_map”.

Raises:

Exception – If export_type is not ExportType.OPENVINO

get_config() → DictConfig | ListConfig[source]#

Get Visual Prompting Config from task environment.

Returns:

Visual Prompting config.

Return type:

Union[DictConfig, ListConfig]

infer(dataset: DatasetEntity, inference_parameters: InferenceParameters) → DatasetEntity[source]#

Perform inference on a dataset.

Parameters:
Returns:

Output dataset with predictions.

Return type:

DatasetEntity

load_model(otx_model: ModelEntity | None = None) → LightningModule[source]#

Create and Load Visual Prompting Module.

Currently, load model through sam_model_registry because there is only SAM. If other visual prompting model is added, loading model process must be changed.

Parameters:

otx_model (Optional[ModelEntity]) – OTX Model from the task environment.

Returns:

Visual prompting model with/without weights.

Return type:

LightningModule

model_info() → Dict[source]#

Return model info to save the model weights.

Returns:

Model info.

Return type:

Dict

save_model(output_model: ModelEntity) → None[source]#

Save the model after training is completed.

Parameters:

output_model (ModelEntity) – Output model onto which the weights are saved.

set_seed()[source]#

Set seed and deterministic.

unload() → None[source]#

Unload the task.

class otx.algorithms.visual_prompting.tasks.inference.ZeroShotTask(task_environment: TaskEnvironment, output_path: str | None = None)[source]#

Bases: InferenceTask

Learn task for Zero-shot learning.

**There are two ways to be decided: 1. use it independently <– temporarily current setting 2. use it depending on template

The objective of this task is to get reference features and export it with decoder modules.

infer(dataset: DatasetEntity, inference_parameters: InferenceParameters) → DatasetEntity[source]#

Perform inference on a dataset.

Parameters:
Returns:

Output dataset with predictions.

Return type:

DatasetEntity

save_model(output_model: ModelEntity) → None[source]#

Save the model after training is completed.

Parameters:

output_model (ModelEntity) – Output model onto which the weights are saved.