otx.core.model.visual_prompting#

Class definition for visual prompting models entity used in OTX.

Classes

OTXVisualPromptingModel(label_info, ...)

Base class for the visual prompting models used in OTX.

OVVisualPromptingModel(model_name, ...)

Visual prompting model compatible for OpenVINO IR inference.
class otx.core.model.visual_prompting.OTXVisualPromptingModel(label_info: LabelInfoTypes = NullLabelInfo(label_names=[], label_ids=[], label_groups=[[]]), input_size: tuple[int, int] = (1024, 1024), optimizer: OptimizerCallable = <function _default_optimizer_callable>, scheduler: LRSchedulerCallable | LRSchedulerListCallable = <function _default_scheduler_callable>, metric: MetricCallable = <function _visual_prompting_metric_callable>, torch_compile: bool = False)[source]#

Bases: OTXModel

Base class for the visual prompting models used in OTX.

get_dummy_input(batch_size: int = 1) → VisualPromptingBatchDataEntity[source]#

Returns a dummy input for VPT model.

test_step(inputs: VisualPromptingBatchDataEntity, batch_idx: int) → None[source]#

Perform a single test step on a batch of data from the test set.

Parameters:

  • inputs (VisualPromptingBatchDataEntity) – The input data for the test step.

  • batch_idx (int) – The index of the current batch.

Raises:

TypeError – If the predictions are not of type VisualPromptingBatchPredEntity.

validation_step(inputs: VisualPromptingBatchDataEntity, batch_idx: int) → None[source]#

Perform a single validation step on a batch of data from the validation set.

Parameters:

  • inputs (VisualPromptingBatchDataEntity) – The input data for the validation step.

  • batch_idx (int) – The index of the current batch.

Raises:

TypeError – If the predictions are not of type VisualPromptingBatchPredEntity.

Returns:

None

class otx.core.model.visual_prompting.OVVisualPromptingModel(model_name: str, model_type: str = 'Visual_Prompting', async_inference: bool = False, max_num_requests: int | None = None, use_throughput_mode: bool = False, model_api_configuration: dict[str, Any] | None = None, metric: MetricCallable = <function _visual_prompting_metric_callable>, **kwargs)[source]#

Bases: OVModel

Visual prompting model compatible for OpenVINO IR inference.

It can only consume OpenVINO IR model path and create the OTX visual prompting model compatible

for OTX testing pipeline.

forward(inputs: VisualPromptingBatchDataEntity) → VisualPromptingBatchPredEntity[source]#

Model forward function.

get_dummy_input(batch_size: int = 1) → VisualPromptingBatchDataEntity[source]#

Returns a dummy input for classification OV model.

optimize(output_dir: Path, data_module: OTXDataModule, ptq_config: dict[str, Any] | None = None) → dict[str, Path][source]#

Runs NNCF quantization.

test_step(inputs: VisualPromptingBatchDataEntity, batch_idx: int) → None[source]#

Perform a single test step on a batch of data from the test set.

Parameters:

  • inputs (VisualPromptingBatchDataEntity) – The input data for the test step.

  • batch_idx (int) – The index of the current batch.

Raises:

TypeError – If the predictions are not of type VisualPromptingBatchPredEntity.

validation_step(inputs: VisualPromptingBatchDataEntity, batch_idx: int) → None[source]#

Perform a single validation step on a batch of data from the validation set.

Parameters:

  • inputs (VisualPromptingBatchDataEntity) – The input data for the validation step.

  • batch_idx (int) – The index of the current batch.

Raises:

TypeError – If the predictions are not of type VisualPromptingBatchPredEntity.

Returns:

None