otx.core.model.visual_prompting#
Class definition for visual prompting models entity used in OTX.
Classes
|
Base class for the visual prompting models used in OTX. |
|
Base class for the zero-shot visual prompting models used in OTX. |
|
Visual prompting model compatible for OpenVINO IR inference. |
|
Zero-shot visual prompting model compatible for OpenVINO IR inference. |
- class otx.core.model.visual_prompting.OTXVisualPromptingModel(label_info: LabelInfoTypes = NullLabelInfo(label_names=[], label_ids=[], label_groups=[[]]), input_size: tuple[int, int] = (1024, 1024), optimizer: OptimizerCallable = <function _default_optimizer_callable>, scheduler: LRSchedulerCallable | LRSchedulerListCallable = <function _default_scheduler_callable>, metric: MetricCallable = <function _visual_prompting_metric_callable>, torch_compile: bool = False)[source]#
Bases:
OTXModel
[VisualPromptingBatchDataEntity
,VisualPromptingBatchPredEntity
]Base class for the visual prompting models used in OTX.
- get_dummy_input(batch_size: int = 1) VisualPromptingBatchDataEntity [source]#
Returns a dummy input for VPT model.
- test_step(inputs: VisualPromptingBatchDataEntity, batch_idx: int) None [source]#
Perform a single test step on a batch of data from the test set.
- class otx.core.model.visual_prompting.OTXZeroShotVisualPromptingModel(input_size: tuple[int, int], label_info: LabelInfoTypes = NullLabelInfo(label_names=[], label_ids=[], label_groups=[[]]), optimizer: OptimizerCallable = <function _default_optimizer_callable>, scheduler: LRSchedulerCallable | LRSchedulerListCallable = <function _default_scheduler_callable>, metric: MetricCallable = <function _visual_prompting_metric_callable>, torch_compile: bool = False)[source]#
Bases:
OTXModel
[ZeroShotVisualPromptingBatchDataEntity
,ZeroShotVisualPromptingBatchPredEntity
]Base class for the zero-shot visual prompting models used in OTX.
- configure_optimizers() None [source]#
Skip configure_optimizers unused in zero-shot visual prompting.
- get_dummy_input(batch_size: int = 1) ZeroShotVisualPromptingBatchDataEntity [source]#
Returns a dummy input for ZSL VPT model.
- on_train_epoch_start() None [source]#
Skip on_train_epoch_start unused in zero-shot visual prompting.
- on_validation_epoch_end() None [source]#
Skip on_validation_epoch_end unused in zero-shot visual prompting.
- on_validation_epoch_start() None [source]#
Skip on_validation_epoch_start unused in zero-shot visual prompting.
- test_step(inputs: ZeroShotVisualPromptingBatchDataEntity, batch_idx: int) None [source]#
Perform a single test step on a batch of data from the test set.
- class otx.core.model.visual_prompting.OVVisualPromptingModel(model_name: str, model_type: str = 'Visual_Prompting', async_inference: bool = False, max_num_requests: int | None = None, use_throughput_mode: bool = False, model_api_configuration: dict[str, Any] | None = None, metric: MetricCallable = <function _visual_prompting_metric_callable>, **kwargs)[source]#
Bases:
OVModel
[VisualPromptingBatchDataEntity
,VisualPromptingBatchPredEntity
]Visual prompting model compatible for OpenVINO IR inference.
- It can only consume OpenVINO IR model path and create the OTX visual prompting model compatible
for OTX testing pipeline.
- forward(inputs: VisualPromptingBatchDataEntity) VisualPromptingBatchPredEntity [source]#
Model forward function.
- get_dummy_input(batch_size: int = 1) VisualPromptingBatchDataEntity [source]#
Returns a dummy input for classification OV model.
- optimize(output_dir: Path, data_module: OTXDataModule, ptq_config: dict[str, Any] | None = None) dict[str, Path] [source]#
Runs NNCF quantization.
- test_step(inputs: VisualPromptingBatchDataEntity, batch_idx: int) None [source]#
Perform a single test step on a batch of data from the test set.
- class otx.core.model.visual_prompting.OVZeroShotVisualPromptingModel(model_name: str, model_type: str = 'Zero_Shot_Visual_Prompting', async_inference: bool = False, max_num_requests: int | None = None, use_throughput_mode: bool = False, model_api_configuration: dict[str, Any] | None = None, metric: MetricCallable = <function _visual_prompting_metric_callable>, reference_info_dir: Path | str = 'reference_infos', infer_reference_info_root: Path | str = '../.latest/train', save_outputs: bool = True, **kwargs)[source]#
Bases:
OVModel
[ZeroShotVisualPromptingBatchDataEntity
,ZeroShotVisualPromptingBatchPredEntity
]Zero-shot visual prompting model compatible for OpenVINO IR inference.
- It can only consume OpenVINO IR model path and create the OTX zero-shot visual prompting model compatible
for OTX testing pipeline.
- configure_optimizers() None [source]#
Skip configure_optimizers unused in zero-shot visual prompting.
- forward(inputs: ZeroShotVisualPromptingBatchDataEntity) tuple[dict[str, ndarray], list[ndarray]] | ZeroShotVisualPromptingBatchPredEntity [source]#
Model forward function.
- get_dummy_input(batch_size: int = 1) ZeroShotVisualPromptingBatchDataEntity [source]#
Returns a dummy input for classification OV model.
- infer(inputs: ZeroShotVisualPromptingBatchDataEntity, reference_feats: np.ndarray, used_indices: np.ndarray) list[dict[int, PredictedMask]] [source]#
Infer for target predictions.
- learn(inputs: ZeroShotVisualPromptingBatchDataEntity, reset_feat: bool = False) tuple[dict[str, ndarray], list[ndarray]] [source]#
Learn for reference features.
- load_reference_info(default_root_dir: Path | str, *args, path_to_directly_load: Path | None = None, **kwargs) bool [source]#
Load latest reference info to be used.
- Parameters:
default_root_dir (Path | str) – Default root directory to be used when inappropriate infer_reference_info_root is given.
path_to_directly_load (Path | None) – Reference info path to directly be loaded. Normally, it is obtained after learn which is executed when trying to do infer without reference features in on_test_start or on_predict_start.
- Returns:
Whether normally loading checkpoint or not.
- Return type:
(bool)
- on_train_epoch_start() None [source]#
Skip on_train_epoch_start unused in zero-shot visual prompting.
- on_validation_epoch_end() None [source]#
Skip on_validation_epoch_end unused in zero-shot visual prompting.
- on_validation_epoch_start() None [source]#
Skip on_validation_epoch_start unused in zero-shot visual prompting.
- optimize(output_dir: Path, data_module: OTXDataModule, ptq_config: dict[str, Any] | None = None) dict[str, Path] [source]#
Runs NNCF quantization.
- test_step(inputs: ZeroShotVisualPromptingBatchDataEntity, batch_idx: int) None [source]#
Perform a single test step on a batch of data from the test set.