otx.algorithms.visual_prompting.tasks.openvino#

OpenVINO Visual Prompting Task.

Classes

`OTXOpenVinoDataLoader`(dataset, inferencer, ...)	DataLoader implementation for VisualPromptingOpenVINOTask.
`OpenVINOVisualPromptingInferencer`(hparams, ...)	Inferencer implementation for Visual Prompting using OpenVINO backend.
`OpenVINOVisualPromptingTask`(task_environment)	Task implementation for Visual Prompting using OpenVINO backend.
`OpenVINOZeroShotVisualPromptingInferencer`(...)	Inferencer implementation for Zero-shot Visual Prompting using OpenVINO backend.
`OpenVINOZeroShotVisualPromptingTask`(...)	Task implementation for Zero-shot Visual Prompting using OpenVINO backend.

class otx.algorithms.visual_prompting.tasks.openvino.OTXOpenVinoDataLoader(dataset: Any, inferencer: OpenVINOVisualPromptingInferencer, module_name: str, shuffle: bool = True, output_model: ModelEntity | None = None, **kwargs)[source]#

Bases: object

DataLoader implementation for VisualPromptingOpenVINOTask.

class otx.algorithms.visual_prompting.tasks.openvino.OpenVINOVisualPromptingInferencer(hparams: VisualPromptingBaseConfig, label_schema: LabelSchemaEntity, model_files: Dict[str, str | Path | bytes], weight_files: Dict[str, str | Path | bytes | None] | None = {}, device: str = 'CPU', num_requests: int = 1)[source]#

Bases: IInferencer

Inferencer implementation for Visual Prompting using OpenVINO backend.

This inferencer has two models, image encoder and decoder.

Parameters:

hparams (VisualPromptingBaseConfig) – Hyper parameters that the model should use.
label_schema (LabelSchemaEntity) – LabelSchemaEntity that was used during model training.
model_files (Dict[str, Union[str, Path, bytes]]) – Path or bytes to model to load, .xml, .bin or .onnx file.
weight_files (Dict[str, Union[str, Path, bytes, None]], optional) – Path or bytes to weights to load, .xml, .bin or .onnx file. Defaults to None.
device (str) – Device to run inference on, such as CPU, GPU or MYRIAD. Defaults to “CPU”.
num_requests (int) – Maximum number of requests that the inferencer can make. Good value is the number of available cores. Defaults to 1.

await_all() → None[source]#: Await all running infer requests if any.

forward_decoder(inputs: Dict[str, ndarray]) → Dict[str, ndarray][source]#: Forward function of OpenVINO Visual Prompting Inferencer.

forward_image_encoder(inputs: Dict[str, ndarray]) → Dict[str, ndarray][source]#: Forward function of OpenVINO Visual Prompting Inferencer.

post_process(prediction: Dict[str, ndarray], metadata: Dict[str, Any]) → Tuple[List[Annotation], Any, Any][source]#: Post-process function of OpenVINO Visual Prompting Inferencer.

pre_process(dataset_item: DatasetItemEntity, extra_processing: bool = False, use_bbox: bool = False, use_point: bool = False) → Tuple[Dict[str, Any], Dict[str, Any], List[Dict[str, Any]]][source]#: Pre-process function of OpenVINO Visual Prompting Inferencer for image encoder.

predict(dataset_item: DatasetItemEntity) → List[Annotation][source]#: Perform a prediction for a given input image.

class otx.algorithms.visual_prompting.tasks.openvino.OpenVINOVisualPromptingTask(task_environment: TaskEnvironment)[source]#

Bases: IInferenceTask, IEvaluationTask, IOptimizationTask, IDeploymentTask

Task implementation for Visual Prompting using OpenVINO backend.

deploy(output_model: ModelEntity) → None[source]#: Deploy function of OpenVINOVisualPromptingTask.

evaluate(output_resultset: ResultSetEntity, evaluation_metric: str | None = None)[source]#: Evaluate function of OpenVINOVisualPromptingTask.

infer(dataset: DatasetEntity, inference_parameters: InferenceParameters | None = None) → DatasetEntity[source]#

Infer function of OpenVINOVisualPromptingTask.

Currently, asynchronous execution is not supported, synchronous execution will be executed instead.

load_inferencer() → OpenVINOVisualPromptingInferencer[source]#: Load OpenVINO Visual Prompting Inferencer.

optimize(optimization_type: ~otx.api.usecases.tasks.interfaces.optimization_interface.OptimizationType, dataset: ~otx.api.entities.datasets.DatasetEntity, output_model: ~otx.api.entities.model.ModelEntity, optimization_parameters: ~otx.api.entities.optimization_parameters.OptimizationParameters | None = None, module_names: ~typing.List[str] = ['image_encoder', 'decoder'], ov_dataloader: ~typing.Type[~otx.algorithms.visual_prompting.tasks.openvino.OTXOpenVinoDataLoader] = <class 'otx.algorithms.visual_prompting.tasks.openvino.OTXOpenVinoDataLoader'>, **kwargs)[source]#: Optimize function of OpenVINOVisualPromptingTask.

property avg_time_per_image: float | None#: Average inference time per image.

property hparams#: Hparams of OpenVINO Visual Prompting Task.

class otx.algorithms.visual_prompting.tasks.openvino.OpenVINOZeroShotVisualPromptingInferencer(hparams: VisualPromptingBaseConfig, label_schema: LabelSchemaEntity, model_files: Dict[str, str | Path | bytes], weight_files: Dict[str, str | Path | bytes | None] | None = {}, device: str = 'CPU', num_requests: int = 1)[source]#

Bases: OpenVINOVisualPromptingInferencer

Inferencer implementation for Zero-shot Visual Prompting using OpenVINO backend.

This inferencer has two models, image encoder and decoder.

Parameters:

hparams (VisualPromptingBaseConfig) – Hyper parameters that the model should use.
label_schema (LabelSchemaEntity) – LabelSchemaEntity that was used during model training.
model_files (Dict[str, Union[str, Path, bytes]]) – Path or bytes to model to load, .xml, .bin or .onnx file.
weight_files (Dict[str, Union[str, Path, bytes, None]], optional) – Path or bytes to weights to load, .xml, .bin or .onnx file. Defaults to None.
device (str) – Device to run inference on, such as CPU, GPU or MYRIAD. Defaults to “CPU”.
num_requests (int) – Maximum number of requests that the inferencer can make. Good value is the number of available cores. Defaults to 1.

expand_reference_info(new_largest_label: int) → None[source]#: Expand reference info dimensions if newly given processed prompts have more lables.

forward_decoder(inputs: Dict[str, ndarray], original_size: ndarray, is_cascade: bool = True) → Dict[str, ndarray][source]#: Forward function of OpenVINO Visual Prompting Inferencer.

infer(images: ndarray, reference_feats: ndarray, used_indices: ndarray, is_cascade: bool = False, threshold: float = 0.0, num_bg_points: int = 1, default_threshold_target: float = 0.65) → Tuple[List[Any], DefaultDict[Any, Any], DefaultDict[Any, Any]][source]#: Perform a prediction for a given input image.

initialize_reference_info() → None[source]#: Initialize reference information.

learn(dataset_item: DatasetItemEntity, reset_feat: bool = False, use_bbox: bool = False, use_point: bool = False, path_reference_info: str = 'vpm_zsl_reference_infos/{}/reference_info.pickle', default_threshold_reference: float = 0.3) → Tuple[Dict[str, ndarray], ndarray][source]#: Learn for reference features.

pre_process_image_encoder(inputs: ndarray, extra_processing: bool = False) → Tuple[Dict[str, ndarray], Dict[str, Any]][source]#: Pre-process function of OpenVINO Zero-shot Visual Prompting Inferencer for image encoder.

predict(dataset_item: DatasetItemEntity) → List[Annotation][source]#: Perform a prediction for a given input image.

class otx.algorithms.visual_prompting.tasks.openvino.OpenVINOZeroShotVisualPromptingTask(task_environment: TaskEnvironment)[source]#

Bases: OpenVINOVisualPromptingTask

Task implementation for Zero-shot Visual Prompting using OpenVINO backend.

infer(dataset: DatasetEntity, inference_parameters: InferenceParameters | None = None, root: str = 'vpm_zsl_reference_infos', path_reference_info: str = '{}/reference_info.pickle') → DatasetEntity[source]#

Infer function of OpenVINOVisualPromptingTask.

Currently, asynchronous execution is not supported, synchronous execution will be executed instead.

load_inferencer() → OpenVINOZeroShotVisualPromptingInferencer[source]#: Load OpenVINO Zero-shot Visual Prompting Inferencer.

optimize(optimization_type: ~otx.api.usecases.tasks.interfaces.optimization_interface.OptimizationType, dataset: ~otx.api.entities.datasets.DatasetEntity, output_model: ~otx.api.entities.model.ModelEntity, optimization_parameters: ~otx.api.entities.optimization_parameters.OptimizationParameters | None = None, module_names: ~typing.List[str] = ['image_encoder', 'decoder'], ov_dataloader: ~typing.Type[~otx.algorithms.visual_prompting.tasks.openvino.OTXOpenVinoDataLoader] = <class 'otx.algorithms.visual_prompting.tasks.openvino.OTXOpenVinoDataLoader'>, **kwargs)[source]#: Optimize function of OpenVINOZeroShotVisualPromptingTask.