Openvino Adapter#

class model_api.adapters.openvino_adapter.OpenvinoAdapter(core, model, weights_path=None, model_parameters={}, device='CPU', plugin_config=None, max_num_requests=0, precision='FP16', download_dir=None, cache_dir=None)#

Bases: InferenceAdapter

Works with OpenVINO model

precision, download_dir and cache_dir are ignored if model is a path to a file

await_all()#

In case of asynchronous execution waits the completion of all busy infer requests.

Return type:

None

await_any()#

In case of asynchronous execution waits the completion of any busy infer request until it becomes available for the data submission.

Return type:

None

copy_raw_result(request)#
embed_preprocessing(layout, resize_mode, interpolation_mode, target_shape, pad_value, dtype=<class 'int'>, brg2rgb=False, mean=None, scale=None, input_idx=0)#

Embeds preprocessing into the model using OpenVINO preprocessing API

Return type:

None

get_input_layers()#
Gets the names of model inputs and for each one creates the Metadata structure,

which contains the information about the input shape, layout, precision in OpenVINO format, meta (optional)

Return type:

dict[str, Metadata]

Returns:

  • the dict containing Metadata for all inputs

get_layout_for_input(input, shape=None)#
Return type:

str

get_model()#

Returns the openvino.Model object

Returns:

openvino.Model object

get_output_layers()#
Gets the names of model outputs and for each one creates the Metadata structure,

which contains the information about the output shape, layout, precision in OpenVINO format, meta (optional)

Return type:

dict[str, Metadata]

Returns:

  • the dict containing Metadata for all outputs

get_raw_result(request)#

Gets raw results from the internal inference framework representation as a dict.

Parameters:

infer_result (-) – framework-specific result of inference from the model

Returns:

{

‘output_layer_name_1’: raw_result_1, ‘output_layer_name_2’: raw_result_2, …

}

Return type:

  • raw result (dict) - model raw output in the following format

get_rt_info(path)#

Forwards to openvino.Model.get_rt_info(path)

Return type:

OVAny

infer_async(dict_data, callback_data)#

Performs the asynchronous model inference and sets the callback for inference completion. Also, it should define get_raw_result() function, which handles the result of inference from the model.

Parameters:

  • dict_data (-) –

    it’s submitted to the model for inference and has the following format: {

    ’input_layer_name_1’: data_1, ‘input_layer_name_2’: data_2, …

    }

  • callback_data (-) – the data for callback, that will be taken after the model inference is ended

Return type:

None

infer_sync(dict_data)#

Performs the synchronous model inference. The infer is a blocking method.

Parameters:

dict_data (-) –

it’s submitted to the model for inference and has the following format: {

’input_layer_name_1’: data_1, ‘input_layer_name_2’: data_2, …

}

Returns:

{

‘output_layer_name_1’: raw_result_1, ‘output_layer_name_2’: raw_result_2, …

}

Return type:

  • raw result (dict) - model raw output in the following format

is_ready()#

In case of asynchronous execution checks if one can submit input data to the model for inference, or all infer requests are busy.

Return type:

bool

Returns:

  • the boolean flag whether the input data can be

    submitted to the model for inference or not

load_model()#

Loads the model on the device.

Return type:

None

log_runtime_settings()#
Return type:

None

operations_by_type(operation_type)#
reshape_model(new_shape)#

Reshapes the model inputs to fit the new input shape.

Parameters:

new_shape (-) –

the dictionary with inputs names as keys and list of new shape as values in the following format: {

’input_layer_name_1’: [1, 128, 128, 3], ‘input_layer_name_2’: [1, 128, 128, 3], …

}

save_model(path, weights_path='', version='UNSPECIFIED')#

Serializes model to the filesystem.

set_callback(callback_fn)#
update_model_info(model_info)#

Populates OV IR RT info with the given model info.

Parameters:

model_info (dict[str, Any]) – a dict representing the serialized parameters.

model_api.adapters.openvino_adapter.create_core()#
Return type:

Core

model_api.adapters.openvino_adapter.get_input_shape(input_tensor)#
Return type:

list[int]

model_api.adapters.openvino_adapter.get_user_config(flags_d, flags_nstreams, flags_nthreads=None)#
Return type:

dict[str, str]

model_api.adapters.openvino_adapter.parse_devices(device_string)#
Return type:

list[str]

model_api.adapters.openvino_adapter.parse_value_per_device(devices, values_string)#

Format: <device1>:<value1>,<device2>:<value2> or just <value>

Return type:

dict[str, int]