Ovms Adapter#

class model_api.adapters.ovms_adapter.OVMSAdapter(target_model)#

Bases: InferenceAdapter

Inference adapter that allows working with models served by the OpenVINO Model Server

Initializes OVMS adapter.

Parameters:

target_model (str) – Model URL. Expected format: <address>:<port>/v2/models/<model_name>[:<model_version>]

await_all()#

In case of asynchronous execution waits the completion of all busy infer requests.

await_any()#

In case of asynchronous execution waits the completion of any busy infer request until it becomes available for the data submission.

embed_preprocessing(layout, resize_mode, interpolation_mode, target_shape, pad_value, dtype=<class 'type'>, brg2rgb=False, mean=None, scale=None, input_idx=0)#

Embeds preprocessing into the model if possible with the adapter being used. In some cases, this method would just add extra python preprocessing steps instaed actuall of embedding it into the model representation.

Parameters:

  • layout (str) – Layout, for instance NCHW.

  • resize_mode (str) – Resize type to use for preprocessing.

  • interpolation_mode (str) – Resize interpolation mode.

  • target_shape (tuple[int, ...]) – Target resize shape.

  • pad_value (int) – Value to pad with if resize implies padding.

  • dtype (type, optional) – Input data type for the preprocessing module. Defaults to int.

  • bgr2rgb (bool, optional) – Defines if we need to swap R and B channels in case of image input.

  • False. (Defaults to)

  • mean (list[Any] | None, optional) – Mean values to perform input normalization. Defaults to None.

  • scale (list[Any] | None, optional) – Scale values to perform input normalization. Defaults to None.

  • input_idx (int, optional) – Index of the model input to apply preprocessing to. Defaults to 0.

get_input_layers()#

Retrieves information about remote model’s inputs.

Returns:

metadata for each input.

Return type:

dict[str, Metadata]

get_model()#

Return the reference to the GrpcClient.

get_output_layers()#

Retrieves information about remote model’s outputs.

Returns:

metadata for each output.

Return type:

dict[str, Metadata]

get_raw_result(infer_result)#

Gets raw results from the internal inference framework representation as a dict.

Parameters:

infer_result (-) – framework-specific result of inference from the model

Returns:

{

‘output_layer_name_1’: raw_result_1, ‘output_layer_name_2’: raw_result_2, …

}

Return type:

  • raw result (dict) - model raw output in the following format

get_rt_info(path)#

Returns an attribute stored in model info.

Return type:

Any

infer_async(dict_data, callback_data)#

A stub method imitating async inference with a blocking call.

infer_sync(dict_data)#

Performs the synchronous model inference. The infer is a blocking method.

Parameters:

dict_data (dict) – data for each input layer.

Returns:

model raw outputs.

Return type:

dict

is_ready()#

In case of asynchronous execution checks if one can submit input data to the model for inference, or all infer requests are busy.

Returns:

  • the boolean flag whether the input data can be

    submitted to the model for inference or not

load_model()#

Loads the model on the device.

reshape_model(new_shape)#

OVMS adapter can not modify the remote model. This method raises an exception.

save_model(path, weights_path=None, version=None)#

OVMS adapter can not retrieve the source model. This method raises an exception.

set_callback(callback_fn)#

Sets callback that grabs results of async inference.

Parameters:

callback_fn (Callable) – Callback function.

update_model_info(model_info)#

OVMS adapter can not update the source model info. This method raises an exception.