datumaro.plugins.inference_server_plugin.base#

Classes

`InferenceServerType`(value)	Types of the dedicated inference server
`LauncherForDedicatedInferenceServer`(...[, ...])	Inference launcher for dedicated inference server
`ProtocolType`(value)	Protocol type for communication with dedicated inference server
`TLSConfig`(client_key_path, client_cert_path, ...)	TLS configuration dataclass

class datumaro.plugins.inference_server_plugin.base.InferenceServerType(value)[source]#

Bases: IntEnum

Types of the dedicated inference server

ovms = 0#

triton = 1#

class datumaro.plugins.inference_server_plugin.base.ProtocolType(value)[source]#

Bases: IntEnum

Protocol type for communication with dedicated inference server

grpc = 0#

http = 1#

class datumaro.plugins.inference_server_plugin.base.TLSConfig(client_key_path: str, client_cert_path: str, server_cert_path: str)[source]#

Bases: object

TLS configuration dataclass

Parameters:

client_key_path – Path to client key file
client_cert_path – Path to client certificate file
server_cert_path – Path to server certificate file

client_key_path: str#

client_cert_path: str#

server_cert_path: str#

as_dict() → Dict[str, str][source]#

as_grpc_creds() → ChannelCredentials[source]#

class datumaro.plugins.inference_server_plugin.base.LauncherForDedicatedInferenceServer(model_name: str, model_interpreter_path: str, model_version: int = 0, host: str = 'localhost', port: int = 9000, timeout: float = 10.0, tls_config: TLSConfig | None = None, protocol_type: ProtocolType = ProtocolType.grpc)[source]#

Bases: Generic[TClient], LauncherWithModelInterpreter

Inference launcher for dedicated inference server

Parameters:

model_name – Name of the model. It should match with the model name loaded in the server instance.
model_interpreter_path – Python source code path which implements a model interpreter. The model interpreter implement pre-processing of the model input and post-processing of the model output.
model_version – Version of the model loaded in the server instance
host – Host address of the server instance
port – Port number of the server instance
timeout – Timeout limit during communication between the client and the server instance
tls_config – Configuration required if the server instance is in the secure mode
protocol_type – Communication protocol type with the server instance

type_check(item)[source]#

Check the media type of dataset item.

If False, the item is excluded from the input batch.