Types of the dedicated inference server

LauncherForDedicatedInferenceServer(...[, ...])

Inference launcher for dedicated inference server


Protocol type for communication with dedicated inference server

TLSConfig(client_key_path, client_cert_path, ...)

TLS configuration dataclass

class datumaro.plugins.inference_server_plugin.base.InferenceServerType(value)[source]#

Bases: IntEnum

Types of the dedicated inference server

ovms = 0#
triton = 1#
class datumaro.plugins.inference_server_plugin.base.ProtocolType(value)[source]#

Bases: IntEnum

Protocol type for communication with dedicated inference server

grpc = 0#
http = 1#
class datumaro.plugins.inference_server_plugin.base.TLSConfig(client_key_path: str, client_cert_path: str, server_cert_path: str)[source]#

Bases: object

TLS configuration dataclass

  • client_key_path – Path to client key file

  • client_cert_path – Path to client certificate file

  • server_cert_path – Path to server certificate file

client_key_path: str#
client_cert_path: str#
server_cert_path: str#
as_dict() Dict[str, str][source]#
as_grpc_creds() ChannelCredentials[source]#
class datumaro.plugins.inference_server_plugin.base.LauncherForDedicatedInferenceServer(model_name: str, model_interpreter_path: str, model_version: int = 0, host: str = 'localhost', port: int = 9000, timeout: float = 10.0, tls_config: TLSConfig | None = None, protocol_type: ProtocolType = ProtocolType.grpc)[source]#

Bases: Generic[TClient], LauncherWithModelInterpreter

Inference launcher for dedicated inference server

  • model_name – Name of the model. It should match with the model name loaded in the server instance.

  • model_interpreter_path – Python source code path which implements a model interpreter. The model interpreter implement pre-processing of the model input and post-processing of the model output.

  • model_version – Version of the model loaded in the server instance

  • host – Host address of the server instance

  • port – Port number of the server instance

  • timeout – Timeout limit during communication between the client and the server instance

  • tls_config – Configuration required if the server instance is in the secure mode

  • protocol_type – Communication protocol type with the server instance


Check the media type of dataset item.

If False, the item is excluded from the input batch.