caikit.runtime.client.remote_module_base

The RemoteModuleBase is a base class that can be mutated to have the same task methods as a ModuleBase but submit requests to a remote runtime instead of loading locally. By design this class/factory does not use any references to the original Module class.

Attributes

`log`
`error`

Classes

RemoteModuleBase

Class to act as the base for remote modules. This class will be subclassed and

Functions

construct_remote_module_class(...)

Factory function to construct unique Remote Module Class.

Module Contents

caikit.runtime.client.remote_module_base.log[source]

caikit.runtime.client.remote_module_base.error

class caikit.runtime.client.remote_module_base.RemoteModuleBase(connection_info: caikit.interfaces.common.data_model.ConnectionInfo, protocol: str, model_key: str, model_name: str)[source]

Bases: caikit.core.modules.ModuleBase

Class to act as the base for remote modules. This class will be subclassed and mutated by construct_remote_module_class to make it have the same functions and parameters as the source module.

_model_name

_connection

_tls

_protocol

_model_key

_channel_lock

_conn_channel: grpc.Channel | requests.Session | None = None

_current_conn_time = None

_max_conn_delta

__del__()[source]: Destructor to ensure channel/session is cleaned up on deletion

classmethod generate_train_function(method: caikit.runtime.client.remote_config.RemoteRPCDescriptor) → Callable[source]: Factory function to construct a train function that will then be set as an attribute

classmethod generate_inference_function(task: Type[caikit.core.task.TaskBase], method: caikit.runtime.client.remote_config.RemoteRPCDescriptor) → Callable[source]: Factory function to construct inference functions that will be set as an attribute.

remote_method_request(method: caikit.runtime.client.remote_config.RemoteRPCDescriptor, service_type: caikit.runtime.names.ServiceType, *args, **kwargs) → Any[source]: Function to run a remote request based on the data stored in RemoteRPCDescriptor

_request_via_http(method: caikit.runtime.client.remote_config.RemoteRPCDescriptor, service_type: caikit.runtime.names.ServiceType, *args, **kwargs) → Any[source]

_request_via_grpc(method: caikit.runtime.client.remote_config.RemoteRPCDescriptor, service_type: caikit.runtime.names.ServiceType, *args, **kwargs) → Any[source]: Helper function to send a grpc request

property _grpc_channel: grpc.Channel: Helper function to construct a GRPC channel with correct credentials and TLS settings.

property _http_session: requests.Session: Helper function to construct a requests Session with with correct credentials and TLS settings.

_get_remote_object(construction_fn: Callable[[None], grpc.Channel | requests.Session]) → grpc.Channel | requests.Session[source]

Helper function to control construction of a grpc channel or http session

Args:: construction_fn (Callable[[None], Union[grpc.Channel, Session]]): _description_
Returns:: Union[grpc.Channel, Session]: _description_

_get_remote_target() → str[source]: Get the current remote target

static _get_streaming_arguments(**kwargs: Dict[str, Any]) → List[str][source]: Helper function to detect which kwargs are streaming

static _rename_union_sequence_types(obj: Any, dm_type: type)[source]

Helper function that renames all references in a dictionary to match the oneOf value of the DataModel and to collapse all Primitive sequences. This is required to match the format of http requests

For example:

{

“union_str”: “test”, “ints”: {

“values”:[1,2,3]

}

Becomes:

{: “union”: “test”, “ints”:[1,2,3]

}

caikit.runtime.client.remote_module_base.construct_remote_module_class(model_config: caikit.runtime.client.remote_config.RemoteModuleConfig, model_class: Type[RemoteModuleBase] = RemoteModuleBase) → Type[caikit.core.modules.ModuleBase][source]: Factory function to construct unique Remote Module Class.