caikit.core.module_backends

Submodules

Classes

`BackendBase`	Interface for creating configuration setup for backends
`LocalBackend`	Interface for creating configuration setup for backends

Package Contents

class caikit.core.module_backends.BackendBase(config: aconfig.Config | None = None)[source]

Bases: abc.ABC

Interface for creating configuration setup for backends

config

_started = False

_start_lock

class property backend_type

Abstractmethod:

Property storing type of the backend

property is_started

abstract register_config(config)[source]: Function to allow dynamic merging of configs. This can be useful, if there are explicit configurations particular implementations (modules) need to register before the starting the backend.

abstract start()[source]: Function to start a distributed backend. This function should set self._started variable to True

abstract stop()[source]: Function to stop a distributed backend. This function should set self._started variable to False

start_lock()[source]

handle_runtime_context(model_id: str, runtime_context: caikit.core.data_model.runtime_context.RuntimeServerContextType)[source]

Update backend state for the given model based on a runtime request.

Some backends may need to handle runtime context information for the target model in order to correctly configure the backend before finding and loading the model. By default, this is a No-Op.

Args:

model_id (str): The unique ID of the model that is referenced by the: runtime context
runtime_context (RuntimeServerContextType): The context for the: given runtime request

class caikit.core.module_backends.LocalBackend(config: aconfig.Config | None = None)[source]

Bases: caikit.core.module_backends.base.BackendBase

Interface for creating configuration setup for backends

backend_type = 'LOCAL': Property storing type of the backend

register_config(config) → None[source]: Function to merge configs with existing configurations

start()[source]: Start local backend. This is a no-op function

stop()[source]: Stop local backend. This is a no-op