caikit.core.module_backends.base

Base class to initialize backend

Classes

BackendBase

Interface for creating configuration setup for backends

Module Contents

class caikit.core.module_backends.base.BackendBase(config: aconfig.Config | None = None)

Bases: abc.ABC

Interface for creating configuration setup for backends

config

_started = False

_start_lock

class property backend_type

Abstractmethod:

Property storing type of the backend

property is_started

abstract register_config(config)

Function to allow dynamic merging of configs. This can be useful, if there are explicit configurations particular implementations (modules) need to register before the starting the backend.

abstract start()

Function to start a distributed backend. This function should set self._started variable to True

abstract stop()

Function to stop a distributed backend. This function should set self._started variable to False

start_lock()

handle_runtime_context(model_id: str, runtime_context: caikit.core.data_model.runtime_context.RuntimeServerContextType)

Update backend state for the given model based on a runtime request.

Some backends may need to handle runtime context information for the target model in order to correctly configure the backend before finding and loading the model. By default, this is a No-Op.

Args:

model_id (str): The unique ID of the model that is referenced by the

runtime context

runtime_context (RuntimeServerContextType): The context for the

given runtime request