caikit.runtime.model_management.model_loader_base

Attributes

log

Classes

ModelLoaderBase

Model Loader Base class which describes how models are loaded.

Module Contents

caikit.runtime.model_management.model_loader_base.log[source]

class caikit.runtime.model_management.model_loader_base.ModelLoaderBase(config: aconfig.Config, instance_name: str)[source]

Bases: caikit.core.toolkit.factory.FactoryConstructible

Model Loader Base class which describes how models are loaded.

_load_thread_pool = None

abstract load_module_instance(model_path: str, model_id: str, model_type: str, finder: str | caikit.core.model_management.ModelFinderBase | None = None, initializer: str | caikit.core.model_management.ModelInitializerBase | None = None) → caikit.core.ModuleBase[source]

Load an instance of a Caikit Model

Args:

model_path (str): The model path to load from model_id (str): The model’s id model_type (str): The type of model being load finder (Optional[Union[str, ModelFinderBase]], optional): The ModelFinder to use for

loading. Defaults to None.

initializer (Optional[Union[str, ModelInitializerBase]], optional): The: ModelInitializer to use for loading. Defaults to None.

Returns:

ModuleBase: a loaded model

load_model(model_id: str, local_model_path: str, model_type: str, fail_callback: Callable | None = None, retries: int = 0, finder: str | caikit.core.model_management.ModelFinderBase | None = None, initializer: str | caikit.core.model_management.ModelInitializerBase | None = None) → caikit.runtime.model_management.loaded_model.LoadedModel[source]

Start loading a model from disk and associate the ID/size with it

Args:

model_id (str): Model ID string for the model to load. local_model_path (str): Local filesystem path to load the model from. model_type (str): Type of the model to load. fail_callback (Optional[Callable]): Optional no-arg callback to call

on load failure

retries (int): Number of times to retry loading

Returns:

model (LoadedModel) : The model that was loaded

_wrapped_load_model(model_path: str, model_id: str, model_type: str, finder: str | caikit.core.model_management.ModelFinderBase | None = None, initializer: str | caikit.core.model_management.ModelInitializerBase | None = None) → caikit.runtime.model_management.batcher.Batcher | caikit.core.ModuleBase[source]

_wrap_in_batcher_if_configured(caikit_core_model: caikit.core.ModuleBase, model_type: str, model_id: str) → caikit.runtime.model_management.batcher.Batcher | caikit.core.ModuleBase[source]: Perform Batcher wrapping on the given module if configured, otherwise return the model as is