caikit.core.model_management.local_model_trainer

The LocalModelTrainer uses a local thread to launch and manage each training job

model_management:

trainers:

<predictor name>:

type: LOCAL config:

# ! Inherits config from LocalJobBase # Whether to train in a subprocess or different thread use_subprocess: <bool: False> # Type of subprocess to use if use_subprocess=True subprocess_start_method: <string: spawn>

Attributes

log
error

Classes

LocalModelTrainer	LocalJobBase is the base class for running background jobs on a local
_SpawnProcessModelWrapper	This class wraps up a model to make it safe to pass to a spawned

Module Contents

caikit.core.model_management.local_model_trainer.log

caikit.core.model_management.local_model_trainer.error

class caikit.core.model_management.local_model_trainer.LocalModelTrainer(config: aconfig.Config, instance_name: str)

Bases: caikit.core.model_management.local_job_base.LocalJobBase, caikit.core.model_management.model_trainer_base.ModelTrainerBase

LocalJobBase is the base class for running background jobs on a local machine and does not depend on any datastore or backend. Any result is cleared after the defined retention_duration

__doc__ = Multiline-String

Show Value

"""
The LocalModelTrainer uses a local thread to launch and manage each training job

model_management:
    trainers:
        <predictor name>:
            type: LOCAL
            config:
                # ! Inherits config from LocalJobBase
                # Whether to train in a subprocess or different thread
                use_subprocess: <bool: False>
                # Type of subprocess to use if use_subprocess=True
                subprocess_start_method: <string: spawn>

"""

class LocalModelTrainFuture(future_name: str, module_class: Type[caikit.core.modules.ModuleBase], save_path: str | caikit.interfaces.common.data_model.stream_sources.S3Path | None, save_with_id: bool, future_id: str | None, args: Iterable[Any], kwargs: Dict[str, Any], model_name: str | None = None, use_subprocess: bool = False, extra_path_args: List[str] | None = None, subprocess_start_method: str | None = 'fork')

Bases: caikit.core.model_management.local_job_base.LocalJobBase.LocalJobFuture, caikit.core.model_management.model_trainer_base.ModelTrainerFutureBase

LocalModelTrainFuture takes a module class and training request and runs it in a thread or subprocess

run(*args, **kwargs)

Function that will run in the worker thread

load() → caikit.core.modules.ModuleBase

Wait for the training to complete, then return the resulting model or raise any errors that happened during training.

LocalModelFuture

name = 'LOCAL'

This is the name of this constructible type that will be used by the factory to identify this class

_use_subprocess

_subprocess_start_method

train(module_class: Type[caikit.core.modules.ModuleBase], *args, save_path: str | None = None, save_with_id: bool = False, external_training_id: str | None = None, model_name: str | None = None, **kwargs) → LocalModelFuture

Start training the given module and return a future to the trained model instance

get_model_future(training_id: str) → LocalModelFuture

Look up the model future for the given id. Use custom implementation to retain error codes

class caikit.core.model_management.local_model_trainer._SpawnProcessModelWrapper(model: caikit.core.modules.ModuleBase)

Bases: caikit.core.modules.ModuleBase

This class wraps up a model to make it safe to pass to a spawned subprocess. It will not be efficient, but it will be safe!

_model

__getattr__(name)

Forward attributes that are not found on the base class to the model

NOTE: This does _not_ forward base class attributes since those are

resolved before __getattr__ is called.

save(*args, **kwargs)

Directly forward save to the model so that it is not called by the base class

run(*args, **kwargs)

Directly forward run to the model so that it is not called by the base class

__getstate__() → bytes

When pickling, only send the serialized model body for non-fork. This is not a general-purpose pickle solution for models, but makes them safe for training jobs that need to move models between processes.

__setstate__(pickled: bytes)

When unpickling, deserialize the body if the model is not already loaded in the model manager. This must be used in conjunction with the above __getstate__ across a process boundary and should not be used as a general-purpose deserialization for models.