caikit.runtime.server_base

Base class with common functionality across all caikit servers

Attributes

log
error

Classes

ServerThreadPool	Simple wrapper for all servers to share a single thread pool
RuntimeServerBase	Helper class that provides a standard way to create an ABC using

Module Contents

caikit.runtime.server_base.log

caikit.runtime.server_base.error

class caikit.runtime.server_base.ServerThreadPool

Simple wrapper for all servers to share a single thread pool

static _build_pool() → concurrent.futures.ThreadPoolExecutor

pool

class caikit.runtime.server_base.RuntimeServerBase(base_port: int, tls_config_override: aconfig.Config | None)

Bases: abc.ABC

Helper class that provides a standard way to create an ABC using inheritance.

__doc__ = 'Base class with common functionality across all caikit servers'

_metrics_server_started = False

config

port

tls_config

enable_inference

enable_training

enable_inference_jobs

inference_service: caikit.runtime.service_factory.ServicePackage | None

inference_job_service: caikit.runtime.service_factory.ServicePackage | None

training_service

runtime_info_service: caikit.runtime.service_factory.ServicePackage | None

thread_pool: concurrent.futures.ThreadPoolExecutor

interrupter: caikit.runtime.work_management.abortable_context.ThreadInterrupter | None

classmethod _start_metrics_server() → None

Start a single instance of the metrics server based on configuration

interrupt(signal_, _stack_frame)

_intercept_interrupt_signal() → None

Intercept signal handlers to allow the server to stop on interrupt. Calling this on a non-main thread has no effect. This does not override any existing non-default signal handlers, it will call them all in the reverse order they are registered.

static _add_signal_handler(sig, handler)

_shut_down_model_manager()

Shared utility for shutting down the model manager

abstract start(blocking: bool = True)

abstract stop()

__enter__()

__exit__(type_, value, traceback)