caikit.runtime.grpc_server

Attributes

log
PROMETHEUS_METRICS_INTERCEPTOR

Classes

RuntimeGRPCServer

An implementation of a gRPC server that serves caikit runtimes

Functions

main([blocking])

Module Contents

caikit.runtime.grpc_server.log

caikit.runtime.grpc_server.PROMETHEUS_METRICS_INTERCEPTOR

class caikit.runtime.grpc_server.RuntimeGRPCServer(tls_config_override: aconfig.Config | None = None)

Bases: caikit.runtime.server_base.RuntimeServerBase

An implementation of a gRPC server that serves caikit runtimes

server

_global_predict_servicer = None

_prediction_job_management_servicer = None

model_management_service = None

training_management_service = None

start(blocking: bool = True)

Boot the gRPC server. Can be non-blocking, or block until shutdown

Args:

blocking (boolean): Whether to block until shutdown

stop(grace_period_seconds: float | int | None = None)

Stop the server, with an optional grace period.

Args:

grace_period_seconds (Union[float, int]): Grace period for service shutdown.

Defaults to application config

render_protos(proto_out_dir: str) → None

Renders all the necessary protos for this service into a directory Args:

proto_out_dir (str): Path to the directory to write proto files to

make_local_channel() → grpc.Channel

Return an insecure grpc channel over localhost for this server. Useful for unit testing or running local inference.

static _load_secret(secret: str) → str

If the secret points to a file, return the contents (plaintext reads). Else return the string

__enter__()

__exit__(type_, value, traceback)

caikit.runtime.grpc_server.main(blocking: bool = True)