caikit.runtime.servicers.model_runtime_servicer

Attributes

log

Classes

ModelRuntimeServicerImpl

This class contains the implementation of all of the RPCs that are required to run a

Module Contents

caikit.runtime.servicers.model_runtime_servicer.log[source]

class caikit.runtime.servicers.model_runtime_servicer.ModelRuntimeServicerImpl(interrupter: caikit.runtime.work_management.abortable_context.ThreadInterrupter = None)[source]

Bases: caikit.runtime.protobufs.model_runtime_pb2_grpc.ModelRuntimeServicer

This class contains the implementation of all of the RPCs that are required to run a service in Model Mesh as a Model-Runtime.

model_manager = None

interrupter = None

loadModel(request, context)[source]

Model loading . Args:

request(model_runtime_pb2.LoadModelRequest):
gRPC request, gen from model-runtime.proto

context(grpc.ServicerContext):
Context object (contains request metadata, etc)

Returns:: model_runtime_pb2.LoadModelResponse: Gen from model-runtime.proto

unloadModel(request, context)[source]

Model unloading.

Args:

request(model_runtime_pb2.UnloadModelRequest):: gRPC request, gen from model-runtime.proto
context(grpc.ServicerContext):: Context object (contains request metadata, etc)

Returns:

model_runtime_pb2.UnloadModelResponse: Gen from model-runtime.proto

predictModelSize(request, context)[source]

Predict size of not-yet-loaded model

Args:

request(model_runtime_pb2.PredictModelSizeRequest):: gRPC request, gen from model-runtime.proto
context(grpc._server._Context):: Context object (contains request metadata, etc)

Returns:

model_runtime_pb2.PredictModelSizeResponse: Gen from model-runtime.proto

modelSize(request, context)[source]

Compute size (memory consumption) of currently-loaded model

Args:

request(model_runtime_pb2.ModelSizeRequest):: gRPC request, gen from model-runtime.proto
context(grpc._server._Context):: Context object (contains request metadata, etc)

Returns:

model_runtime_pb2.ModelSizeResponse: Gen from model-runtime.proto

runtimeStatus(request, context)[source]

Runtime status checking.

Args:

request(model_runtime_pb2.RuntimeStatusRequest):: GRPC request, gen from model-runtime.proto
context(grpc.ServicerContext):: Context object (contains request metadata, etc)

Returns:

model_runtime_pb2.RuntimeStatusResponse:: Gen from model-runtime.proto