caikit.runtime.interceptors.caikit_runtime_server_wrapper
Attributes
Classes
This class wraps an underlying gRPC server for the purpose of |
Module Contents
- caikit.runtime.interceptors.caikit_runtime_server_wrapper.IN_PROGRESS_GAUGE
- class caikit.runtime.interceptors.caikit_runtime_server_wrapper.CaikitRuntimeServerWrapper(server, rpc_callable: Callable | Dict[caikit.runtime.service_generation.rpcs.CaikitRPCBase, Callable], intercepted_svc_package: caikit.runtime.service_factory.ServicePackage, service_type: caikit.runtime.names.ServiceType)[source]
Bases:
grpc.ServerThis class wraps an underlying gRPC server for the purpose of intercepting the binding of servicers (e.g., the CaikitRuntimeServicer) to the server so that the RPC handlers that are registered to the server can optionally be replaced with a generic global predict RPC handler instead.
- _server
- _service_type
- _rpc_callable
- _intercepted_svc_package
- _intercepted_methods = []
- intercepted_service()[source]
Get the fully-qualified name of the intercepted service
- Returns:
- string:
The fully-qualified name of the service whose RPC handlers are intercepted by this server wrapper
- intercepted_methods()[source]
Get the list of intercepted predict RPC methods
- Returns:
- list((string, string)):
A list of two-element tuples containing the short name (e.g., ‘SyntaxIzumoPredict’) and fully-qualified name (e.g., ‘/natural_language_understanding.CaikitRuntime/SyntaxIzumoPredict’) of every RPC method intercepted by this server wrapper
- static safe_rpc_wrapper(rpc: Callable, caikit_rpc: caikit.runtime.service_generation.rpcs.CaikitRPCBase | None = None)[source]
This wrapper should be used to safely invoke an RPC. If used, it adds automatic error handling and conversion to the appropriate response for gRPC, as well as logging indicating if the the error was intentional (i.e., thrown as CaikitRuntimeException directly) or unexpected (i.e., thrown as a non GRPC error).
- Args:
rpc(Function): Method attached to a servicer instance to be invoked in a safe manner.
- Returns:
A function that takes a gRPC request message and ServicerContext, and safely invokes the provided RPC.
- add_generic_rpc_handlers(generic_rpc_handlers)[source]
Registers GenericRpcHandlers with this Server.
This method will intercept the generic_rpc_handlers
- Args:
generic_rpc_handlers: An iterable of GenericRpcHandlers that will be used to service RPCs.
- _make_new_handler(original_rpc_handler: grpc._utilities.RpcMethodHandler, caikit_rpc: caikit.runtime.service_generation.rpcs.CaikitRPCBase | None = None)[source]
- add_insecure_port(address)[source]
Opens an insecure port for accepting RPCs.
This method may only be called before starting the server.
- Args:
address: The address for which to open a port. if the port is 0, or not specified in the address, then gRPC runtime will choose a port.
- Returns:
integer: An integer port on which server will accept RPC requests.
- add_secure_port(address, server_credentials)[source]
Opens a secure port for accepting RPCs.
This method may only be called before starting the server.
- Args:
- address: The address for which to open a port.
if the port is 0, or not specified in the address, then gRPC runtime will choose a port.
server_credentials: A ServerCredentials object.
- Returns:
integer: An integer port on which server will accept RPC requests.
- start()[source]
Starts this Server.
This method may only be called once. (i.e. it is not idempotent).
- stop(grace)[source]
Stops this Server.
This method immediately stop service of new RPCs in all cases.
If a grace period is specified, this method returns immediately and all RPCs active at the end of the grace period are aborted. If a grace period is not specified (by passing None for grace), all existing RPCs are aborted immediately and this method blocks until the last RPC handler terminates.
This method is idempotent and may be called at any time. Passing a smaller grace value in a subsequent call will have the effect of stopping the Server sooner (passing None will have the effect of stopping the server immediately). Passing a larger grace value in a subsequent call will not have the effect of stopping the server later (i.e. the most restrictive grace value is used).
- Args:
grace: A duration of time in seconds or None.
- Returns:
A threading.Event that will be set when this Server has completely stopped, i.e. when running RPCs either complete or are aborted and all handlers have terminated.
- wait_for_termination(timeout=None)[source]
Block current thread until the server stops. The wait will not consume computational resources during blocking, and it will block until one of the two following conditions are met: 1. The server is stopped or terminated; 2. A timeout occurs if timeout is not None.
The timeout argument works in the same way as threading.Event.wait(). Args:
timeout: A floating point number specifying a timeout for the operation in seconds.
- Returns:
A bool indicates if the operation times out.