caikit.runtime.client
=====================

.. py:module:: caikit.runtime.client

.. autoapi-nested-parse::

   This module holds common utilities for connecting to caikit.runtime servers from
   client code



Submodules
----------

.. toctree::
   :maxdepth: 1

   /autoapi/caikit/runtime/client/remote_config/index
   /autoapi/caikit/runtime/client/remote_model_finder/index
   /autoapi/caikit/runtime/client/remote_model_initializer/index
   /autoapi/caikit/runtime/client/remote_module_base/index
   /autoapi/caikit/runtime/client/utils/index


Classes
-------

.. autoapisummary::

   caikit.runtime.client.RemoteModuleConfig
   caikit.runtime.client.RemoteModelFinder
   caikit.runtime.client.RemoteModelInitializer
   caikit.runtime.client.RemoteModuleBase


Package Contents
----------------

.. py:class:: RemoteModuleConfig(config_dict)

   Bases: :py:obj:`caikit.core.modules.config.ModuleConfig`


   Helper class to differentiate a local ModuleConfig and a RemoteModuleConfig. The structure
   should contain the following fields/structure


   .. py:attribute:: connection
      :type:  caikit.interfaces.common.data_model.remote.ConnectionInfo


   .. py:attribute:: protocol
      :type:  str


   .. py:attribute:: model_key
      :type:  str


   .. py:attribute:: task_methods
      :type:  List[Tuple[Type[caikit.core.task.TaskBase], List[RemoteRPCDescriptor]]]


   .. py:attribute:: train_method
      :type:  RemoteRPCDescriptor


   .. py:attribute:: model_path
      :type:  str


   .. py:attribute:: module_id
      :type:  str


   .. py:attribute:: module_name
      :type:  str


   .. py:attribute:: reserved_keys
      :value: []



   .. py:method:: load_from_module(module_reference: Union[str, Type[caikit.core.modules.base.ModuleBase], caikit.core.modules.base.ModuleBase], connection_info: caikit.interfaces.common.data_model.remote.ConnectionInfo, protocol: str, model_key: str, model_path: str) -> RemoteModuleConfig
      :classmethod:


      Construct a new remote module configuration from an existing local Module

      Args:
          module_reference: Union[str, Type[ModuleBase]]:
              Module_reference should either be the id of the locally loaded module,
              or a module class

          model_path (str):
              The path used to load this module

          connection_info ConnectionInfo:
              The connection information of the remote to use

          protocol: str
              The protocol to connect with

          model_key: str
              The model key to use when sending GRPC requests. An example is mm-model-id

         Returns:
             model_config (RemoteModuleConfig): Instantiated RemoteModuleConfig for
             model given model_path.



.. py:class:: RemoteModelFinder(config: aconfig.Config, instance_name: str)

   Bases: :py:obj:`caikit.core.model_management.model_finder_base.ModelFinderBase`


   A class can be constructed by a factory if its constructor takes exactly
   one argument that is an aconfig.Config object and it has a name to identify
   itself with the factory.


   .. py:attribute:: __doc__
      :value: Multiline-String

      .. raw:: html

         <details><summary>Show Value</summary>

      .. code-block:: python

         """
         The RemoteModelFinder locates models that are loaded in a remote runtime.
         
         Configuration for RemoteModelFinder lives under the config as follows:
         
         model_management:
             finders:
                 <finder name>:
                     type: REMOTE
                     config:
                         connection: ConnectionInfo <Required Connection Information>
                         model_key: Optional[str]=MODEL_MESH_MODEL_ID_KEY <Optional setting to override the
                             grpc model name>
                         protocol: Optional[str]="grpc" <protocol the remote server is using (grpc or http)>
                         min_poll_time: Optional[int]=30 <minimum time before attempting to rediscover
                             models>
                         discover_models: Optional[bool]=True <bool to automatically discover remote models
                             via the /info/models endpoint>
                         supported_models: Optional[Dict[str, str]]={} <mapping of model names to module_ids
                             that this remote supports. This is automatically populated by discover_models>
                             <model_path>: <module_id>
         
         """

      .. raw:: html

         </details>




   .. py:attribute:: name
      :value: 'REMOTE'


      This is the name of this constructible type that will be used by
      the factory to identify this class



   .. py:attribute:: _instance_name


   .. py:attribute:: _connections
      :type:  Dict[str, caikit.interfaces.common.data_model.remote.ConnectionInfo]


   .. py:attribute:: _connection_template
      :type:  Optional[caikit.interfaces.common.data_model.remote.ConnectionInfo]
      :value: None



   .. py:attribute:: _model_key


   .. py:attribute:: _protocol


   .. py:attribute:: _supported_models
      :type:  Dict[str, ModuleConnectionInfo]


   .. py:attribute:: _discover_models


   .. py:attribute:: _min_poll_time


   .. py:method:: find_model(model_path: str, **__) -> Optional[caikit.runtime.client.remote_config.RemoteModuleConfig]

      Check if the remote runtime supports the model_path



   .. py:method:: _discover(model_name: Optional[str] = None) -> Dict[str, ModuleConnectionInfo]

      Helper method to discover models from a remote
      runtime. This is a separate function to help with subclassing

      Returns:
          model_map: Dict[str, str]
              The map of models to modules



   .. py:method:: _safe_discover(model_name: Optional[str] = None) -> Dict[str, ModuleConnectionInfo]

      Helper function that lazily discovers models in a
      thread safe manor. This function also ensures we don't overload
      the remote server with discovery requests

      Returns:
          Dict[str, str]: Result of discover_models



   .. py:method:: _discover_grpc_models(model_name: Optional[str]) -> Dict[str, ModuleConnectionInfo]

      Helper function to get all the supported models and modules
      from a remote GRPC runtime

      Returns:
         support_models: Dict[str, str
             Mapping of remote model names to module ids



   .. py:method:: _discover_http_models(model_name: Optional[str]) -> Dict[str, caikit.interfaces.common.data_model.remote.ConnectionInfo]

      Helper function to get all the supported models and modules
      from a remote HTTP runtime

      Returns:
          supported_models:Dict[str, str]
              Mapping of remote model names to module_ids



   .. py:method:: _render_conn_template(model_name: str) -> Optional[caikit.interfaces.common.data_model.remote.ConnectionInfo]

      Common utility to get the connection for a given model



   .. py:method:: _get_conn_candidates(model_name: Optional[str]) -> List[caikit.interfaces.common.data_model.remote.ConnectionInfo]

      Common utility to get all connections to try



.. py:class:: RemoteModelInitializer(config: aconfig.Config, instance_name: str)

   Bases: :py:obj:`caikit.core.model_management.model_initializer_base.ModelInitializerBase`


   A class can be constructed by a factory if its constructor takes exactly
   one argument that is an aconfig.Config object and it has a name to identify
   itself with the factory.


   .. py:attribute:: __doc__
      :value: Multiline-String

      .. raw:: html

         <details><summary>Show Value</summary>

      .. code-block:: python

         """
         The RemoteModelInitializer loads a RemoteModuleConfig as an empty Module that
         sends all requests to an external runtime server
         
         Configuration for RemoteModelInitializer lives under the config as follows:
         
         model_management:
             initializers:
                 <initializer name>:
                     type: REMOTE
         """

      .. raw:: html

         </details>




   .. py:attribute:: name
      :value: 'REMOTE'


      This is the name of this constructible type that will be used by
      the factory to identify this class



   .. py:attribute:: _instance_name


   .. py:attribute:: _module_class_map


   .. py:method:: init(model_config: caikit.runtime.client.remote_config.RemoteModuleConfig, **kwargs) -> Optional[caikit.core.modules.ModuleBase]

      Given a RemoteModuleConfig, initialize a RemoteModule instance



   .. py:method:: construct_module_class(model_config: caikit.runtime.client.remote_config.RemoteModuleConfig) -> Type[caikit.core.modules.ModuleBase]

      Helper function to construct a ModuleClass. This is a separate function to allow
       for easy overloading

       Args:
           model_config: RemoteModuleConfig
              The model config to construct the module from

      Returns:
          module: Type[ModuleBase]
              The constructed module



.. py:class:: RemoteModuleBase(connection_info: caikit.interfaces.common.data_model.ConnectionInfo, protocol: str, model_key: str, model_name: str)

   Bases: :py:obj:`caikit.core.modules.ModuleBase`


   Class to act as the base for remote modules. This class will be subclassed and
   mutated by construct_remote_module_class to make it have the same functions and parameters
   as the source module.


   .. py:attribute:: _model_name


   .. py:attribute:: _connection


   .. py:attribute:: _tls


   .. py:attribute:: _protocol


   .. py:attribute:: _model_key


   .. py:attribute:: _channel_lock


   .. py:attribute:: _conn_channel
      :type:  Optional[Union[grpc.Channel, requests.Session]]
      :value: None



   .. py:attribute:: _current_conn_time
      :value: None



   .. py:attribute:: _max_conn_delta


   .. py:method:: __del__()

      Destructor to ensure channel/session is cleaned up on deletion



   .. py:method:: generate_train_function(method: caikit.runtime.client.remote_config.RemoteRPCDescriptor) -> Callable
      :classmethod:


      Factory function to construct a train function that will then be set as an attribute



   .. py:method:: generate_inference_function(task: Type[caikit.core.task.TaskBase], method: caikit.runtime.client.remote_config.RemoteRPCDescriptor) -> Callable
      :classmethod:


      Factory function to construct inference functions that will be set as an attribute.



   .. py:method:: remote_method_request(method: caikit.runtime.client.remote_config.RemoteRPCDescriptor, service_type: caikit.runtime.names.ServiceType, *args, **kwargs) -> Any

      Function to run a remote request based on the data stored in RemoteRPCDescriptor



   .. py:method:: _request_via_http(method: caikit.runtime.client.remote_config.RemoteRPCDescriptor, service_type: caikit.runtime.names.ServiceType, *args, **kwargs) -> Any


   .. py:method:: _request_via_grpc(method: caikit.runtime.client.remote_config.RemoteRPCDescriptor, service_type: caikit.runtime.names.ServiceType, *args, **kwargs) -> Any

      Helper function to send a grpc request



   .. py:property:: _grpc_channel
      :type: grpc.Channel


      Helper function to construct a GRPC channel
      with correct credentials and TLS settings.



   .. py:property:: _http_session
      :type: requests.Session


      Helper function to construct a requests Session with
      with correct credentials and TLS settings.



   .. py:method:: _get_remote_object(construction_fn: Callable[[None], Union[grpc.Channel, requests.Session]]) -> Union[grpc.Channel, requests.Session]

      Helper function to control construction of a grpc channel or http session

      Args:
          construction_fn (Callable[[None], Union[grpc.Channel, Session]]): _description_

      Returns:
          Union[grpc.Channel, Session]: _description_



   .. py:method:: _get_remote_target() -> str

      Get the current remote target



   .. py:method:: _get_streaming_arguments(**kwargs: Dict[str, Any]) -> List[str]
      :staticmethod:


      Helper function to detect which kwargs are streaming



   .. py:method:: _rename_union_sequence_types(obj: Any, dm_type: type)
      :staticmethod:


      Helper function that renames all references in a dictionary
      to match the oneOf value of the DataModel and to collapse all Primitive
      sequences. This is required to match the format of http requests

      For example:
         {
             "union_str": "test",
             "ints": {
                 "values":[1,2,3]
             }
         }

      Becomes:
          {
              "union": "test",
              "ints":[1,2,3]
          }