caikit.core.toolkit.concurrency.destroyable_process

A DestroyableProcess implements a multiprocessing Process that captures errors and communicates them to its parent with clean semantics for being destroyed.

NOTE: The “fork” start method is used for two reasons:

1. It’s faster

2. For auto-generated classes (e.g. stream sources), “spawn” can result in

   missing classes since it performs a full re-import, but may not regenerate these classes.

Attributes

log
error
OOM_EXIT_CODES
FORK_CTX
SPAWN_CTX
FORKSERVER_CTX
_PROCESS_TYPES

Classes

_DestroyableProcess	The _DestroyableProcess base class implements a context-agnostic process
_ForkDestroyableProcess	Process objects represent activity that is run in a separate process
_SpawnDestroyableProcess	Process objects represent activity that is run in a separate process
_ForkserverDestroyableProcess	Process objects represent activity that is run in a separate process

Functions

DestroyableProcess(start_method, *args, **kwargs)

Class wrapper that returns the appropriate process type based on the

Module Contents

caikit.core.toolkit.concurrency.destroyable_process.log

caikit.core.toolkit.concurrency.destroyable_process.error

caikit.core.toolkit.concurrency.destroyable_process.OOM_EXIT_CODES

caikit.core.toolkit.concurrency.destroyable_process.FORK_CTX = None

caikit.core.toolkit.concurrency.destroyable_process.SPAWN_CTX = None

caikit.core.toolkit.concurrency.destroyable_process.FORKSERVER_CTX = None

class caikit.core.toolkit.concurrency.destroyable_process._DestroyableProcess(target: Callable | None = None, completion_event: multiprocessing.Event | None = None, args: Tuple | None = None, kwargs: dict | None = None, destroy_grace_period: float = 10, return_result: bool = False, **_kwargs)

Bases: multiprocessing.process.BaseProcess, caikit.core.toolkit.concurrency.destroyable.Destroyable

The _DestroyableProcess base class implements a context-agnostic process class that manages the subprocess and allows it to be destroyed

_completion_event

_destroy_grace_period = 10

_return_result = False

__result = None

__started = False

__destroyed = False

__canceled = False

property destroyed: bool

Return True if destroy was called, regardless of whether the destroyable was alive at the time

property canceled: bool

Returns True if destroyed while actively working

property ran: bool

Return True if the destroyable completed execution in any state

property threw: bool

Return True if any exception was raised during execution

get_or_throw() → Any

Get the result of the execution or raise an error if one occurred

destroy()

Cancel any in-progress work

start()

Start child process

join(*args, **kwargs)

Wait until child process terminates

run()

Method to be run in sub-process; can be overridden in sub-class

_update_result()

property error: Exception | None

Return the error information to user if one occurred

property completion_event: multiprocessing.Event

_target_wrapper(target, args, kwargs)

caikit.core.toolkit.concurrency.destroyable_process._PROCESS_TYPES

class caikit.core.toolkit.concurrency.destroyable_process._ForkDestroyableProcess(group=None, target=None, name=None, args=(), kwargs={}, *, daemon=None)

Bases: FORK_CTX, _DestroyableProcess

Process objects represent activity that is run in a separate process

The class is analogous to threading.Thread

_MP_CTX

class caikit.core.toolkit.concurrency.destroyable_process._SpawnDestroyableProcess(group=None, target=None, name=None, args=(), kwargs={}, *, daemon=None)

Bases: SPAWN_CTX, _DestroyableProcess

Process objects represent activity that is run in a separate process

The class is analogous to threading.Thread

_MP_CTX

class caikit.core.toolkit.concurrency.destroyable_process._ForkserverDestroyableProcess(group=None, target=None, name=None, args=(), kwargs={}, *, daemon=None)

Bases: FORKSERVER_CTX, _DestroyableProcess

Process objects represent activity that is run in a separate process

The class is analogous to threading.Thread

_MP_CTX

caikit.core.toolkit.concurrency.destroyable_process.DestroyableProcess(start_method: str, *args, **kwargs)

Class wrapper that returns the appropriate process type based on the requested start method.

NOTE: Naming intentionally looks like a class!