Plugins

AIMM system relies on user-written plugins for implementations of machine learning models, preprocessing and data-fetching methods. Both server and its clients may use the plugin API. The plugins are implemented as Python modules that are dynamically imported at some point during servers or clients runtime. Within these modules, plugin implementators should use the decorators from the aimm.plugins module to indicate which functions and classes are entry points to various machine learning workflow actions. They may also, through decorator arguments, pass information to plugin callers, allowing them to further declare semantics of some of their arguments (e.g. declare that an argument should receive a callback function for progress notification).

Before accessing any plugins, they need to be imported and initialized. One way to do this (other then importing them manually) is by calling the initialization function:

aimm.plugins.initialize(conf: Dict)

Imports the plugin modules, registering the entry point functions

Parameters:: conf – configuration that follows schema under id aimm://plugins/schema.yaml#

Initialize configuration schema:

---
$schema: 'http://json-schema.org/schema#'
id: 'aimm://plugins.yaml#'
type: object
required:
    - names
properties:
    names:
        type: array
        items:
            type: string
            description: Python module name
...

Warning

The AIMM plugin interface does not create a sandbox environment when executing plugins, it is up to the implementator to make sure not to perform unsafe actions

Decorators

@aimm.plugins.data_access(name: str, state_cb_arg_name: str | None = None) → Callable

Decorator used to indicate that the wrapped function is a data access function. The decorated function can take any number of positional and keyword arguments and should return the accessed data.

Parameters:

name – name of the data access type
state_cb_arg_name – if set, indicates that the caller should pass a state callback function as a keyword argument and use the passed value as the argument name. The function is of type Callable[Any], where the only argument is JSON serializable data.

Returns:

Decorated function

@aimm.plugins.instantiate(model_type: str, state_cb_arg_name: str | None = None) → Callable

Decorator used to indicate that the wrapped function is a model instance creation function. The decorated function should take any number of positional and keyword arguments and should return the newly created model instance.

Parameters:

model_type – name of the model type
state_cb_arg_name – if set, indicates that the caller should pass a state callback function as a keyword argument and use the passed value as the argument name. The function is of type Callable[Any].

Returns:

Decorated function

@aimm.plugins.fit(model_types: List[str], state_cb_arg_name: str | None = None, instance_arg_name: str | None = None) → Callable

Decorator used to indicate that the wrapped function is a fitting function. The decorated function should take at least one argument - model instance (passed as the first positional argument by default). It may also take any number of additional positional and keyword arguments and should return the updated model instance.

Parameters:

model_types – types of models supported by the decorated function
state_cb_arg_name – if set, indicates that the caller should pass a state callback function as a keyword argument and use the passed value as the argument name. The function is of type Callable[Any]
instance_arg_name – if set, indicates under which argument name to pass the concrete model instance. If not set, it is passed in the first positional argument

Returns:

Decorated function

@aimm.plugins.predict(model_types: List[str], state_cb_arg_name: str | None = None, instance_arg_name: str | None = None) → Callable

Decorator used to indicate that the wrapped function is a prediction function. The decorated function should take at least one argument - model instance (passed as the first positional argument by default). It may also take any number of additional positional and keyword arguments and should return the updated model instance.

Parameters:

model_types – types of models supported by the decorated function
state_cb_arg_name – if set, indicates that the caller should pass a state callback function as a keyword argument and use the passed value as the argument name. The function is of type Callable[Any]
instance_arg_name – if set, indicates under which argument name to pass the concrete model instance. If not set, it is passed in the first positional argument

Returns:

Decorated function

@aimm.plugins.serialize

Decorator used to indicate that the wrapped function is a serialize function. The decorated function should have the following signature:

(instance: Any) -> ByteString

The return value is the byte representation of the model instance.

Parameters:: model_types – types of models supported by the decorated function
Returns:: Decorated function

@aimm.plugins.deserialize

Decorator used to indicate that the wrapped function is a deserialize function. The decorated function should have the following signature:

(instance_bytes: ByteString) -> Any

The return value is the deserialized model instance.

Parameters:: model_types – types of models supported by the decorated function
Returns:: Decorated function

It is common for a model type to have all of the above defined functions, for this reason the following class and decorator are introduced:

class aimm.plugins.Model

Interface unifying multiple plugin entry points under same type. __init__ method is treated as instantiation function.

abstract fit(*args: Any, **kwargs: Any) → Any: Fit method for model instances

abstract predict(*args: Any, **kwargs: Any) → Any: Predict method for model instances

abstract serialize() → ByteString: Serialize method for model instances

abstract classmethod deserialize(instance_bytes: ByteString) → Model: Deserialize method for model instances

aimm.plugins.model(cls: Type) → Type

Model class decorator, used to mark that a class may be used as a model implementation. Model class unifies different plugin actions (instantiate() as __init__, fit(), predict(), serialize(), deserialize() as their same-named class methods) under the same model type (module + class name). The class should implement the Model interface.

Parameters:: cls (Model) – model class
Returns:: Decorated class

Calling plugins

After implementing and loading plugins, they can be called using following functions:

aimm.plugins.exec_data_access(name: str, state_cb: ~typing.Callable[[~typing.Dict], None] = <function <lambda>>, *args: ~typing.Any, **kwargs: ~typing.Any) → Any: Uses a loaded plugin to access data

aimm.plugins.exec_instantiate(model_type: str, state_cb: ~typing.Callable[[~typing.Dict], None] = <function <lambda>>, *args: ~typing.Any, **kwargs: ~typing.Any) → Any: Uses a loaded plugin to create a model instance

aimm.plugins.exec_fit(model_type: str, instance: ~typing.Any, state_cb: ~typing.Callable[[~typing.Dict], None] = <function <lambda>>, *args: ~typing.Any, **kwargs: ~typing.Any) → Any: Uses a loaded plugin to fit a model instance

aimm.plugins.exec_predict(model_type: str, instance: ~typing.Any, state_cb: ~typing.Callable[[~typing.Dict], None] = <function <lambda>>, *args: ~typing.Any, **kwargs: ~typing.Any) → tuple[Any, Any]: Uses a loaded plugin to perform a prediction with a given model instance. Also returns the instance because it might be altered during the prediction, e.g. with reinforcement learning models.

aimm.plugins.exec_serialize(model_type: str, instance: Any) → ByteString: Uses a loaded plugin to convert model into bytes

aimm.plugins.exec_deserialize(model_type: str, instance_bytes: ByteString) → Any: Uses a loaded plugin to convert bytes into a model instance

State callbacks have the following signature:

aimm.plugins.StateCallback

Generic state callback function signature a plugin would receive

alias of Callable[[Dict], None]