mistk.evaluation.service module

class mistk.evaluation.service.EvaluationPluginEndpoint

Bases: object

add_task(task)

Adds a task to this EvaluationPluginService and starts it using a pool of worker threads

Parameters

task – The Task to register and start

Returns

The running task

delete_task(task)

Deletes the specified Task. This function is currently not supported

Parameters

task – The task to delete

evaluate(initParams)

Performs the evaluation defined for this plugin

Parameters

initParams (dict | bytes) – Init Parameters for the evaluation. Based on EvaluationSpecificationInitParams specification

Return type

None

property evaluation_plugin

Retrieves the model associated with this EvaluationPluginEndpoint

Returns

the model

get_api_version()

Returns the version of the MISTK API

Returns

The MISTK API version as text

get_metrics()

Returns metrics that can be evaluated for this plugin

Return type

List[Metric]

get_status(watch=None, resourceVersion=None)

Retrieves the status of the evaluation plugin

Return type

EvaluationInstanceStatus

load_metrics_spec(module)

Gets the plug-in manager for defined metrics of the module specified

Parameters

module – The name of the module

Returns

the plugin manager

start_server(port=8080)

Starts the http server of the Service

Parameters

port – The port on which to start the server, defaults to 8080

property state_machine

Retrieves the state machine associated with this EvaluationPluginEndpoint

Returns

the state machine

terminate()

Shutdowns the evaluation plugin and cleans up any resources.

Return type

None

update_state(state=None, payload=None)

Updates the state of the EvaluationEndpointService

Parameters

• state – The new state. If not given, the current state will be used.

• payload – Additional data to attach to the state

class mistk.evaluation.service.EvaluationPluginTask(operation, parameters=None, submitted=None, completed=None, status=None, message=None)

Bases: object

A task to be submitted to the Evaluation Plugin instance that will be performed and report status of.

mistk.evaluation.service.initializeEndpointController(handler, *modules)

This method iterates over the functions defined in the auto-generated flask modules And creates a function in this package which points bound member methods of the handler

Parameters

• handler – The handler object which must implement bound member methods which otherwise have the same signature as those defined in the controller_modules

• modules – These are the autogenerated swagger controller modules