mistk.model.abstract_model module¶

class mistk.model.abstract_model.AbstractModel¶

Bases: object

The definition of an abstract Model to be implemented by Model developers.

build_model(path=None)¶

Triggers the model to enter the building_model state. A subsequent call to do_build_model with the given parameters will be made as a result.

This method should not be implemented or overwritten by subclasses. It will be created by the state machine.

Parameters: path – The path to the model file

abstract do_build_model(path=None)¶

Instructs the service to build all necessary data structures given the architecture and selected hyperparameters.

Parameters: path – The path to the model file

abstract do_generate()¶: Executes/resumes the generate activity This operation is currently not supported.

abstract do_initialize(objectives: list, props: dict, hparams: dict)¶

Called once the endpoint service has launched. This would typically be the first call made to the service.

Parameters

objective – The objectives for the model. Possible values include ‘training’, ‘prediction’, ‘streaming_prediction’, ‘generation’, and ‘transfer_learning’.
props – A dictionary that is parsed from a JSON string. These are settings that are passed from the ecosystem, but are not really considered hyperparameters. They are not used by the model, but rather the endpoint itself (e.g., where should heartbeats be sent and how often, etc).
hparams – A dictionary that is parsed from a JSON string. These are the hyperparameters that are used by the model.

abstract do_load_data(dataset_map: dict)¶

Instructs the container to load data (or at least record in memory where the data is, if it’s actually to be loaded during training).

Parameters: dataset_map – A dictionary that maps string keys {train, test, generation} to a Dataset object that contains information on the dataset to load

abstract do_pause()¶: Pauses the current activity (training or prediction). This operation is currently not supported.

abstract do_predict()¶: Executes/resumes the prediction activity This operation is currently not supported.

abstract do_reset()¶: Resets the model into its initial state

abstract do_resume_predict()¶: Executes/resumes the prediction activity

abstract do_resume_training()¶: Unpauses the current activity (training or prediction). This operation is currently not supported.

abstract do_save_generations(dataPath)¶

Saves the current generations to the location specified

Parameters: dataPath – The location on the local file system or distributed file system where the generations will be saved

abstract do_save_model(path)¶

Save the model to an agreed upon location. It will be up to some other process to make sure the saved model ends up back in the repository. This call could be a NOOP if, for instance, the model is saved periodically throughout the training process.

Parameters: path – The path to which the model should be saved.

abstract do_save_predictions(dataPath)¶

Saves the current prediction to the location specified

Parameters: dataPath – The location on the local file system or distributed file system where the predictions will be saved

abstract do_stream_predict(data_map: dict, details: bool = False)¶

Executes/resumes the prediction activity

Parameters

data_map – A dictionary of ID keys to base64 encoded data
details – Optional parameter for the model to provide additional details

abstract do_terminate()¶: Stops all processing and releases any resources that are in use in preparation for being shut down.

abstract do_train()¶: Executes/resumes the training activity

do_update_stream_properties(props: dict)¶

Updates the stream prediction properties

Parameters: props – A dictionary of streaming properties

property endpoint_service¶: Returns the endpoint service for this Model

fail(reason)¶

Triggers the model to enter the failed state. This method should not be implemented or overwritten by subclasses. It will be created by the state machine.

Parameters: reason – The reason why the state machine is transitioned into the failure state

generate()¶

Triggers the model to enter the generating state. A subsequent call to do_generate with the given parameters will be made as a result.

This method should not be implemented or overwritten by subclasses. It will be created by the state machine.

initialize(objectives, props, hparams)¶

Triggers the model to enter the initializing state. A subsequent call to do_initialize with the given parameters will be made as a result. This method should not be implemented or overwritten by subclasses. It will be created by the state machine.

Parameters

objectives – The objectives of this model. Possible values include ‘training’, ‘prediction’, ‘streaming_prediction’, and ‘transfer_learning’.
props – A dictionary that is parsed from a JSON string. These are settings that are passed from the ecosystem, but are not really considered hyperparameters. They are not used by the model, but rather the endpoint itself (e.g., where should heartbeats be sent and how often, etc).
hparams – A dictionary that is parsed from a JSON string. These are the hyperparameters that are used by the model.

initialized()¶

Triggers the model to enter the initialized state. It is expected that this method will only be called by the implementing model subclass.

This method should not be implemented or overwritten by subclasses. It will be created by the state machine.

load_data(dataset_map)¶

Triggers the model to enter the loading_data state. A subsequent call to do_load_data with the given parameters will be made as a result.

This method should not be implemented or overwritten by subclasses. It will be created by the state machine.

Parameters: dataset_map – A dictionary that maps string keys {train, test, generation} to a Dataset object that contains information on the dataset to load

new_state_entered(*args, **kwargs)¶

Notifies the endpoint service that the current state of the state machine has been update

Parameters

args – Optional non-keyworded variable length arguments to pass in
kwargs – Optional keyworded variable length arguments to pass in

pause()¶

Triggers the model to enter the pausing state. A subsequent call to do_pause with the given parameters will be made as a result.

This method should not be implemented or overwritten by subclasses. It will be created by the state machine.

paused()¶

Triggers the model to enter the paused state. It is expected that this method will only be called by the implementing model subclass.

This method should not be implemented or overwritten by subclasses. It will be created by the state machine.

predict()¶

Triggers the model to enter the predicting state. A subsequent call to do_predict with the given parameters will be made as a result.

This method should not be implemented or overwritten by subclasses. It will be created by the state machine.

ready()¶

Triggers the model to enter the ready state. It is expected that this method will only be called by the implementing model subclass.

This method should not be implemented or overwritten by subclasses. It will be created by the state machine.

report_failure(reason)¶

Reports the failure of a model during its operations

Parameters: reason – The error message explaining why the model failed.

reset()¶: Resets the model container instance

save_generations(dataPath)¶

Triggers the model to enter the saving_generations state. A subsequent call to do_save_generations with the given parameters will be made as a result.

This method should not be implemented or overwritten by subclasses. It will be created by the state machine.

Parameters: dataPath – The file path to where the model generations will be saved. This can be the local file system or on the distributed file system

save_model(path)¶

Triggers the model to enter the unpausing state. A subsequent call to do_unpause with the given parameters will be made as a result.

This method should not be implemented or overwritten by subclasses. It will be created by the state machine.

save_predictions(dataPath)¶

Triggers the model to enter the saving_predictions state. A subsequent call to do_save_predictions with the given parameters will be made as a result.

This method should not be implemented or overwritten by subclasses. It will be created by the state machine.

Parameters: dataPath – The file path to where the model predictions will be saved. This can be the local file system or on the distributed file system

stream_predict(data_map, details)¶

Triggers the model to enter the predicting state. A subsequent call to do_stream_predict with the given parameters will be made as a result.

This method should not be implemented or overwritten by subclasses. It will be created by the state machine.

Parameters

data_map – A dictionary of ID keys to base64 encoded data
details – Optional parameter for the model to provide additional details

terminated()¶: Terminates the model

train()¶

Triggers the model to enter the training state. A subsequent call to do_train with the given parameters will be made as a result.

This method should not be implemented or overwritten by subclasses. It will be created by the state machine.

unpause()¶

Triggers the model to enter the unpausing state. A subsequent call to do_unpause with the given parameters will be made as a result.

This method should not be implemented or overwritten by subclasses. It will be created by the state machine.

update_status(payload)¶

Provides additional details to the current state of the model. If the model is training, then this might contain the number of records that have been trained.

Parameters: payload – Extra information regarding the status update

update_stream_properties(props)¶

Triggers the model to update the streaming properties. A subsequent call to do_update_stream_properties with the given parameters will be made as a result.

This method should not be implemented or overwritten by subclasses. It will be created by the state machine.

Parameters: props – A dictionary of metadata properties to be used by the model