Scorer

ScorerBase

RelevanceScorer

class ml4ir.base.model.scoring.scoring_model.RelevanceScorer(model_config: dict, feature_config: ml4ir.base.features.feature_config.FeatureConfig, interaction_model: ml4ir.base.model.scoring.interaction_model.InteractionModel, loss: ml4ir.base.model.losses.loss_base.RelevanceLossBase, file_io: ml4ir.base.io.file_io.FileIO, aux_loss: Optional[ml4ir.base.model.losses.loss_base.RelevanceLossBase] = None, aux_loss_weight: float = 0.0, aux_metrics: Optional[List[Union[keras.metrics.base_metric.Metric, str]]] = None, output_name: str = 'score', logger: Optional[logging.Logger] = None, logs_dir: Optional[str] = '', **kwargs)

Bases: keras.engine.training.Model

Base Scorer class that defines the neural network layers that convert the input features into scores

Defines the feature transformation layer(InteractionModel), dense neural network layers combined with activation layers and the loss function

Notes

  • This is a Keras model subclass and is built recursively using keras Layer instances
  • This is an abstract class. In order to use a Scorer, one must define and override the architecture_op and the final_activation_op functions

Constructor method for creating a RelevanceScorer object

Parameters:
  • model_config (dict) – Dictionary defining the model layer configuration
  • feature_config (FeatureConfig object) – FeatureConfig object defining the features and their configurations
  • interaction_model (InteractionModel object) – InteractionModel that defines the feature transformation layers on the input model features
  • loss (RelevanceLossBase object) – Relevance loss object that defines the final activation layer and the loss function for the model
  • file_io (FileIO object) – FileIO object that handles read and write
  • aux_loss (RelevanceLossBase object) – Auxiliary loss to be used in conjunction with the primary loss
  • aux_loss_weight (float) – Floating point number in [0, 1] to indicate the proportion of the auxiliary loss in the total final loss value computed using a linear combination total loss = (1 - aux_loss_weight) * loss + aux_loss_weight * aux_loss
  • aux_metrics (List of keras.metrics.Metric) – Keras metric list to be computed on the aux label
  • output_name (str, optional) – Name of the output that captures the score computed by the model
  • logger (Logger, optional) – Logging handler
  • logs_dir (str, optional) – Path to the logging directory

Notes

logs_dir : Used to point model architectures to local logging directory,
primarily for saving visualizations.
classmethod from_model_config_file(model_config_file: str, interaction_model: ml4ir.base.model.scoring.interaction_model.InteractionModel, loss: ml4ir.base.model.losses.loss_base.RelevanceLossBase, file_io: ml4ir.base.io.file_io.FileIO, aux_loss: Optional[ml4ir.base.model.losses.loss_base.RelevanceLossBase] = None, aux_loss_weight: float = 0.0, output_name: str = 'score', feature_config: Optional[ml4ir.base.features.feature_config.FeatureConfig] = None, logger: Optional[logging.Logger] = None, **kwargs)

Get a Scorer object from a YAML model config file

Parameters:
  • model_config_file (str) – Path to YAML file defining the model layer configuration
  • feature_config (FeatureConfig object) – FeatureConfig object defining the features and their configurations
  • interaction_model (InteractionModel object) – InteractionModel that defines the feature transformation layers on the input model features
  • loss (RelevanceLossBase object) – Relevance loss object that defines the final activation layer and the loss function for the model
  • file_io (FileIO object) – FileIO object that handles read and write
  • aux_loss (RelevanceLossBase object) – Auxiliary loss to be used in conjunction with the primary loss
  • aux_loss_weight (float) – Floating point number in [0, 1] to indicate the proportion of the auxiliary loss in the total final loss value computed using a linear combination total loss = (1 - aux_loss_weight) * loss + aux_loss_weight * aux_loss
  • output_name (str, optional) – Name of the output that captures the score computed by the model
  • logger (Logger, optional) – Logging handler
Returns:

RelevanceScorer object that computes the scores from the input features of the model
Return type:

RelevanceScorer object
plot_abstract_model()

Visualize the model architecture if defined by the architecture op

call(inputs: Dict[str, tensorflow.python.framework.ops.Tensor], training=None)

Compute score from input features

Parameters:inputs (dict of tensors) – Dictionary of input feature tensors
Returns:scores – Tensor object of the score computed by the model
Return type:dict of tensor object
get_architecture_op()

Get the model architecture instance based on the configs

compile(**kwargs)

Compile the keras model and defining a loss metrics to track any custom loss

train_step(data)

Defines the operations performed within a single training step. Called implicitly by tensorflow-keras when using model.fit()

Parameters:data (tuple of tensor objects) – Tuple of features and corresponding labels to be used to learn the model weights
Returns:Dictionary of metrics and loss computed for this training step
Return type:dict
test_step(data)

Defines the operations performed within a single prediction or evaluation step. Called implicitly by tensorflow-keras when using model.predict() or model.evaluate()

Parameters:data (tuple of tensor objects) – Tuple of features and corresponding labels to be used to evaluate the model
Returns:Dictionary of metrics and loss computed for this evaluation step
Return type:dict
metrics

Get the metrics for the keras model along with the custom loss metric