neuralmonkey.learning_utils module

neuralmonkey.learning_utils.evaluation(evaluators, batch, runners, execution_results, result_data)

Evaluate the model outputs.

  • evaluators – List of tuples of series and evaluation functions.
  • batch – Batch of data against which the evaluation is done.
  • runners – List of runners (contains series ids and loss names).
  • execution_results – Execution results that include the loss values.
  • result_data – Dictionary from series names to list of outputs.

Dictionary of evaluation names and their values which includes the metrics applied on respective series loss and loss values from the run.

neuralmonkey.learning_utils.initialize_model(tf_manager: neuralmonkey.tf_manager.TensorFlowManager, initial_variables: Union[List[str], NoneType], executables: List[neuralmonkey.runners.base_runner.GraphExecutor])
neuralmonkey.learning_utils.log_model_variables(trainers: List[Union[neuralmonkey.trainers.generic_trainer.GenericTrainer, neuralmonkey.trainers.multitask_trainer.MultitaskTrainer]]) → None
neuralmonkey.learning_utils.print_final_evaluation(eval_result: Dict[str, float], name: str = None) → None

Print final evaluation from a test dataset.

neuralmonkey.learning_utils.run_on_dataset(tf_manager: neuralmonkey.tf_manager.TensorFlowManager, runners: List[neuralmonkey.runners.base_runner.BaseRunner], dataset_runner: neuralmonkey.runners.dataset_runner.DatasetRunner, dataset: neuralmonkey.dataset.Dataset, postprocess: Union[List[Tuple[str, Callable]], NoneType], batching_scheme: neuralmonkey.dataset.BatchingScheme, write_out: bool = False, log_progress: int = 0) → Tuple[List[neuralmonkey.runners.base_runner.ExecutionResult], Dict[str, List], Dict[str, List]]

Apply the model on a dataset and optionally write outputs to files.

This function processes the dataset in batches and optionally prints out the execution progress.

  • tf_manager – TensorFlow manager with initialized sessions.
  • runners – A function that runs the code
  • dataset_runner – A runner object that fetches the data inputs
  • dataset – The dataset on which the model will be executed.
  • evaluators – List of evaluators that are used for the model evaluation if the target data are provided.
  • postprocess – Dataset-level postprocessors
  • write_out – Flag whether the outputs should be printed to a file defined in the dataset object.
  • batching_scheme – Scheme used for batching.
  • log_progress – log progress every X seconds
  • extra_fetches – Extra tensors to evaluate for each batch.

Tuple of resulting sentences/numpy arrays, and evaluation results if they are available which are dictionary function -> value.

neuralmonkey.learning_utils.training_loop(cfg: argparse.Namespace) → None

Execute the training loop for given graph and data.

Parameters:cfg – Experiment configuration namespace.