neuralmonkey.learning_utils module

neuralmonkey.learning_utils.evaluation(evaluators, batch, runners, execution_results, result_data)

Evaluate the model outputs.

Parameters:

• evaluators – List of tuples of series and evaluation functions.
• batch – Batch of data against which the evaluation is done.
• runners – List of runners (contains series ids and loss names).
• execution_results – Execution results that include the loss values.
• result_data – Dictionary from series names to list of outputs.

Returns:

Dictionary of evaluation names and their values which includes the metrics applied on respective series loss and loss values from the run.

neuralmonkey.learning_utils.initialize_model(tf_manager: neuralmonkey.tf_manager.TensorFlowManager, initial_variables: Union[List[str], NoneType], executables: List[neuralmonkey.runners.base_runner.GraphExecutor])

neuralmonkey.learning_utils.log_model_variables(trainers: List[Union[neuralmonkey.trainers.generic_trainer.GenericTrainer, neuralmonkey.trainers.multitask_trainer.MultitaskTrainer]]) → None

neuralmonkey.learning_utils.print_final_evaluation(eval_result: Dict[str, float], name: str = None) → None

Print final evaluation from a test dataset.

neuralmonkey.learning_utils.run_on_dataset(tf_manager: neuralmonkey.tf_manager.TensorFlowManager, runners: List[neuralmonkey.runners.base_runner.BaseRunner], dataset_runner: neuralmonkey.runners.dataset_runner.DatasetRunner, dataset: neuralmonkey.dataset.Dataset, postprocess: Union[List[Tuple[str, Callable]], NoneType], batching_scheme: neuralmonkey.dataset.BatchingScheme, write_out: bool = False, log_progress: int = 0) → Tuple[List[neuralmonkey.runners.base_runner.ExecutionResult], Dict[str, List], Dict[str, List]]

Apply the model on a dataset and optionally write outputs to files.

This function processes the dataset in batches and optionally prints out the execution progress.

Parameters:

• tf_manager – TensorFlow manager with initialized sessions.
• runners – A function that runs the code
• dataset_runner – A runner object that fetches the data inputs
• dataset – The dataset on which the model will be executed.
• evaluators – List of evaluators that are used for the model evaluation if the target data are provided.
• postprocess – Dataset-level postprocessors
• write_out – Flag whether the outputs should be printed to a file defined in the dataset object.
• batching_scheme – Scheme used for batching.
• log_progress – log progress every X seconds
• extra_fetches – Extra tensors to evaluate for each batch.

Returns:

Tuple of resulting sentences/numpy arrays, and evaluation results if they are available which are dictionary function -> value.

neuralmonkey.learning_utils.training_loop(cfg: argparse.Namespace) → None

Execute the training loop for given graph and data.

Parameters:cfg – Experiment configuration namespace.