neuralmonkey.evaluators.sacrebleu module

class neuralmonkey.evaluators.sacrebleu.SacreBLEUEvaluator(name: str, smooth: str = 'exp', smooth_floor: float = 0.0, force: bool = False, lowercase: bool = False, tokenize: str = 'none', use_effective_order: bool = False) → None

Bases: neuralmonkey.evaluators.evaluator.Evaluator

SacreBLEU evaluator wrapper.

__init__(name: str, smooth: str = 'exp', smooth_floor: float = 0.0, force: bool = False, lowercase: bool = False, tokenize: str = 'none', use_effective_order: bool = False) → None

Initialize self. See help(type(self)) for accurate signature.

score_batch(hypotheses: List[List[str]], references: List[List[str]]) → float

Score a batch of hyp/ref pairs.

The default implementation of this method calls score_instance for each instance in the batch and returns the average score.

Parameters:
  • hypotheses – List of model predictions.
  • references – List of golden outputs.
Returns:

A float.