neuralmonkey.evaluators.beer module

class neuralmonkey.evaluators.beer.BeerWrapper(wrapper: str, name: str = 'BEER', encoding: str = 'utf-8') → None

Bases: neuralmonkey.evaluators.evaluator.Evaluator

Wrapper for BEER scorer.

Paper: http://aclweb.org/anthology/D14-1025 Code: https://github.com/stanojevic/beer

__init__(wrapper: str, name: str = 'BEER', encoding: str = 'utf-8') → None

Initialize the BEER wrapper.

Parameters:
  • name – Name of the evaluator.
  • wrapper – Path to the BEER’s executable.
  • encoding – Data encoding.
score_batch(hypotheses: List[List[str]], references: List[List[str]]) → float

Score a batch of hyp/ref pairs.

The default implementation of this method calls score_instance for each instance in the batch and returns the average score.

Parameters:
  • hypotheses – List of model predictions.
  • references – List of golden outputs.
Returns:

A float.
serialize_to_bytes(sentences: List[List[str]]) → bytes