neuralmonkey.evaluators.chrf module

class neuralmonkey.evaluators.chrf.ChrFEvaluator(n: int = 6, beta: float = 1.0, ignored_symbols: List[str] = None, name: str = None) → None

Bases: neuralmonkey.evaluators.evaluator.Evaluator

Compute ChrF score.

See http://www.statmt.org/wmt15/pdf/WMT49.pdf

__init__(n: int = 6, beta: float = 1.0, ignored_symbols: List[str] = None, name: str = None) → None

Initialize self. See help(type(self)) for accurate signature.

chr_p(hyp_ngrams: List[Dict[str, int]], ref_ngrams: List[Dict[str, int]]) → float

chr_r(hyp_ngrams: List[Dict[str, int]], ref_ngrams: List[Dict[str, int]]) → float

score_instance(hypothesis: List[str], reference: List[str]) → float

Score a single hyp/ref pair.

The default implementation of this method returns 1.0 when the hypothesis and the reference are equal and 0.0 otherwise.

Parameters:

• hypothesis – The model prediction.
• reference – The golden output.

Returns:

A float.