neuralmonkey.evaluators package¶
Submodules¶
neuralmonkey.evaluators.accuracy module¶
neuralmonkey.evaluators.bleu module¶
-
class
neuralmonkey.evaluators.bleu.
BLEUEvaluator
(n=4, deduplicate=False, name=None)¶ Bases:
object
-
static
bleu
(hypotheses, references, ngrams=4, case_sensitive=True)¶ Computes BLEU on a corpus with multiple references using uniform weights. Default is to use smoothing as in reference implementation on: https://github.com/ufal/qtleap/blob/master/cuni_train/bin/mteval-v13a.pl#L831-L873
Parameters: - hypotheses – List of hypotheses
- references – LIst of references. There can be more than one reference.
- ngram – Maximum order of n-grams. Default 4.
- case_sensitive – Perform case-sensitive computation. Default True.
-
static
compare_scores
(score1, score2)¶
-
static
effective_reference_length
(hypotheses, references_list)¶ Computes the effective reference corpus length (based on best match length)
Parameters: - hypotheses – List of output sentences as lists of words
- references – List of lists of references (as lists of words)
-
static
merge_max_counters
(counters: typing.List[collections.Counter]) → collections.Counter¶ Merge counters using maximum values
-
static
minimum_reference_length
(hypotheses, references_list)¶ Computes the effective reference corpus length (based on the shortest reference sentence length)
Parameters: - hypotheses – List of output sentences as lists of words
- references – List of lists of references (as lists of words)
-
static
modified_ngram_precision
(hypotheses: typing.List[typing.List[str]], references_list: typing.List[typing.List[typing.List[str]]], n: int, case_sensitive: bool) → typing.Tuple[float, int]¶ Computes the modified n-gram precision on a list of sentences
Parameters: - hypothesis – List of output sentences as lists of words
- references – List of lists of reference sentences (as lists of words)
- n – n-gram order
- case_sensitive – Whether to perform case-sensitive computation
-
static
ngram_counts
(sentence: typing.List[str], n: int, lowercase: bool, delimiter=' ') → collections.Counter¶ Get n-grams from a sentence
Parameters: - sentence – Sentence as a list of words
- n – n-gram order
- lowercase – Convert ngrams to lowercase
- delimiter – delimiter to use to create counter entries
-
static
neuralmonkey.evaluators.bleu_ref module¶
neuralmonkey.evaluators.edit_distance module¶
neuralmonkey.evaluators.perplexity module¶
-
class
neuralmonkey.evaluators.perplexity.
Perplexity
(name='Perplexity')¶ Bases:
object