neuralmonkey.processors package

Submodules

neuralmonkey.processors.alignment module

class neuralmonkey.processors.alignment.WordAlignmentPreprocessor(source_len, target_len, dtype=<class 'numpy.float32'>, normalize=True, zero_based=True)

Bases: object

A preprocessor for word alignments in a text format.

One of the following formats is expected:

s1-t1 s2-t2 ...

s1:1/w1 s2:t2/w2 ...

where each s and t is the index of a word in the source and target sentence, respectively, and w is the corresponding weight. If the weight is not given, it is assumend to be 1. The separators - and : are interchangeable.

The output of the preprocessor is an alignment matrix of the fixed shape (target_len, source_len) for each sentence.

neuralmonkey.processors.bpe module

class neuralmonkey.processors.bpe.BPEPostprocessor(**kwargs)

Bases: object

decode(sentence)
class neuralmonkey.processors.bpe.BPEPreprocessor(**kwargs)

Bases: object

Wrapper class for Byte-Pair-Encoding from Edinburgh

neuralmonkey.processors.editops module

neuralmonkey.processors.german module

class neuralmonkey.processors.german.GermanPostprocessor(compounding=True, contracting=True, pronouns=True)

Bases: object

decode(sentence)
class neuralmonkey.processors.german.GermanPreprocessor(compounding=True, contracting=True, pronouns=True)

Bases: object

neuralmonkey.processors.helpers module

neuralmonkey.processors.helpers.pipeline(processors: typing.List[typing.Callable]) → typing.Callable

Concatenate processors.

neuralmonkey.processors.helpers.postprocess_char_based(sentence: typing.List[str]) → typing.List[str]
neuralmonkey.processors.helpers.preprocess_char_based(sentence: typing.List[str]) → typing.List[str]
neuralmonkey.processors.helpers.untruecase(sentences: typing.List[typing.List[str]]) → typing.Generator[[typing.List[str], NoneType], NoneType]

Module contents