neuralmonkey.processors.bpe module¶
-
class
neuralmonkey.processors.bpe.
BPEPostprocessor
(separator: str = '@@') → None¶ Bases:
object
-
__init__
(separator: str = '@@') → None¶ Initialize self. See help(type(self)) for accurate signature.
-
decode
(sentence: List[str]) → List[str]¶
-
-
class
neuralmonkey.processors.bpe.
BPEPreprocessor
(merge_file: str, separator: str = '@@', encoding: str = 'utf-8') → None¶ Bases:
object
Wrapper class for Byte-Pair Encoding.
Paper: https://arxiv.org/abs/1508.07909 Code: https://github.com/rsennrich/subword-nmt
-
__init__
(merge_file: str, separator: str = '@@', encoding: str = 'utf-8') → None¶ Initialize self. See help(type(self)) for accurate signature.
-