neuralmonkey.encoders package

Submodules

neuralmonkey.encoders.attentive module

class neuralmonkey.encoders.attentive.Attentive(attention_type, **kwargs)

Bases: object

A base class fro an attentive part of graph (typically encoder).

Objects inheriting this class are able to generate an attention object that allows a decoder to perform attention over an attention_object provided by the encoder (e.g., input word representations in case of MT or convolutional maps in case of image captioning).

create_attention_object()

Attention object that can be used in decoder.

neuralmonkey.encoders.cnn_encoder module

CNN for image processing.

class neuralmonkey.encoders.cnn_encoder.CNNEncoder(name: str, data_id: str, convolutions: typing.List[typing.Tuple[int, int, typing.Union[int, NoneType]]], image_height: int, image_width: int, pixel_dim: int, fully_connected: typing.Union[typing.List[int], NoneType] = None, dropout_keep_prob: float = 0.5, attention_type: typing.Type = <class 'neuralmonkey.decoding_function.Attention'>, save_checkpoint: typing.Union[str, NoneType] = None, load_checkpoint: typing.Union[str, NoneType] = None) → None

Bases: neuralmonkey.model.model_part.ModelPart, neuralmonkey.encoders.attentive.Attentive

An image encoder.

It projects the input image through a serie of convolutioal operations. The projected image is vertically cut and fed to stacked RNN layers which encode the image into a single vector.

encoded

Output vector of the CNN.

If there are specified some fully connected layers, there are applied on top of the last convolutional map. Dropout is applied between all layers, default activation function is ReLU. There are only projection layers, no softmax is applied.

If there is fully_connected layer specified, average-pooled last convolutional map is used as a vector output.

feed_dict(dataset: neuralmonkey.dataset.Dataset, train: bool = False) → typing.Dict[tensorflow.python.framework.ops.Tensor, typing.Any]

image_input

image_processing_layers

Do all convolutions and return the last conditional map.

Applies convolutions on the input tensor with optional max pooling. All the intermediate layers are stored in the image_processing_layers attribute. There is not dropout between the convolutional layers, by default the activation function is ReLU.

states

train_mode

neuralmonkey.encoders.encoder_wrapper module

Attention combination strategies.

This modules implements attention combination strategies for multi-encoder scenario when we may want to combine the hidden states of the encoders in more complicated fashion.

Currently there are two attention combination strategies flat and hierarchical (see paper Attention Combination Strategies for Multi-Source Sequence-to-Sequence Learning).

The combination strategies may use the sentinel mechanism which allows the decoder not to attend to the, and extract information on its own hidden state (see paper Knowing when to Look: Adaptive Attention via a Visual Sentinel for Image Captioning).

class neuralmonkey.encoders.encoder_wrapper.EncoderWrapper(name: str, encoders: typing.List[typing.Any], attention_type: typing.Type, attention_state_size: int, use_sentinels=False, share_attn_projections=False) → None

Bases: neuralmonkey.model.model_part.ModelPart, neuralmonkey.encoders.attentive.Attentive

Wrapper doing attention combination behaving as a single encoder.

This class wraps encoders and performs the attention combination in such a way that for the decoder, it looks like a single encoder capable to generate a single context vector.

create_attention_object()

feed_dict(dataset: neuralmonkey.dataset.Dataset, train: bool) → typing.Dict[tensorflow.python.framework.ops.Tensor, typing.Any]

class neuralmonkey.encoders.encoder_wrapper.FlatMultiAttention(*args, **kwargs)

Bases: neuralmonkey.encoders.encoder_wrapper.MultiAttention

Flat attention combination strategy.

Using this attention combination strategy, hidden states of the encoders are first projected to the same space (different projection for different encoders) and then we compute a joint distribution over all the hidden states. The context vector is then a weighted sum of another / then projection of the encoders hidden states. The sentinel vector can be added as an additional hidden state.

See equations 8 to 10 in the Attention Combination Strategies paper.

attention(decoder_state: tensorflow.python.framework.ops.Tensor, decoder_prev_state: tensorflow.python.framework.ops.Tensor, decoder_input: tensorflow.python.framework.ops.Tensor, loop_state: neuralmonkey.decoding_function.AttentionLoopState, step: tensorflow.python.framework.ops.Tensor) → typing.Tuple[tensorflow.python.framework.ops.Tensor, neuralmonkey.decoding_function.AttentionLoopState]

finalize_loop(key: str, last_loop_state: neuralmonkey.decoding_function.AttentionLoopState) → None

get_encoder_projections(scope)

initial_loop_state() → neuralmonkey.decoding_function.AttentionLoopState

class neuralmonkey.encoders.encoder_wrapper.HierarchicalLoopState(child_loop_states, loop_state)

Bases: tuple

child_loop_states

Alias for field number 0

loop_state

Alias for field number 1

class neuralmonkey.encoders.encoder_wrapper.HierarchicalMultiAttention(*args, **kwargs) → None

Bases: neuralmonkey.encoders.encoder_wrapper.MultiAttention

Hierarchical attention combination.

Hierarchical attention combination strategy first computes the context vector for each encoder separately using whatever attention type the encoders have. After that it computes a second attention over the resulting context vectors and optionally the sentinel vector.

See equations 6 and 7 in the Attention Combination Strategies paper.

attention(decoder_state: tensorflow.python.framework.ops.Tensor, decoder_prev_state: tensorflow.python.framework.ops.Tensor, decoder_input: tensorflow.python.framework.ops.Tensor, loop_state: neuralmonkey.encoders.encoder_wrapper.HierarchicalLoopState, step: tensorflow.python.framework.ops.Tensor) → typing.Tuple[tensorflow.python.framework.ops.Tensor, neuralmonkey.encoders.encoder_wrapper.HierarchicalLoopState]

finalize_loop(key: str, last_loop_state: typing.Any) → None

initial_loop_state() → neuralmonkey.encoders.encoder_wrapper.HierarchicalLoopState

class neuralmonkey.encoders.encoder_wrapper.MultiAttention(encoders: typing.List[neuralmonkey.encoders.attentive.Attentive], attention_state_size: int, scope: typing.Union[tensorflow.python.ops.variable_scope.VariableScope, str], share_projections: bool = False, use_sentinels: bool = False) → None

Bases: neuralmonkey.decoding_function.BaseAttention

Base class for attention combination.

attention(decoder_state, decoder_prev_state, decoder_input, _, step)

Get context vector for given decoder state.

attn_size

neuralmonkey.encoders.facebook_conv module

From the paper Convolutional Sequence to Sequence Learning

http://arxiv.org/abs/1705.03122

class neuralmonkey.encoders.facebook_conv.SentenceEncoder(name: str, input_sequence: neuralmonkey.model.sequence.EmbeddedSequence, conv_features: int, encoder_layers: int, kernel_width: int = 5, dropout_keep_prob: float = 1.0, attention_type: type = None, attention_state_size: int = None, attention_fertility: int = 3, save_checkpoint: str = None, load_checkpoint: str = None) → None

Bases: neuralmonkey.model.model_part.ModelPart, neuralmonkey.encoders.attentive.Attentive

encoded

feed_dict(dataset: neuralmonkey.dataset.Dataset, train: bool = False) → typing.Dict[tensorflow.python.framework.ops.Tensor, typing.Any]

order_embeddings

ordered_embedded_inputs

states

states_mask

train_mode

neuralmonkey.encoders.imagenet_encoder module

Pre-trained ImageNet networks.

class neuralmonkey.encoders.imagenet_encoder.ImageNet(name: str, data_id: str, network_type: str, attention_layer: typing.Union[str, NoneType] = None, attention_state_size: typing.Union[int, NoneType] = None, attention_type: typing.Type = <class 'neuralmonkey.decoding_function.Attention'>, fine_tune: bool = False, encoded_layer: typing.Union[str, NoneType] = None, load_checkpoint: typing.Union[str, NoneType] = None, save_checkpoint: typing.Union[str, NoneType] = None) → None

Bases: neuralmonkey.model.model_part.ModelPart, neuralmonkey.encoders.attentive.Attentive

Pre-trained ImageNet network.

HEIGHT = 224

WIDTH = 224

cnn_states

encoded

feed_dict(dataset: neuralmonkey.dataset.Dataset, train: bool = False) → typing.Dict[tensorflow.python.framework.ops.Tensor, typing.Any]

input_image

states

neuralmonkey.encoders.numpy_encoder module

class neuralmonkey.encoders.numpy_encoder.PostCNNImageEncoder(name: str, input_shape: typing.List[int], output_shape: int, data_id: str, attention_type: typing.Callable = None, save_checkpoint: typing.Union[str, NoneType] = None, load_checkpoint: typing.Union[str, NoneType] = None) → None

Bases: neuralmonkey.model.model_part.ModelPart, neuralmonkey.encoders.attentive.Attentive

feed_dict(dataset: neuralmonkey.dataset.Dataset, train: bool = False) → typing.Dict[tensorflow.python.framework.ops.Tensor, typing.Any]

class neuralmonkey.encoders.numpy_encoder.VectorEncoder(name: str, dimension: int, data_id: str, output_shape: int = None, save_checkpoint: str = None, load_checkpoint: str = None) → None

Bases: neuralmonkey.model.model_part.ModelPart

feed_dict(dataset: neuralmonkey.dataset.Dataset, train: bool = False) → typing.Dict[tensorflow.python.framework.ops.Tensor, typing.Any]

neuralmonkey.encoders.raw_rnn_encoder module

class neuralmonkey.encoders.raw_rnn_encoder.RNNSpec(size, direction, cell_type)

Bases: tuple

cell_type

Alias for field number 2

direction

Alias for field number 1

size

Alias for field number 0

class neuralmonkey.encoders.raw_rnn_encoder.RawRNNEncoder(name: str, data_id: str, input_size: int, rnn_layers: typing.List[typing.Union[typing.Tuple[int], typing.Tuple[int, str], typing.Tuple[int, str, str]]], max_input_len: typing.Union[int, NoneType] = None, dropout_keep_prob: float = 1.0, attention_type: typing.Any = None, save_checkpoint: typing.Union[str, NoneType] = None, load_checkpoint: typing.Union[str, NoneType] = None) → None

Bases: neuralmonkey.model.model_part.ModelPart, neuralmonkey.encoders.attentive.Attentive

A raw RNN encoder that gets input as a tensor.

feed_dict(dataset: neuralmonkey.dataset.Dataset, train: bool = False) → typing.Dict[tensorflow.python.framework.ops.Tensor, typing.Any]

Populate the feed dictionary with the encoder inputs.

Parameters:

• dataset – The dataset to use
• train – Boolean flag telling whether it is training time

neuralmonkey.encoders.recurrent module

class neuralmonkey.encoders.recurrent.FactoredEncoder(name: str, vocabularies: typing.List[neuralmonkey.vocabulary.Vocabulary], data_ids: typing.List[str], embedding_sizes: typing.List[int], rnn_size: int, max_input_len: int = None, dropout_keep_prob: float = 1.0, rnn_cell: str = 'GRU', attention_type: type = None, attention_fertility: int = 3, attention_state_size: int = None, save_checkpoint: str = None, load_checkpoint: str = None) → None

Bases: neuralmonkey.encoders.recurrent.RecurrentEncoder

class neuralmonkey.encoders.recurrent.RecurrentEncoder(name: str, input_sequence: neuralmonkey.model.sequence.Sequence, rnn_size: int, dropout_keep_prob: float = 1.0, rnn_cell: str = 'GRU', attention_type: type = None, attention_state_size: int = None, attention_fertility: int = 3, save_checkpoint: str = None, load_checkpoint: str = None) → None

Bases: neuralmonkey.model.model_part.ModelPart, neuralmonkey.encoders.attentive.Attentive

bidirectional_rnn

encoded

feed_dict(dataset: neuralmonkey.dataset.Dataset, train: bool = False) → typing.Dict[tensorflow.python.framework.ops.Tensor, typing.Any]

states

states_mask

train_mode

class neuralmonkey.encoders.recurrent.SentenceEncoder(name: str, vocabulary: neuralmonkey.vocabulary.Vocabulary, data_id: str, embedding_size: int, rnn_size: int, max_input_len: int = None, dropout_keep_prob: float = 1.0, rnn_cell: str = 'GRU', attention_type: type = None, attention_fertility: int = 3, attention_state_size: int = None, save_checkpoint: str = None, load_checkpoint: str = None) → None

Bases: neuralmonkey.encoders.recurrent.RecurrentEncoder

neuralmonkey.encoders.sentence_cnn_encoder module

Encoder for sentences withou explicit segmentation.

class neuralmonkey.encoders.sentence_cnn_encoder.SentenceCNNEncoder(name: str, input_sequence: neuralmonkey.model.sequence.Sequence, segment_size: int, highway_depth: int, rnn_size: int, filters: typing.List[typing.Tuple[int, int]], dropout_keep_prob: float = 1.0, attention_type: type = None, attention_fertility: int = 3, use_noisy_activations: bool = False, save_checkpoint: typing.Union[str, NoneType] = None, load_checkpoint: typing.Union[str, NoneType] = None) → None

Bases: neuralmonkey.model.model_part.ModelPart, neuralmonkey.encoders.attentive.Attentive

Encoder processing a sentence using a CNN then running a bidirectional RNN on the result.

Based on: Jason Lee, Kyunghyun Cho, Thomas Hofmann: Fully Character-Level Neural Machine Translation without Explicit Segmentation (https://arxiv.org/pdf/1610.03017.pdf)

bidirectional_rnn

cnn_encoded

1D convolution with max-pool that processing characters.

encoded

feed_dict(dataset: neuralmonkey.dataset.Dataset, train: bool = False) → typing.Dict[tensorflow.python.framework.ops.Tensor, typing.Any]

Populate the feed dictionary with the encoder inputs.

Parameters:

• dataset – The dataset to use
• train – Boolean flag telling whether it is training time

highway_layer

Highway net projection following the CNN.

rnn_cells() → typing.Tuple[tensorflow.python.ops.rnn_cell_impl._RNNCell, tensorflow.python.ops.rnn_cell_impl._RNNCell]

Return the graph template to for creating RNN memory cells

states

train_mode

neuralmonkey.encoders.sequence_cnn_encoder module

Encoder for sentence classification with 1D convolutions and max-pooling.

class neuralmonkey.encoders.sequence_cnn_encoder.SequenceCNNEncoder(name: str, vocabulary: neuralmonkey.vocabulary.Vocabulary, data_id: str, embedding_size: int, filters: typing.List[typing.Tuple[int, int]], max_input_len: typing.Union[int, NoneType] = None, dropout_keep_prob: float = 1.0, save_checkpoint: typing.Union[str, NoneType] = None, load_checkpoint: typing.Union[str, NoneType] = None) → None

Bases: neuralmonkey.model.model_part.ModelPart

Encoder processing a sequence using a CNN.

embedded_inputs

encoded

feed_dict(dataset: neuralmonkey.dataset.Dataset, train: bool = False) → typing.Dict[tensorflow.python.framework.ops.Tensor, typing.Any]

Populate the feed dictionary with the encoder inputs.

Encoder input placeholders:

encoder_input: Stores indices to the vocabulary,

shape (batch, time)

encoder_padding: Stores the padding (ones and zeros,

indicating valid words and positions after the end of sentence, shape (batch, time)

train_mode: Boolean scalar specifying the mode (train

vs runtime)

Parameters:

• dataset – The dataset to use
• train – Boolean flag telling whether it is training time

input_mask

inputs

train_mode

Module contents