neuralmonkey.decoders package¶
Submodules¶
neuralmonkey.decoders.decoder module¶
-
class
neuralmonkey.decoders.decoder.
Decoder
(encoders: typing.List[typing.Any], vocabulary: neuralmonkey.vocabulary.Vocabulary, data_id: str, name: str, max_output_len: int, dropout_keep_prob: float = 1.0, rnn_size: typing.Union[int, NoneType] = None, embedding_size: typing.Union[int, NoneType] = None, output_projection: typing.Union[typing.Callable[[tensorflow.python.framework.ops.Tensor, tensorflow.python.framework.ops.Tensor, typing.List[tensorflow.python.framework.ops.Tensor]], tensorflow.python.framework.ops.Tensor], NoneType] = None, encoder_projection: typing.Union[typing.Callable[[tensorflow.python.framework.ops.Tensor, typing.Union[int, NoneType], typing.Union[typing.List[typing.Any], NoneType]], tensorflow.python.framework.ops.Tensor], NoneType] = None, use_attention: bool = False, embeddings_encoder: typing.Any = None, attention_on_input: bool = True, rnn_cell: str = 'GRU', conditional_gru: bool = False, save_checkpoint: typing.Union[str, NoneType] = None, load_checkpoint: typing.Union[str, NoneType] = None) → None¶ Bases:
neuralmonkey.model.model_part.ModelPart
A class that manages parts of the computation graph that are used for the decoding.
-
feed_dict
(dataset: neuralmonkey.dataset.Dataset, train: bool = False) → typing.Dict[tensorflow.python.framework.ops.Tensor, typing.Any]¶ Populate the feed dictionary for the decoder object
Parameters: - dataset – The dataset to use for the decoder.
- train – Boolean flag, telling whether this is a training run
-
get_attention_object
(encoder, train_mode: bool)¶
-
neuralmonkey.decoders.encoder_projection module¶
This module contains different variants of projection of encoders into the initial state of the decoder.
-
neuralmonkey.decoders.encoder_projection.
concat_encoder_projection
(train_mode: tensorflow.python.framework.ops.Tensor, rnn_size: typing.Union[int, NoneType] = None, encoders: typing.Union[typing.List[typing.Any], NoneType] = None) → tensorflow.python.framework.ops.Tensor¶ Create the initial state by concatenating the encoders’ encoded values
Parameters: - train_mode – tf 0-D bool Tensor specifying the training mode (not used)
- rnn_size – The size of the resulting vector (not used)
- encoders – The list of encoders
-
neuralmonkey.decoders.encoder_projection.
empty_initial_state
(train_mode: tensorflow.python.framework.ops.Tensor, rnn_size: typing.Union[int, NoneType], encoders: typing.Union[typing.List[typing.Any], NoneType] = None) → tensorflow.python.framework.ops.Tensor¶ Return an empty vector
Parameters: - train_mode – tf 0-D bool Tensor specifying the training mode (not used)
- rnn_size – The size of the resulting vector
- encoders – The list of encoders (not used)
-
neuralmonkey.decoders.encoder_projection.
linear_encoder_projection
(dropout_keep_prob: float) → typing.Callable[[tensorflow.python.framework.ops.Tensor, typing.Union[int, NoneType], typing.Union[typing.List[typing.Any], NoneType]], tensorflow.python.framework.ops.Tensor]¶ Return a projection function which applies dropout on concatenated encoder final states and returns a linear projection to a rnn_size-sized tensor.
Parameters: dropout_keep_prob – The dropout keep probability
neuralmonkey.decoders.multi_decoder module¶
-
class
neuralmonkey.decoders.multi_decoder.
MultiDecoder
(main_decoder, regularization_decoders)¶ Bases:
object
The MultiDecoder class wraps a several child decoders into one parent encoder. The Neural Monkey architecture requires the model to have only one decoder, so this class can be used when more than one output sequence should be generated (i.e. multi-task learning).
The multi decoder object composes of one main decoder and an arbitrary number of additional decoders, called ‘regularization’ decoders.
The reason for this division is that during validation, we need to report a single score of the model as a whole, and based on this score, the training process decides whether to save the model variables or not.
So if the task is translation with POS tagging of the source sentence, the main decoder should be the decoder that generates the target sentence, whereas the sequence labeler used for POS tagging should be included in the regularization decoders list.
During training, the multi decoder works in the following way: According to the value of the
_input_selector
placeholder, the loss corresponds to one of the child decoders (so in multi-task learning, the weights in each batch are updated with respect only to one sub-task). It is therefore a good practice to alternate between batches of different task. This is because we often do not have the training data that cover all tasks in one corpus.-
all_decoded
()¶
-
cost
¶
-
data_id
¶
-
decoded
¶
-
feed_dict
(dataset, train=False)¶ Populate the feed dictionary for the decoder object
- Decoder placeholders:
input_selector
: the index of the child decoder used for- computing the loss
-
learning_step
¶
-
runtime_loss
¶
-
train_loss
¶
-
vocabulary
¶
-
vocabulary_size
¶
-
neuralmonkey.decoders.output_projection module¶
This module contains different variants of projection functions for RNN outputs.
-
neuralmonkey.decoders.output_projection.
maxout_output
(maxout_size)¶ Compute RNN output out of the previous state and output, and the context tensors returned from attention mechanisms, as described in the article
This function corresponds to the equations for computation the t_tilde in the Bahdanau et al. (2015) paper, on page 14, with the maxout projection, before the last linear projection.
Parameters: maxout_size – The size of the hidden maxout layer in the deep output Returns: Returns the maxout projection of the concatenated inputs
-
neuralmonkey.decoders.output_projection.
mlp_output
(layer_sizes, dropout_plc=None, activation=<function tanh>)¶ Compute RNN deep output using the multilayer perceptron with a specified activation function. (Pascanu et al., 2013 [https://arxiv.org/pdf/1312.6026v5.pdf])
Parameters: - layer_sizes – A list of sizes of the hiddel layers of the MLP
- dropout_plc – Dropout placeholder. TODO this is not going to work with current configuration
- activation – The activation function to use in each layer.
-
neuralmonkey.decoders.output_projection.
no_deep_output
(prev_state, prev_output, ctx_tensors)¶ Compute RNN output out of the previous state and output, and the context tensors returned from attention mechanisms.
This function corresponds to the equations for computation the t_tilde in the Bahdanau et al. (2015) paper, on page 14, before the linear projection.
Parameters: - prev_state – Previous decoder RNN state. (Denoted s_i-1)
- prev_output – Embedded output of the previous step. (y_i-1)
- ctx_tensors – Context tensors computed by the attentions. (c_i)
Returns: This function returns the concatenation of all its inputs.
neuralmonkey.decoders.sequence_classifier module¶
-
class
neuralmonkey.decoders.sequence_classifier.
SequenceClassifier
(name: str, encoders: typing.List[typing.Any], vocabulary: neuralmonkey.vocabulary.Vocabulary, data_id: str, layers: typing.List[int], activation_fn: typing.Callable[[tensorflow.python.framework.ops.Tensor], tensorflow.python.framework.ops.Tensor] = <function relu>, dropout_keep_prob: float = 0.5, save_checkpoint: typing.Union[str, NoneType] = None, load_checkpoint: typing.Union[str, NoneType] = None) → None¶ Bases:
neuralmonkey.model.model_part.ModelPart
A simple MLP classifier over encoders.
The API pretends it is an RNN decoder which always generates a sequence of length exactly one.
-
decoded
¶
-
feed_dict
(dataset: neuralmonkey.dataset.Dataset, train: bool = False) → typing.Dict[tensorflow.python.framework.ops.Tensor, typing.Any]¶
-
runtime_loss
¶
-
train_loss
¶
-
neuralmonkey.decoders.sequence_labeler module¶
-
class
neuralmonkey.decoders.sequence_labeler.
SequenceLabeler
(name: str, encoder: neuralmonkey.encoders.sentence_encoder.SentenceEncoder, vocabulary: neuralmonkey.vocabulary.Vocabulary, data_id: str, dropout_keep_prob: float = 1.0, save_checkpoint: typing.Union[str, NoneType] = None, load_checkpoint: typing.Union[str, NoneType] = None) → None¶ Bases:
neuralmonkey.model.model_part.ModelPart
Classifier assing a label to each encoder’s state.
-
cost
¶
-
decoded
¶
-
feed_dict
(dataset: neuralmonkey.dataset.Dataset, train: bool = False) → typing.Dict[tensorflow.python.framework.ops.Tensor, typing.Any]¶
-
logits
¶
-
logprobs
¶
-
runtime_loss
¶
-
train_loss
¶
-
neuralmonkey.decoders.word_alignment_decoder module¶
-
class
neuralmonkey.decoders.word_alignment_decoder.
WordAlignmentDecoder
(encoder: neuralmonkey.encoders.sentence_encoder.SentenceEncoder, decoder: neuralmonkey.decoders.decoder.Decoder, data_id: str, name: str) → None¶ Bases:
neuralmonkey.model.model_part.ModelPart
A decoder that computes soft alignment from an attentive encoder. Loss is computed as cross-entropy against a reference alignment.
-
cost
¶
-
feed_dict
(dataset: neuralmonkey.dataset.Dataset, train: bool = False) → typing.Dict[tensorflow.python.framework.ops.Tensor, typing.Any]¶
-