neuralmonkey.decoders.encoder_projection module¶
Encoder Projection Module.
This module contains different variants of projection of encoders into the initial state of the decoder.
Encoder projections are specified in the configuration file. Each encoder
projection function has a unified type EncoderProjection
, which is a
callable that takes three arguments:
train_mode
– boolean tensor specifying whether the train mode is onrnn_size
– the size of the resulting initial stateencoders
– a list ofStateful
objects used as the encoders.
To enable further parameterization of encoder projection functions, one can use higher-order functions.
-
neuralmonkey.decoders.encoder_projection.
concat_encoder_projection
(train_mode: tensorflow.python.framework.ops.Tensor, rnn_size: int = None, encoders: List[neuralmonkey.model.stateful.Stateful] = None) → tensorflow.python.framework.ops.Tensor¶ Concatenate the encoded values of the encoders.
-
neuralmonkey.decoders.encoder_projection.
empty_initial_state
(train_mode: tensorflow.python.framework.ops.Tensor, rnn_size: int, encoders: List[neuralmonkey.model.stateful.Stateful] = None) → tensorflow.python.framework.ops.Tensor¶ Return an empty vector.
-
neuralmonkey.decoders.encoder_projection.
linear_encoder_projection
(dropout_keep_prob: float) → Callable[[tensorflow.python.framework.ops.Tensor, int, List[neuralmonkey.model.stateful.Stateful]], tensorflow.python.framework.ops.Tensor]¶ Return a linear encoder projection.
Return a projection function which applies dropout on concatenated encoder final states and returns a linear projection to a rnn_size-sized tensor.
Parameters: dropout_keep_prob – The dropout keep probability
-
neuralmonkey.decoders.encoder_projection.
nematus_projection
(dropout_keep_prob: float = 1.0) → Callable[[tensorflow.python.framework.ops.Tensor, int, List[neuralmonkey.model.stateful.Stateful]], tensorflow.python.framework.ops.Tensor]¶ Return encoder projection used in Nematus.
The initial state is a dense projection with tanh activation computed on the averaged states of the encoders. Dropout is applied to the means (before the projection).
Parameters: dropout_keep_prob – The dropout keep probability.