neuralmonkey.decoders.encoder_projection module

Encoder Projection Module.

This module contains different variants of projection of encoders into the initial state of the decoder.

Encoder projections are specified in the configuration file. Each encoder projection function has a unified type EncoderProjection, which is a callable that takes three arguments:

  1. train_mode – boolean tensor specifying whether the train mode is on
  2. rnn_size – the size of the resulting initial state
  3. encoders – a list of Stateful objects used as the encoders.

To enable further parameterization of encoder projection functions, one can use higher-order functions.

neuralmonkey.decoders.encoder_projection.concat_encoder_projection(train_mode: tensorflow.python.framework.ops.Tensor, rnn_size: int = None, encoders: List[neuralmonkey.model.stateful.Stateful] = None) → tensorflow.python.framework.ops.Tensor

Concatenate the encoded values of the encoders.

neuralmonkey.decoders.encoder_projection.empty_initial_state(train_mode: tensorflow.python.framework.ops.Tensor, rnn_size: int, encoders: List[neuralmonkey.model.stateful.Stateful] = None) → tensorflow.python.framework.ops.Tensor

Return an empty vector.

neuralmonkey.decoders.encoder_projection.linear_encoder_projection(dropout_keep_prob: float) → Callable[[tensorflow.python.framework.ops.Tensor, int, List[neuralmonkey.model.stateful.Stateful]], tensorflow.python.framework.ops.Tensor]

Return a linear encoder projection.

Return a projection function which applies dropout on concatenated encoder final states and returns a linear projection to a rnn_size-sized tensor.

Parameters:dropout_keep_prob – The dropout keep probability
neuralmonkey.decoders.encoder_projection.nematus_projection(dropout_keep_prob: float = 1.0) → Callable[[tensorflow.python.framework.ops.Tensor, int, List[neuralmonkey.model.stateful.Stateful]], tensorflow.python.framework.ops.Tensor]

Return encoder projection used in Nematus.

The initial state is a dense projection with tanh activation computed on the averaged states of the encoders. Dropout is applied to the means (before the projection).

Parameters:dropout_keep_prob – The dropout keep probability.