neuralmonkey.attention.feed_forward module

The feed-forward attention mechanism.

This is the attention mechanism used in Bahdanau et al. (2015)

See arxiv.org/abs/1409.0473

class neuralmonkey.attention.feed_forward.Attention(name: str, encoder: Union[neuralmonkey.model.stateful.TemporalStateful, neuralmonkey.model.stateful.SpatialStateful], dropout_keep_prob: float = 1.0, state_size: int = None, reuse: neuralmonkey.model.model_part.ModelPart = None, save_checkpoint: str = None, load_checkpoint: str = None, initializers: List[Tuple[str, Callable]] = None) → None

Bases: neuralmonkey.attention.base_attention.BaseAttention

__init__(name: str, encoder: Union[neuralmonkey.model.stateful.TemporalStateful, neuralmonkey.model.stateful.SpatialStateful], dropout_keep_prob: float = 1.0, state_size: int = None, reuse: neuralmonkey.model.model_part.ModelPart = None, save_checkpoint: str = None, load_checkpoint: str = None, initializers: List[Tuple[str, Callable]] = None) → None

Create a new BaseAttention object.

attention(query: tensorflow.python.framework.ops.Tensor, decoder_prev_state: tensorflow.python.framework.ops.Tensor, decoder_input: tensorflow.python.framework.ops.Tensor, loop_state: neuralmonkey.attention.namedtuples.AttentionLoopState) → Tuple[tensorflow.python.framework.ops.Tensor, neuralmonkey.attention.namedtuples.AttentionLoopState]

Get context vector for a given query.

attention_mask

attention_states

bias_term

context_vector_size

Return the static size of the context vector.

Returns:An integer specifying the context vector dimension.

finalize_loop(key: str, last_loop_state: neuralmonkey.attention.namedtuples.AttentionLoopState) → None

Store the attention histories from loop state under a given key.

Parameters:

• key – The key to the histories dictionary to store the data in.
• last_loop_state – The loop state object from the last state of the decoding loop.

get_energies(y, _)

hidden_features

initial_loop_state() → neuralmonkey.attention.namedtuples.AttentionLoopState

Get initial loop state for the attention object.

Returns:The newly created initial loop state object.

key_projection_matrix

projection_bias_vector

query_projection_matrix

similarity_bias_vector

state_size