neuralmonkey.attention.feed_forward module¶
The feed-forward attention mechanism.
This is the attention mechanism used in Bahdanau et al. (2015)
See arxiv.org/abs/1409.0473
-
class
neuralmonkey.attention.feed_forward.
Attention
(name: str, encoder: Union[neuralmonkey.model.stateful.TemporalStateful, neuralmonkey.model.stateful.SpatialStateful], dropout_keep_prob: float = 1.0, state_size: int = None, reuse: neuralmonkey.model.model_part.ModelPart = None, save_checkpoint: str = None, load_checkpoint: str = None, initializers: List[Tuple[str, Callable]] = None) → None¶ Bases:
neuralmonkey.attention.base_attention.BaseAttention
-
__init__
(name: str, encoder: Union[neuralmonkey.model.stateful.TemporalStateful, neuralmonkey.model.stateful.SpatialStateful], dropout_keep_prob: float = 1.0, state_size: int = None, reuse: neuralmonkey.model.model_part.ModelPart = None, save_checkpoint: str = None, load_checkpoint: str = None, initializers: List[Tuple[str, Callable]] = None) → None¶ Create a new
BaseAttention
object.
-
attention
(query: tensorflow.python.framework.ops.Tensor, decoder_prev_state: tensorflow.python.framework.ops.Tensor, decoder_input: tensorflow.python.framework.ops.Tensor, loop_state: neuralmonkey.attention.namedtuples.AttentionLoopState) → Tuple[tensorflow.python.framework.ops.Tensor, neuralmonkey.attention.namedtuples.AttentionLoopState]¶ Get context vector for a given query.
-
attention_mask
¶
-
attention_states
¶
-
bias_term
¶
-
context_vector_size
¶ Return the static size of the context vector.
Returns: An integer specifying the context vector dimension.
-
finalize_loop
(key: str, last_loop_state: neuralmonkey.attention.namedtuples.AttentionLoopState) → None¶ Store the attention histories from loop state under a given key.
Parameters: - key – The key to the histories dictionary to store the data in.
- last_loop_state – The loop state object from the last state of the decoding loop.
-
get_energies
(y, _)¶
-
initial_loop_state
() → neuralmonkey.attention.namedtuples.AttentionLoopState¶ Get initial loop state for the attention object.
Returns: The newly created initial loop state object.
-
key_projection_matrix
¶
-
projection_bias_vector
¶
-
query_projection_matrix
¶
-
similarity_bias_vector
¶
-
state_size
¶
-