neuralmonkey.encoders.sentence_cnn_encoder module¶
Encoder for sentences withou explicit segmentation.
-
class
neuralmonkey.encoders.sentence_cnn_encoder.
SentenceCNNEncoder
(name: str, input_sequence: neuralmonkey.model.sequence.Sequence, segment_size: int, highway_depth: int, rnn_size: int, filters: List[Tuple[int, int]], dropout_keep_prob: float = 1.0, use_noisy_activations: bool = False, save_checkpoint: Union[str, NoneType] = None, load_checkpoint: Union[str, NoneType] = None, initializers: List[Tuple[str, Callable]] = None) → None¶ Bases:
neuralmonkey.model.model_part.ModelPart
,neuralmonkey.model.stateful.TemporalStatefulWithOutput
Recurrent over Convolutional Encoder.
Encoder processing a sentence using a CNN then running a bidirectional RNN on the result.
Based on: Jason Lee, Kyunghyun Cho, Thomas Hofmann: Fully Character-Level Neural Machine Translation without Explicit Segmentation.
See https://arxiv.org/pdf/1610.03017.pdf
-
__init__
(name: str, input_sequence: neuralmonkey.model.sequence.Sequence, segment_size: int, highway_depth: int, rnn_size: int, filters: List[Tuple[int, int]], dropout_keep_prob: float = 1.0, use_noisy_activations: bool = False, save_checkpoint: Union[str, NoneType] = None, load_checkpoint: Union[str, NoneType] = None, initializers: List[Tuple[str, Callable]] = None) → None¶ Create a new instance of the sentence encoder.
Parameters: - name – An unique identifier for this encoder
- segment_size – The size of the segments over which we apply max-pooling.
- highway_depth – Depth of the highway layer.
- rnn_size – The size of the encoder’s hidden state. Note that the actual encoder output state size will be twice as long because it is the result of concatenation of forward and backward hidden states.
- filters – Specification of CNN filters. It is a list of tuples specifying the filter size and number of channels.
Keyword Arguments: dropout_keep_prob – The dropout keep probability (default 1.0)
-
bidirectional_rnn
¶
-
cnn_encoded
¶ 1D convolution with max-pool that processing characters.
-
highway_layer
¶ Highway net projection following the CNN.
-
output
¶ Return the object output.
A 2D Tensor of shape (batch, state_size) which contains the resulting state of the object.
-
rnn_cells
() → Tuple[tensorflow.python.ops.rnn_cell_impl.RNNCell, tensorflow.python.ops.rnn_cell_impl.RNNCell]¶ Return the graph template to for creating RNN memory cells.
-
temporal_mask
¶ Return mask for the temporal_states.
A 2D Tensor of shape (batch, time) of type float32 which masks the temporal states so each sequence can have a different length. It should only contain ones or zeros.
-
temporal_states
¶ Return object states in time.
A 3D Tensor of shape (batch, time, state_size) which contains the states of the object in time (e.g. hidden states of a recurrent encoder.
-