neuralmonkey.encoders.sentence_cnn_encoder module

Encoder for sentences withou explicit segmentation.

class neuralmonkey.encoders.sentence_cnn_encoder.SentenceCNNEncoder(name: str, input_sequence: neuralmonkey.model.sequence.Sequence, segment_size: int, highway_depth: int, rnn_size: int, filters: List[Tuple[int, int]], dropout_keep_prob: float = 1.0, use_noisy_activations: bool = False, reuse: neuralmonkey.model.model_part.ModelPart = None, save_checkpoint: str = None, load_checkpoint: str = None, initializers: List[Tuple[str, Callable]] = None) → None

Bases: neuralmonkey.model.model_part.ModelPart, neuralmonkey.model.stateful.TemporalStatefulWithOutput

Recurrent over Convolutional Encoder.

Encoder processing a sentence using a CNN then running a bidirectional RNN on the result.

Based on: Jason Lee, Kyunghyun Cho, Thomas Hofmann: Fully Character-Level Neural Machine Translation without Explicit Segmentation.

See https://arxiv.org/pdf/1610.03017.pdf

__init__(name: str, input_sequence: neuralmonkey.model.sequence.Sequence, segment_size: int, highway_depth: int, rnn_size: int, filters: List[Tuple[int, int]], dropout_keep_prob: float = 1.0, use_noisy_activations: bool = False, reuse: neuralmonkey.model.model_part.ModelPart = None, save_checkpoint: str = None, load_checkpoint: str = None, initializers: List[Tuple[str, Callable]] = None) → None

Create a new instance of the sentence encoder.

Parameters:
  • name – An unique identifier for this encoder
  • segment_size – The size of the segments over which we apply max-pooling.
  • highway_depth – Depth of the highway layer.
  • rnn_size – The size of the encoder’s hidden state. Note that the actual encoder output state size will be twice as long because it is the result of concatenation of forward and backward hidden states.
  • filters – Specification of CNN filters. It is a list of tuples specifying the filter size and number of channels.
Keyword Arguments:
 

dropout_keep_prob – The dropout keep probability (default 1.0)

bidirectional_rnn
cnn_encoded

1D convolution with max-pool that processing characters.

highway_layer

Highway net projection following the CNN.

output

Return the object output.

A 2D Tensor of shape (batch, state_size) which contains the resulting state of the object.

rnn_cells() → Tuple[tensorflow.python.ops.rnn_cell_impl.RNNCell, tensorflow.python.ops.rnn_cell_impl.RNNCell]

Return the graph template to for creating RNN memory cells.

temporal_mask

Return mask for the temporal_states.

A 2D Tensor of shape (batch, time) of type float32 which masks the temporal states so each sequence can have a different length. It should only contain ones or zeros.

temporal_states

Return object states in time.

A 3D Tensor of shape (batch, time, state_size) which contains the states of the object in time (e.g. hidden states of a recurrent encoder.