neuralmonkey.nn.projection module

Module which implements various types of projections.

neuralmonkey.nn.projection.glu(input_: tensorflow.python.framework.ops.Tensor, gating_fn: Callable[[tensorflow.python.framework.ops.Tensor], tensorflow.python.framework.ops.Tensor] = <function sigmoid>) → tensorflow.python.framework.ops.Tensor

Apply a Gated Linear Unit.

Gated Linear Unit - Dauphin et al. (2016).

http://arxiv.org/abs/1612.08083

neuralmonkey.nn.projection.maxout(inputs: tensorflow.python.framework.ops.Tensor, size: int, scope: str = 'MaxoutProjection') → tensorflow.python.framework.ops.Tensor

Apply a maxout operation.

Implementation of Maxout layer (Goodfellow et al., 2013).

http://arxiv.org/pdf/1302.4389.pdf

z = Wx + b y_i = max(z_{2i-1}, z_{2i})

Parameters:

• inputs – A tensor or list of tensors. It should be 2D tensors with equal length in the first dimension (batch size)
• size – The size of dimension 1 of the output tensor.
• scope – The name of the scope used for the variables

Returns:

A tensor of shape batch x size

neuralmonkey.nn.projection.multilayer_projection(input_: tensorflow.python.framework.ops.Tensor, layer_sizes: List[int], train_mode: tensorflow.python.framework.ops.Tensor, activation: Callable[[tensorflow.python.framework.ops.Tensor], tensorflow.python.framework.ops.Tensor] = <function relu>, dropout_keep_prob: float = 1.0, scope: str = 'mlp') → tensorflow.python.framework.ops.Tensor