neuralmonkey.tf_utils module

A set of helper functions for TensorFlow.

neuralmonkey.tf_utils.append_tensor(tensor: tensorflow.python.framework.ops.Tensor, appendval: tensorflow.python.framework.ops.Tensor) → tensorflow.python.framework.ops.Tensor

Append an N-D Tensor to an (N+1)-D Tensor.

Parameters:

• tensor – The original Tensor
• appendval – The Tensor to add

Returns:

An (N+1)-D Tensor with appendval on the last position.

neuralmonkey.tf_utils.gather_flat(x: tensorflow.python.framework.ops.Tensor, indices: tensorflow.python.framework.ops.Tensor, batch_size: Union[int, tensorflow.python.framework.ops.Tensor] = 1, beam_size: Union[int, tensorflow.python.framework.ops.Tensor] = 1) → tensorflow.python.framework.ops.Tensor

Gather values from the flattened (shape=[batch * beam, …]) input.

This function expects a flattened tensor with first dimension of size batch x beam elements. Using the given batch and beam size, it reshapes the input tensor to a tensor of shape (batch, beam, ...) and gather the values from it using the index tensor.

Parameters:

• x – A flattened Tensor from which to gather values.
• indices – Index tensor.
• batch_size – The size of the batch.
• beam_size – The size of the beam.

Returns:

The Tensor of gathered values.

neuralmonkey.tf_utils.get_initializer(var_name: str, default: Callable = None) → Union[Callable, NoneType]

Return the initializer associated with the given variable name.

The name of the current variable scope is prepended to the variable name.

This should only be called during model building.

neuralmonkey.tf_utils.get_shape_list(x: tensorflow.python.framework.ops.Tensor) → List[Union[int, tensorflow.python.framework.ops.Tensor]]

Return list of dims, statically where possible.

Compute the static shape of a tensor. Where the dimension is not static (e.g. batch or time dimension), symbolic Tensor is returned.

Based on tensor2tensor.

Parameters:x – The Tensor to process. Returns:A list of integers and Tensors.

neuralmonkey.tf_utils.get_state_shape_invariants(state: tensorflow.python.framework.ops.Tensor) → tensorflow.python.framework.tensor_shape.TensorShape

Return the shape invariant of a tensor.

This function computes the loosened shape invariant of a state tensor. Only invariant dimension is the state size dimension, which is the last.

Based on tensor2tensor.

Parameters:state – The state tensor. Returns:A TensorShape object with all but the last dimensions set to None.

neuralmonkey.tf_utils.get_variable(name: str, shape: List[int] = None, dtype: tensorflow.python.framework.dtypes.DType = None, initializer: Callable = None, **kwargs) → tensorflow.python.ops.variables.Variable

Get an existing variable with these parameters or create a new one.

This is a wrapper around tf.get_variable. The initializer parameter is treated as a default which can be overriden by a call to update_initializers.

This should only be called during model building.

neuralmonkey.tf_utils.layer_norm(x: tensorflow.python.framework.ops.Tensor, epsilon: float = 1e-06) → tensorflow.python.framework.ops.Tensor

Layer normalize the tensor x, averaging over the last dimension.

Implementation based on tensor2tensor.

Parameters:

• x – The Tensor to normalize.
• epsilon – The smoothing parameter of the normalization.

Returns:

The normalized tensor.

neuralmonkey.tf_utils.partial_transpose(x: tensorflow.python.framework.ops.Tensor, indices: List[int]) → tensorflow.python.framework.ops.Tensor

Do a transpose on a subset of tensor dimensions.

Compute a permutation of first k dimensions of a tensor.

Parameters:

• x – The Tensor to transpose.
• indices – The permutation of the first k dimensions of x.

Returns:

The transposed tensor.

neuralmonkey.tf_utils.tf_print(tensor: tensorflow.python.framework.ops.Tensor, message: str = None, debug_label: str = None) → tensorflow.python.framework.ops.Tensor

Print the value of a tensor to the debug log.

Better than tf.Print, logs to console only when the “tensorval” debug subject is turned on.

Idea found at: https://stackoverflow.com/a/39649614

Parameters:tensor – The tensor whose value to print Returns:As tf.Print, this function returns a tensor identical to the input tensor, with the printing side-effect added.

neuralmonkey.tf_utils.update_initializers(initializers: Iterable[Tuple[str, Callable]]) → None