neuralmonkey.functions module

Collection of various functions and function wrappers.

neuralmonkey.functions.inverse_sigmoid_decay(param, rate, min_value: float = 0.0, max_value: float = 1.0, name: Union[str, NoneType] = None, dtype=tf.float32) → tensorflow.python.framework.ops.Tensor

Compute an inverse sigmoid decay: k/(k+exp(x/k)).

The result will be scaled to the range (min_value, max_value).

  • param – The parameter x from the formula.
  • rate – Non-negative k from the formula.
neuralmonkey.functions.noam_decay(learning_rate: float, model_dimension: int, warmup_steps: int) → tensorflow.python.framework.ops.Tensor

Return decay function as defined in Vaswani et al., 2017, Equation 3.

lrate = (d_model)^-0.5 * min(step_num^-0.5, step_num * warmup_steps^-1.5)

  • model_dimension – Size of the hidden states of decoder and encoder
  • warmup_steps – Number of warm-up steps
neuralmonkey.functions.piecewise_function(param, values, changepoints, name=None, dtype=tf.float32)

Compute a piecewise function.

  • param – The function parameter.
  • values – List of function values (numbers or tensors).
  • changepoints – Sorted list of points where the function changes from one value to the next. Must be one item shorter than values.