neuralmonkey.functions module¶
Collection of various functions and function wrappers.
-
neuralmonkey.functions.
inverse_sigmoid_decay
(param, rate, min_value: float = 0.0, max_value: float = 1.0, name: Union[str, NoneType] = None, dtype=tf.float32) → tensorflow.python.framework.ops.Tensor¶ Compute an inverse sigmoid decay: k/(k+exp(x/k)).
The result will be scaled to the range (min_value, max_value).
Parameters: - param – The parameter x from the formula.
- rate – Non-negative k from the formula.
-
neuralmonkey.functions.
noam_decay
(learning_rate: float, model_dimension: int, warmup_steps: int) → tensorflow.python.framework.ops.Tensor¶ Return decay function as defined in Vaswani et al., 2017, Equation 3.
https://arxiv.org/abs/1706.03762
lrate = (d_model)^-0.5 * min(step_num^-0.5, step_num * warmup_steps^-1.5)
Parameters: - model_dimension – Size of the hidden states of decoder and encoder
- warmup_steps – Number of warm-up steps
-
neuralmonkey.functions.
piecewise_function
(param, values, changepoints, name=None, dtype=tf.float32)¶ Compute a piecewise function.
Parameters: - param – The function parameter.
- values – List of function values (numbers or tensors).
- changepoints – Sorted list of points where the function changes from one value to the next. Must be one item shorter than values.