neuralmonkey.model package

Submodules

neuralmonkey.model.model_part module

Basic functionality of all model parts.

class neuralmonkey.model.model_part.ModelPart(name: str, save_checkpoint: str = None, load_checkpoint: str = None) → None

Bases: object

Base class of all model parts.

feed_dict(dataset: neuralmonkey.dataset.Dataset, train: bool) → typing.Dict[tensorflow.python.framework.ops.Tensor, typing.Any]

Prepare feed dicts for part’s placeholders from a dataset.

get_dependencies() → typing.Set[typing.ModelPart]

Collect recusively all encoders and decoders.

load(session: tensorflow.python.client.session.Session) → None

Load model part from a checkpoint file.

name

Name of the model part and its variable scope.

save(session: tensorflow.python.client.session.Session) → None

Save model part to a checkpoint file.

use_scope()

Return a context manager.

Return a context manager that (re)opens the model part’s variable and name scope.

neuralmonkey.model.sequence module

Module which impements the sequence class and a few of its subclasses.

class neuralmonkey.model.sequence.EmbeddedFactorSequence(name: str, vocabularies: typing.List[neuralmonkey.vocabulary.Vocabulary], data_ids: typing.List[str], embedding_sizes: typing.List[int], max_length: int = None, add_start_symbol: bool = False, add_end_symbol: bool = False, save_checkpoint: str = None, load_checkpoint: str = None) → None

Bases: neuralmonkey.model.sequence.Sequence

A Sequence that stores one or more embedded inputs (factors).

data

Return the sequence data.

A 3D Tensor of shape (batch, time, dimension), where dimension is the sum of the embedding sizes supplied to the constructor.

dimension

Return the sequence dimension.

The sum of the embedding sizes supplied to the constructor.

embedding_matrices

Return a list of embedding matrices for each factor.

feed_dict(dataset: neuralmonkey.dataset.Dataset, train: bool = False) → typing.Dict[tensorflow.python.framework.ops.Tensor, typing.Any]

Feed the placholders with the data.

Parameters:

• dataset – The dataset.
• train – A flag whether the train mode is enabled.

Returns:

The constructed feed dictionary that contains the factor data and the mask.

input_factors

Return a list of 2D placeholders for each factor.

Each placeholder has shape (batch size, time).

mask

Return a 2D placeholder for the sequence mask.

This is shared across factors and must be the same for each of them.

tb_embedding_visualization(logdir: str, prj: <module 'tensorflow.contrib.tensorboard.plugins.projector' from '/home/docs/checkouts/readthedocs.org/user_builds/neural-monkey/envs/0.2.3/lib/python3.5/site-packages/tensorflow/contrib/tensorboard/plugins/projector/__init__.py'>)

Link embeddings with vocabulary wordlist.

Used for tensorboard visualization.

Parameters:

• logdir – directory where model is stored
• projector – TensorBoard projector for storing linking info.

class neuralmonkey.model.sequence.EmbeddedSequence(name: str, vocabulary: neuralmonkey.vocabulary.Vocabulary, data_id: str, embedding_size: int, max_length: int = None, add_start_symbol: bool = False, add_end_symbol: bool = False, save_checkpoint: str = None, load_checkpoint: str = None) → None

Bases: neuralmonkey.model.sequence.EmbeddedFactorSequence

A sequence of embedded inputs (for a single factor).

data_id

Return the input data series indentifier.

embedding_matrix

Return the embedding matrix for the sequence.

inputs

Return a 2D placeholder for the sequence inputs.

vocabulary

Return the input vocabulary.

class neuralmonkey.model.sequence.Sequence(name: str, max_length: int = None, save_checkpoint: str = None, load_checkpoint: str = None) → None

Bases: neuralmonkey.model.model_part.ModelPart

Base class for a data sequence.

This class represents a batch of sequences of Tensors of possibly different lengths.

data

Return the sequence data.

A Tensor representing the data in the sequence. The first and second dimension correspond to batch size and time respectively.

dimension

Return the sequence dimension.

The dimension of the sequence. For 3D sequences, this is the size of the last dimension of the data tensor.

lengths

Return the sequence lengths.

A 1D Tensor of type int32 that stores the lengths of the sequences in the batch.

mask

Return the sequence mask.

A 2D Tensor of type float32 and shape (batch size, time) that masks the sequences in the batch.

max_length

Return the maximum length of sequences in the data tensor.

neuralmonkey.model.stateful module

Module that provides classes that encapsulate model parts with states.

There are three classes: Stateful, TemporalStateful, and SpatialStateful.

Model parts that do not keep states in time but have a single tensor on the output should be instances of Stateful. Model parts that keep their hidden states in a time-oriented list (e.g. recurrent encoder) should be instances of TemporalStateful. Model parts that keep the states in a 2D matrix (e.g. image encoders) should be instances of SpatialStateful.

There are also classes that inherit from both stateful and temporal or spatial stateful (e.g. TemporalStatefulWithOutput) that can be used for model parts that satisfy more requirements (e.g. recurrent encoder).

class neuralmonkey.model.stateful.SpatialStateful

Bases: object

spatial_mask

Return mask for the spatial_states.

A 3D Tensor of shape (batch, width, height) of type float32 which masks the spatial states that they can be of different shapes. The mask should only contain ones or zeros.

spatial_states

Return object states in space.

A 4D Tensor of shape (batch, width, height, state_size) which contains the states of the object in space (e.g. final layer of a convolution network processing an image.

class neuralmonkey.model.stateful.SpatialStatefulWithOutput

Bases: neuralmonkey.model.stateful.Stateful, neuralmonkey.model.stateful.SpatialStateful

class neuralmonkey.model.stateful.Stateful

Bases: object

output

Return the object output.

A 2D Tensor of shape (batch, state_size) which contains the resulting state of the object.

class neuralmonkey.model.stateful.TemporalStateful

Bases: object

temporal_mask

Return mask for the temporal_states.

A 2D Tensor of shape (batch, time) of type float32 which masks the temporal states so each sequence can have a different length. It should only contain ones or zeros.

temporal_states

Return object states in time.

A 3D Tensor of shape (batch, time, state_size) which contains the states of the object in time (e.g. hidden states of a recurrent encoder.

class neuralmonkey.model.stateful.TemporalStatefulWithOutput

Bases: neuralmonkey.model.stateful.Stateful, neuralmonkey.model.stateful.TemporalStateful

Module contents