trw.simple_layers

The purpose to this module is to provide a convenient way to create static neural network

Submodules

• trw.simple_layers.compiled_net
• trw.simple_layers.convs
• trw.simple_layers.denses
• trw.simple_layers.global_pooling
• trw.simple_layers.ordered_set
• trw.simple_layers.shift_scale
• trw.simple_layers.simple_layers
• trw.simple_layers.simple_layers_implementations
• trw.simple_layers.sub_tensor

Package Contents

Classes

OrderedSet	Ordered set
SimpleOutputBase	Base class to calculate an output
SimpleMergeBase	Base class for nodes with multiple inputs
SimpleLayerBase	Base layer for our simplified network specification
SimpleModule	Generic module
Input	Represent an input (i.e., a feature) to a network
OutputClassification	Output class for classification
Flatten	Generic module
Conv2d	Generic module
ReLU	Generic module
MaxPool2d	Generic module
Linear	Generic module
ConcatChannels	Implement a channel concatenation layer
OutputEmbedding	Create an embedding for display purposes
Conv3d	Generic module
MaxPool3d	Generic module
Reshape	Reshape a tensor to another shape
BatchNorm2d	Generic module
BatchNorm3d	Generic module
CompiledNet	Encapsulate a compiled network so that we can efficiently calculate the outputs
ShiftScale	Normalize a tensor with a mean and standard deviation
SubTensor	Select a region of a tensor (without copy), excluded the first component (N)

Functions

compile_nn(output_nodes: list, other_outputs_to_keep_alive=None, remove_checks=False)	Compile a network to calculate output_nodes
find_layer_type(nodes: list, layer_type)	Find all layer with a given type
nodes_mark_output_dependencies(output_nodes: list)	Marks nodes by output IDs
denses(parent, sizes, *args, **kwargs)
convs_3d(parent, channels, *args, **kwargs)
convs_2d(parent, channels, *args, **kwargs)
global_average_pooling_2d(parent)
global_average_pooling_3d(parent)
global_max_pooling_2d(parent)
global_max_pooling_3d(parent)

class trw.simple_layers.OrderedSet(iterable=None)

Bases: collections.MutableSet

Ordered set

Implementation based on a doubly linked link and an internal dictionary. This design gives OrderedSet the same big-Oh running times as regular sets including O(1) adds, removes, and lookups as well as O(n) iteration.

__len__(self)

__contains__(self, key)

add(self, key)

Add an element.

discard(self, key)

Remove an element. Do not raise an exception if absent.

__iter__(self)

__reversed__(self)

pop(self, last=True)

Return the popped value. Raise KeyError if empty.

__repr__(self)

Return repr(self).

__eq__(self, other)

Return self==value.

class trw.simple_layers.SimpleOutputBase(node, output_name, shape)

Bases: SimpleLayerBase

Base class to calculate an output

forward(self, inputs, batch)

Create a trw.train.Output from the inputs

Parameters

• inputs – a list of inputs of the output node

• batch – the batch of data fed to the network

Returns

a trw.train.Output object

class trw.simple_layers.SimpleMergeBase(parents, shape)

Bases: SimpleLayerBase

Base class for nodes with multiple inputs

class trw.simple_layers.SimpleLayerBase(parents, shape)

Base layer for our simplified network specification

Record the network node by node and keep track of the important information: parents, children, size.

Note

• nn.Module must be created during the initialization. This is to make sure we can easily share the

  network for different sub-models

get_module(self)

Return a nn.Module

class trw.simple_layers.SimpleModule(node, module, shape=None)

Bases: SimpleLayerBase

Generic module

Module must have a single input and all the module’s parameters should be on the same device.

static calculate_shape(shape, module, parents)

get_module(self)

Return a nn.Module

class trw.simple_layers.Input(shape: list, feature_name: str)

Bases: trw.simple_layers.simple_layers.SimpleLayerBase

Represent an input (i.e., a feature) to a network

get_module(self)

Return a nn.Module

class trw.simple_layers.OutputClassification(node, output_name, classes_name, **kwargs)

Bases: trw.simple_layers.simple_layers.SimpleOutputBase

Output class for classification

forward(self, inputs, batch)

Create a trw.train.Output from the inputs

Parameters

• inputs – a list of inputs of the output node

• batch – the batch of data fed to the network

Returns

a trw.train.Output object

get_module(self)

Return a nn.Module

class trw.simple_layers.Flatten(node)

Bases: trw.simple_layers.simple_layers.SimpleModule

Generic module

Module must have a single input and all the module’s parameters should be on the same device.

class trw.simple_layers.Conv2d(node, out_channels, kernel_size, stride=1, padding='same')

Bases: trw.simple_layers.simple_layers.SimpleModule

Generic module

Module must have a single input and all the module’s parameters should be on the same device.

class trw.simple_layers.ReLU(node)

Bases: trw.simple_layers.simple_layers.SimpleModule

Generic module

Module must have a single input and all the module’s parameters should be on the same device.

class trw.simple_layers.MaxPool2d(node, kernel_size, stride=None)

Bases: trw.simple_layers.simple_layers.SimpleModule

Generic module

Module must have a single input and all the module’s parameters should be on the same device.

class trw.simple_layers.Linear(node, out_features)

Bases: trw.simple_layers.simple_layers.SimpleModule

Generic module

Module must have a single input and all the module’s parameters should be on the same device.

class trw.simple_layers.ConcatChannels(nodes, flatten=False)

Bases: trw.simple_layers.simple_layers.SimpleMergeBase

Implement a channel concatenation layer

static calculate_shape(parents)

get_module(self)

Return a nn.Module

class trw.simple_layers.OutputEmbedding(node, output_name, functor=None)

Bases: trw.simple_layers.simple_layers.SimpleOutputBase

Create an embedding for display purposes

forward(self, inputs, batch)

Create a trw.train.Output from the inputs

Parameters

• inputs – a list of inputs of the output node

• batch – the batch of data fed to the network

Returns

a trw.train.Output object

get_module(self)

Return a nn.Module

class trw.simple_layers.Conv3d(node, out_channels, kernel_size, stride=1, padding='same')

Bases: trw.simple_layers.simple_layers.SimpleModule

Generic module

Module must have a single input and all the module’s parameters should be on the same device.

class trw.simple_layers.MaxPool3d(node, kernel_size, stride=None)

Bases: trw.simple_layers.simple_layers.SimpleModule

Generic module

Module must have a single input and all the module’s parameters should be on the same device.

class trw.simple_layers.Reshape(node, shape)

Bases: trw.simple_layers.simple_layers.SimpleModule

Reshape a tensor to another shape

class trw.simple_layers.BatchNorm2d(node, eps=1e-05, momentum=0.1, affine=True)

Bases: trw.simple_layers.simple_layers.SimpleModule

Generic module

Module must have a single input and all the module’s parameters should be on the same device.

class trw.simple_layers.BatchNorm3d(node, eps=1e-05, momentum=0.1, affine=True)

Bases: trw.simple_layers.simple_layers.SimpleModule

Generic module

Module must have a single input and all the module’s parameters should be on the same device.

trw.simple_layers.compile_nn(output_nodes: list, other_outputs_to_keep_alive=None, remove_checks=False)

Compile a network to calculate output_nodes

Parameters

• output_nodes – the output nodes to calculate. The order of the nodes indicates the order of the calculation and impacts the book-keeping of the shared calculation in multiple output networks

• other_outputs_to_keep_alive – keeps alive unused output nodes

• remove_checks – if True, some runtime checks will be disabled. This can be useful for example FCNN where the output shape will depend on the input shape

Returns

a CompiledNet

trw.simple_layers.find_layer_type(nodes: list, layer_type)

Find all layer with a given type

Parameters

• nodes – the starting nodes [list]

• layer_type – the type of the nodes to collect

Returns

a list of nodes of the corresponding type

trw.simple_layers.nodes_mark_output_dependencies(output_nodes: list)

Marks nodes by output IDs

Parameters

output_nodes – a list of output nodes to be marked

Returns

nodes with a set of output IDs

class trw.simple_layers.CompiledNet(remove_checks=False)

Bases: torch.nn.Module

Encapsulate a compiled network so that we can efficiently calculate the outputs

of the network.

forward(self, batch)

Calculate the outputs of a network

Parameters

batch – (dict) a dictionary like of features

Returns

a dictionary of outputs

__getstate__(self)

__setstate__(self, state)

trw.simple_layers.denses(parent, sizes, *args, **kwargs)

trw.simple_layers.convs_3d(parent, channels, *args, **kwargs)

trw.simple_layers.convs_2d(parent, channels, *args, **kwargs)

trw.simple_layers.global_average_pooling_2d(parent)

trw.simple_layers.global_average_pooling_3d(parent)

trw.simple_layers.global_max_pooling_2d(parent)

trw.simple_layers.global_max_pooling_3d(parent)

class trw.simple_layers.ShiftScale(node, mean, standard_deviation)

Bases: trw.simple_layers.simple_layers.SimpleModule

Normalize a tensor with a mean and standard deviation

The output tensor will be (x - mean) / standard_deviation

This layer simplify the preprocessing for the trw.simple_layers package

class trw.simple_layers.SubTensor(node, min_indices, max_indices_exclusive)

Bases: trw.simple_layers.simple_layers.SimpleModule

Select a region of a tensor (without copy), excluded the first component (N)