dlsia.core.networks package

Submodules

dlsia.core.networks.aggnet module

dlsia.core.networks.aggnet.AggNet_from_file(filename)

Read network parameter file from disc and reinstantiate it.

Parameters:: filename (str) – The filename
Returns:: The Aggnet of interest
Return type:: AggregateNet

class dlsia.core.networks.aggnet.AggregateNet(in_channels, out_channels, layers=1, relu=True, bias=True, final_activation=None)

Bases: Module

Final 1x1 convolutional layer that can be used to combine the results of multiple individual models

Parameters:

in_channels (int) – number of channels in the input
out_channels (int) – number of channels in the output
relu (bool) – perform ReLU after the layer
bias (bool) – include a bias parameter in the convolution layer

forward(x)

Define the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

save_network_parameters(name)

Save an ordered dictionary of the state and topology parameters of the network. Can be used to save / reinstantiate the network.

Parameters:: name (str) – filename
Returns:: None
Return type:: None

training: bool

dlsia.core.networks.baggins module

dlsia.core.networks.baggins.arithmetic_mean_std(results)

Computes Arithmatic mean and standard deviation

Parameters:: results (an tensor with results. we average over channel 1) –
Return type:: mean and standard deviation (arithmatic)

class dlsia.core.networks.baggins.autoencoder_labeling_model_baggin(models, returns_normalized=False)

Bases: Module

Bagg a number of models

forward(x, device='cpu', return_std=False)

Standard forward model

Parameters:

x (input tensor) –
device (where will we do the calculations?) –
return_std (If True standard deviations will be returned) –

Return type:

mean and standard deviation

training: bool

dlsia.core.networks.baggins.geometric_mean_std(results, eps=1e-12)

Computes geometric mean and standard deviation

Parameters:: results (an tensor with results. we average over channel 1) –
Return type:: mean and standard deviation (geometric)

class dlsia.core.networks.baggins.model_baggin(models, model_type='classification', returns_normalized=False, average_type='arithmetic')

Bases: Module

Bagg a number of models

forward(x, device='cpu', return_std=False)

Standard forward model

Parameters:

x (input tensor) –
device (where will we do the calculations?) –
return_std (If true the standard deviation will be returned) –

Return type:

mean and standard deviation

training: bool

dlsia.core.networks.msdnet module

dlsia.core.networks.msdnet.MSDNetwork_from_file(filename)

Construct an MSDNet from a file with network parameters

Parameters:: filename (str) – the filename
Returns:: An SMSNet
Return type:: SMSNet

class dlsia.core.networks.msdnet.MixedScaleDenseLayer(convolution, in_channels, dilations, padding_mode, conv_kernel_size=3)

Bases: Module

Object which builds a single ‘layer’ in our MSDNetwork

forward(x)

Standard forward operator

Parameters:: x (input tensor) –
Return type:: output tensor

training: bool

class dlsia.core.networks.msdnet.MixedScaleDenseNetwork(in_channels, out_channels, num_layers=10, layer_width=1, max_dilation=10, custom_msdnet=None, activation=ReLU(), normalization=<class 'torch.nn.modules.batchnorm.BatchNorm2d'>, final_layer=None, conv_kernel_size=3, dropout=None, convolution=<class 'torch.nn.modules.conv.Conv2d'>, padding_mode='zeros')

Bases: Sequential

Defines Mixed Scale Dense Network based on topology (number of layers and channels per layer) and morphology (dilation sizes for each feature map). Input and output layer sizes are passed as input, including user-defined activation dropout rates and activation layers.

Parameters:

in_channels (int) – number of channels in input data
out_channels (int) – number of channels in output data
num_layers (int or None) – depth of network
layer_width (int or None) – width of each layer
max_dilation (int or None) – maximum dilation size the network will cycle through
custom_msdnet (List[int]) – n by m numnpy array whose dimensions define the network topology (m number of layers and n channels per layer) and entries define the morphology (size of dilated kernel at each channel)
activation (torch.nn class instance or list of torch.nn class instances) – instance of PyTorch activation class applied to each layer. If passing a list of multiple activation class instances, each will be applied in the order given. ex) activation=nn.ReLU() ex) activation=nn.RReLU(lower=.1, upper=.9) ex) activation=[nn.ReLU(), nn.Sigmoid()]
normalization (torch.nn class) – PyTorch normalization class applied to each layer. Passed as class without parentheses since we need a different instance per layer ex) normalization=nn.BatchNorm2d
final_layer –
instance of PyTorch activation class applied after final layer. If passing a list of multiple activation class instances, each will be applied in the order given.

ex) normalization=nn.Sigmoid() ex) normalization=nn.Softmax(dim=1)
conv_kernel_size (int) – the n by n size of filter mask applied to all but final convolution (3 or 5 chosen almost universally for CNNs)
dropout (List[Union[int,float]]) – 1 by 3 numpy.ndarray defining drop out rate for [initial, hidden, final] layers. If none, dropout is not used
convolution (torch.nn class instance) – instance of PyTorch convolution class. Accepted are nn.Conv1d, nn.Conv2d, and nn.Conv3d.
padding_mode (str) – padding mode to be used in convolution to fill boundary space. Accepted input are ‘zeros’, ‘reflect’, ‘replicate’ or ‘circular’

Instructions: User has two options for building network topology and morphology:

to generate a network based on number of layers, channels, and the number of dilations to cycle through, pass integers in for num_layers, layer_width, and max_dilation and leave custom_msdnet = None

ex) num_layers=5, layer_width=2, max_dilation=5

yields the network [1, 3, 5, 2, 4, 1
2, 4, 1, 3, 5, 2]

to create a custom network with user-specified dilation sizes, network depth, and layer width, pass in:

num_layers = layer_width
= max_dilation = None

numpy array populated with desired integer-valued dilations whose columns correspond to network layers and number of rows corresponds to number of channels per layer

Referenece paper: A mixed-scale dense convolutional neural network for: image analysis

Published: PNAS, Jan. 2018 Link: http://www.pnas.org/content/early/2017/12/21/1715832114

Note: Bias=True is Conv2d and Conv3d default

save_network_parameters(name=None): Save the network parameters :param name: The filename :type name: str :return: None :rtype: None

topology_dict()

Get all parameters needed to build this network

Returns:: An orderdict with all parameters needed
Return type:: OrderedDict

dlsia.core.networks.msdnet.tst(show_network=True)

Defines and test several Mixed Scale Dense Networks consisting of 2D convolutions, provides a printout of the network, and checks to make sure tensors pass through the network

Parameters:: show_network – if True, print out each network

dlsia.core.networks.msd_graph_tools module

class dlsia.core.networks.msd_graph_tools.RandomMultiSourceMultiSinkGraph(in_channels, out_channels, layers, dilations, LL_alpha=0.25, LL_gamma=1.5, LL_min_degree=1, LL_max_degree=None, IL_target=0.15, LO_target=0.15, IO_target=False)

Bases: object

Builds a random graph with specified sparsity parameters. Can specify the source and sink parameters, and control the distribution of length of skip connections between hidden layers.

build_matrix(return_numpy=True): Build full random matrix given specifications :param return_numpy: return a numpy array :type return_numpy: bool :return: the graph

draw_dilations(): Draws dilations :return: Return a random dilation vector

dlsia.core.networks.msd_graph_tools.draw_random_dag(n, p, tol=0.01, max_count=100)

dlsia.core.networks.msd_graph_tools.random_DAG(n, p)

dlsia.core.networks.msd_graph_tools.random_DAG_scale_free(n, p)

dlsia.core.networks.msd_graph_tools.sparsity(g)

dlsia.core.networks.msd_graph_tools.tst()

dlsia.core.networks.smsnet module

dlsia.core.networks.smsnet.Conv2DReLU(in_channels, out_channels, conv_kernel_size, dilation_size, stride, output_padding, padding_mode='reflect')

A simple Conv2D + ReLU operator

Parameters:

in_channels – in channels
out_channels – out channels
conv_kernel_size – kernel size
dilation_size – dilation size
stride – Stride (for down sampling)
output_padding – padding needed for transposed convolution.
padding_mode – padding mode

Returns:

a sequential module

dlsia.core.networks.smsnet.Identity(in_channels, out_channels)

Identity operator.

Parameters:

in_channels – not used
out_channels – not used

Returns:

the identity operator

dlsia.core.networks.smsnet.Linear(in_channels, out_channels, stride, output_padding, bias=True)

A linear transformation

Parameters:

in_channels – in channels
out_channels – out channels
stride – stride
output_padding – output padding for transpose convolution
bias – refine a bias term?

Returns:

an nn.sequential module

dlsia.core.networks.smsnet.LinearReLU(in_channels, out_channels, stride, output_padding=None)

A linear transformation with subsequent ReLU

Parameters:

in_channels – in channels
out_channels – out channels
stride – the stride
output_padding – output padding needed to get image at right size. ignore is None.

Returns:

a sequential module

dlsia.core.networks.smsnet.LinearSigmoid(in_channels, out_channels, stride, output_padding, bias=True)

A linear transformation

Parameters:

output_padding – output padding needed for transpose convolutions
bias – Refine a bias?
stride – The stride
in_channels – in channels
out_channels – out channels

Returns:

a sequential module

class dlsia.core.networks.smsnet.SMSNet(in_channels, out_channels, in_shape, out_shape, scaling_table, network_graph, channel_count, convolution_kernel_size=3, first_action=<function Identity>, hidden_action=<function Conv2DReLU>, last_action=<function Linear>)

Bases: Module

A Sparse Mixed Scale Network

aggregate(this_node, data)

Aggregate the data in place

Parameters:

this_node –
data –

build_network(): Put the network and its operations together

forward(x)

The forward method

Parameters:: x – The data
Returns:: the resulting tensor after it has been passed through the network

reset()

save_network_parameters(name=None): Save the network parameters :param name: The filename :type name: str :return: None :rtype: None

topology_dict()

Get all parameters needed to build this network

Returns:: An orderdict with all parameters needed
Return type:: OrderedDict

training: bool

dlsia.core.networks.smsnet.SMSNetwork_from_file(filename)

Construct an SMSNet from a file with network parameters

Parameters:: filename (str) – the filename
Returns:: An SMSNet
Return type:: SMSNet

class dlsia.core.networks.smsnet.SMS_reservoir(kitchen_sink_obj, in_channels, out_channels, last_action=<function Linear>)

Bases: Module

An SMS Network but on the last layer is trained.

forward(x)

Standard forward method.

Parameters:: x – input tensor
Returns:: network output

training: bool

dlsia.core.networks.smsnet.buildSMSdilations(G, dilation_choices, mode='Node', p_dilation_choice=None, p_sink_choice=None)

Assign dilations to the graph

Parameters:

G – a digraph
dilation_choices – dilation choices
mode – Either “Node”, “NodeCyclic”, “Edges”
p_dilation_choice – probabilities at which dilations are chosen
p_sink_choice – probabilities at which dilations are chosen to the sink

Returns:

A digraph with listed dilations as edge properties

dlsia.core.networks.smsnet.buildSMSnodechannels(G, in_channels, in_channel_choices=None, hidden_out_channel_choices=None, p_hidden_out_channel_choices=None, p_in_channel_choices=None)

This function builds a distribution of input and output channels, in a random fashion given a SMS graph.

Parameters:

G – input graph
in_channels – the number of input channels
in_channel_choices – governs number of channels from input to first node
hidden_out_channel_choices – governs number of output channels in hidden layers
p_hidden_out_channel_choices – a distribution for the hidden channel choices
p_in_channel_choices – a distribution for the in channel choices

Returns:

An graph with the desired network parameters along the edges

dlsia.core.networks.smsnet.edges_source_sink_list(G)

Simple utility function to get a list of source, sink and edges

Parameters:: G – input graph
Returns:: edge_list, source_nodes, between_nodes, sink_nodes

dlsia.core.networks.smsnet.initialize_weights(m)

dlsia.core.networks.smsnet.network_stats(net)

dlsia.core.networks.smsnet.random_SMS_network(in_channels, out_channels, layers, dilation_choices, in_shape=(64, 64), out_shape=(64, 64), hidden_out_channels=None, layer_probabilities=None, sizing_settings=None, dilation_mode='Edges', network_type='Regression', network_mode='Full')

Build a random sparse mixed scale network

Parameters:

in_channels – input channels
out_channels – output channels
in_shape – input shape - if no scaling /up down, this can be anything
out_shape – output shape - if no scaling /up down, this can be anything
layers – The umber of hidden layers between input and output node
dilation_choices – An array of dilation values to choose from
hidden_out_channels – An array of output channel choices
layer_probabilities – A dictionary that describes how the graph is constructed. LL_alpha: 0 uniform skips ; the higher this gets, the less likely it is a skip connection is ‘lengthy’ LL_gamma: Degree probability P(degree) propto degree^-LL_gamma LL_max_degree: limits the maximal degree per node IL: probability of a connection between input and hidden layer LO: probability of a connection between hidden and output layer IO: bool, sets connections between input and output layer
sizing_settings – Governs sizing up and down nodes. Leave alone.
dilation_mode – If “Edges” dilations will be assigned at random. “Nodes” all edges to the same node have equal dilation “NodeCyclic” cyle through the list of dilations in a cyclic fashion, per node
network_type – If “Regression” or “Classification” the final action is a linear operator. “Regression_Sigmoid” does softmax as the final action
network_mode – “Full” train all parameters “Reservoir” only train the final (linear) layer.

Returns:

A network with above settings.

dlsia.core.networks.smsnet.sort_and_rename(G)

Topologoically sort and rename a graph

Parameters:: G – The input network
Returns:: The sorted network

dlsia.core.networks.sparsenet module

Tools for sparse mixed scale networks where size changes of tensors are needed

class dlsia.core.networks.sparsenet.SparseAEC(in_shape=(28, 28), latent_shape=(7, 7), in_channels=1, out_classes=10, depth=20, dilations=None, hidden_channels=10, out_channels=1, alpha_range=(0.75, 1.0), gamma_range=(0.0, 0.5), max_degree=5, min_degree=1, pIL=1.0, pLO=1.0, IO=True, stride_base=2, dropout_rate=0.05, encoder=None, decoder=None)

Bases: Module

A randomized Sparse Mixed Scale Autoencoder, with a classifier attached to the compressed image latent space

forward(x)

Standard forward operator

Parameters:: x (input tensor) –
Return type:: Class probabilities (non-softmax)

latent(x)

reinitialize_fc()

save_network_parameters(name=None)

to(device)

A to function

Parameters:: device (target device) –
Return type:: object on device

topology_dict()

training: bool

dlsia.core.networks.sparsenet.SparseAEC_from_file(filename)

Construct an SparseAEC from a file with network parameters

Parameters:: filename (str) – the filename
Returns:: An SparseAEC
Return type:: SparseAEC

class dlsia.core.networks.sparsenet.SparseAutoEncoder(in_shape=(28, 28), latent_shape=(7, 7), latent_sequence=[32, 2], dropout_p=0.05, in_channels=1, depth=20, dilations=None, hidden_channels=10, out_channels=1, alpha_range=(0.75, 1.0), gamma_range=(0.0, 0.5), max_degree=5, min_degree=1, pIL=1.0, pLO=1.0, IO=True, stride_base=2, final_transform=None, encoder=None, decoder=None)

Bases: Module

An Autoencoder based on randomized Sparse Mixed Scale Networks

forward(x)

A standard forward operator

Parameters:: x (input tensor) –

latent_vector(x)

Get the latent space vector

Parameters:: x (input tensor) –
Return type:: output latent space vector

save_network_parameters(name=None)

to(device)

A To function :param device: :type device: target device

Return type:: object on device

topology_dict()

training: bool

dlsia.core.networks.sparsenet.SparseAutoEncoder_from_file(filename)

Construct an SparseAEC from a file with network parameters

Parameters:: filename (str) – the filename
Returns:: An SparseAEC
Return type:: SparseAEC

class dlsia.core.networks.sparsenet.SparseLabeler(in_shape=(28, 28), latent_shape=(7, 7), in_channels=1, out_classes=10, depth=20, dilations=None, hidden_channels=10, out_channels=1, alpha_range=(0.75, 1.0), gamma_range=(0.0, 0.5), max_degree=5, min_degree=1, pIL=0.25, pLO=0.25, IO=True, stride_base=2, dropout_rate=0.05, encoder=None)

Bases: Module

This object takes a (C Y X) tensor, compresses it to a vector of length class_size

forward(x)

Standard forward operator

Parameters:: x (input tensor) –
Return type:: Class probabilities (non-softmax)

save_network_parameters(name=None)

to(device)

A to function.

Parameters:: device (target device) –
Return type:: object on device

topology_dict()

training: bool

dlsia.core.networks.sparsenet.SparseLabeler_from_file(filename)

Construct an SMSNet from a file with network parameters

Parameters:: filename (str) – the filename
Returns:: An SMSNet
Return type:: smsnet

dlsia.core.networks.sparsenet.fc_weights_init(m)

dlsia.core.networks.tunet module

class dlsia.core.networks.tunet.TUNet(image_shape, in_channels, out_channels, depth, base_channels, growth_rate=2, hidden_rate=1, conv_kernel=<class 'torch.nn.modules.conv.Conv2d'>, kernel_down=<class 'torch.nn.modules.pooling.MaxPool2d'>, kernel_up=<class 'torch.nn.modules.conv.ConvTranspose2d'>, normalization=<class 'torch.nn.modules.batchnorm.BatchNorm2d'>, activation=ReLU(), final_activation=None, conv_kernel_size=3, maxpool_kernel_size=2, dilation=1)

Bases: Module

This function creates a U-Net model commonly used for image semantic segmentation. The model takes in an input image and outputs a segmented image, with the number of output classes dictated by the out_channels parameter.

In this dlsia implementation, a number of architecture-governing hyperparameters may be tuned by the user, including the network depth, convolutional channel growth rate both within & between layers, and the normalization & activation operations following each convolution.

Parameters:

image_shape – image shape we use
in_channels – input channels
out_channels – output channels
depth – the total depth
base_channels – the first operator take in_channels->base_channels.
growth_rate – The growth rate of number of channels per depth layer
hidden_rate – How many ‘inbetween’ channels do we want? This is relative to the feature channels at a given depth
conv_kernel – The convolution kernel we want to us. Conv2D or Conv3D
kernel_down – How do we steps down? MaxPool2D or MaxPool3D
kernel_up – How do we step up? nn.ConvTranspose2d or nn.ConvTranspose3d
normalization – A normalization action
activation – Activation function
conv_kernel_size – The size of the convolutional kernel we use
maxpool_kernel_size – The size of the max pooling kernel we use to step down
stride – The stride we want to use.
dilation – The dilation we want to use.

build_output_layer(in_channels, in_between_channels1, in_between_channels2, final_channels)

For final output layer, builds a sequence of convolutions with activations functions and normalization layers

Parameters:

final_channels (int) – The output channels
in_channels – input channels
in_between_channels1 – the in between channels after first convolution
in_between_channels2 – the in between channels after second convolution

“param final_channels: number of channels the network outputs :return:

build_unet_layer(in_channels, in_between_channels, out_channels)

Build a sequence of convolutions with activations functions and normalization layers

Parameters:

in_channels – input channels
in_between_channels – the in between channels
out_channels – the output channels

Returns:

forward(x)

Default forward operator.

Parameters:: x – input tensor.
Returns:: output of neural network

save_network_parameters(name=None): Save the network parameters :param name: The filename :type name: str :return: None :rtype: None

topology_dict()

Get all parameters needed to build this network

Returns:: An orderdict with all parameters needed
Return type:: OrderedDict

training: bool

dlsia.core.networks.tunet.TUNetwork_from_file(filename)

Construct an MSDNet from a file with network parameters

Parameters:: filename (str) – the filename
Returns:: An SMSNet
Return type:: SMSNet

dlsia.core.networks.tunet.build_down_operator(chart, from_depth, to_depth, maxpool_kernel, key='Pool_Settings')

Build a down sampling operator

Parameters:

chart – Array of sizing charts (one for each dimension)
from_depth – we start at this depth
to_depth – and go here
maxpool_kernel – the max pooling kernel we want to use (MaxPool2D or MaxPool3D)
key – a key we can use - default is fine

Returns:

An operator with given specs

dlsia.core.networks.tunet.build_up_operator(chart, from_depth, to_depth, in_channels, out_channels, conv_kernel, key='convT_settings')

Build an up sampling operator

Parameters:

chart – An array of sizing charts (one for each dimension)
from_depth – The sizing is done at this depth
to_depth – and goes to this depth
in_channels – number of input channels
out_channels – number of output channels
conv_kernel – the convolutional kernel we want to use
key – a key we can use - default is fine

Returns:

returns an operator

dlsia.core.networks.tunet.max_pool_size_result(Nin, kernel, stride, dilation=1, padding=0)

Determine the spatial dimension size after a max pooling operation

Parameters:

Nin – dimension of 1d array
kernel – kernel size
stride – stride; might need to match kernel size
dilation – dilation factor
padding – padding parameter

Returns:

the resulting array length

dlsia.core.networks.tunet.tst()

dlsia.core.networks.tunet.unet_sizing_chart(N, depth, stride, maxpool_kernel_size, up_down_padding=0, dilation=1)

Build a set of dictionaries that are useful to make sure that we can map arrays back to the right sizes for each downsampling and upsampling operation.

Parameters:

N – dimension of array
depth – the total depth of the unet
stride – the stride - we fix this for a single UNet
maxpool_kernel_size – the max pooling kernel size
up_down_padding – max pooling and convT padding, Default is 0
dilation – the dilation factor. default is 1

Returns:

a dictionary with information

The data associated with key “Sizes” provides images size per depth The data associated with key “Pool Setting” provides info needed to construct a MaxPool operator The data associated with key “convT Setting” provides info need to construct transposed convolutions such that the image of a the right size is constructed.

dlsia.core.networks.tunet3plus module

class dlsia.core.networks.tunet3plus.TUNet3Plus(image_shape, in_channels, out_channels, depth, base_channels, growth_rate=2, hidden_rate=1, carryover_channels=0, conv_kernel=<class 'torch.nn.modules.conv.Conv2d'>, kernel_down=<class 'torch.nn.modules.pooling.MaxPool2d'>, kernel_up=<class 'torch.nn.modules.upsampling.Upsample'>, normalization=<class 'torch.nn.modules.batchnorm.BatchNorm2d'>, activation=ReLU(), conv_kernel_size=3, maxpool_kernel_size=2, stride=2, dilation=1)

Bases: Module

This function creates a U-Net3+ model, a new variant of the classic U-Net mode commonly used for image semantic segmentation. While the model takes in an input image and outputs a class-segmented image, this new 3+ variant supplants the classic U-net layer-to-layer skip connections with dense skip connections aggregating features from all network layers, thus encouraging more multi-scale learning of features

In this dlsia implementation, a number of architecture-governing hyperparameters may be tuned by the user, including the network depth, the number of convolutional channels carried over in each individual skip connection, convolutional channel growth rate both within & between layers, and the normalization & activation operations following each convolution.

Parameters:

image_shape – Image shape we use
in_channels – Number of input channels
out_channels – Number of output channels
depth – The total depth, or the number of layers
base_channels – Number of image channels in the first operator; i.e. in_channels->base_channels.
growth_rate – The growth rate of number of channels per depth layer
hidden_rate – How many ‘in between’ channels do we want? This is relative to the feature channels at a given depth
carryover_channels – Number of channels carried over in each skip connection. Each decoder layer will have (depth * carryover_channels) number of channels. If None, defaults to base_channels.
conv_kernel – The convolution kernel we want to us. Conv2D or Conv3D
kernel_down – How do we steps down? MaxPool2D or MaxPool3D
kernel_up – How do we step up? nn.ConvTranspose2d or nn.ConvTranspose3d
normalization – The normalization operation applied after each convolution
activation – The activation function applied in tandem with the normalization operator
conv_kernel_size – The size of the convolutional kernel we use
maxpool_kernel_size – The size of the max pooling kernel we use to step down
stride – The stride we want to use.
dilation – The dilation we want to use.

build_output_layer(in_channels, in_between_channels1, in_between_channels2, final_channels)

For final output layer, builds a sequence of convolutions with activations functions and normalization layers

Parameters:

final_channels (int) – The output channels
in_channels – input channels
in_between_channels1 – the in between channels after first convolution
in_between_channels2 – the in between channels after second convolution

“param final_channels: number of channels the network outputs :return:

build_skip_connections(chart, from_depth, to_depth, squeeze_channels)

Single skip connections are built here. Each skip connection consists of either: 1) concatenation if incoming layer matches terminal layer, 2) downsampling operation (MaxPool default) if incoming layer is higher than terminal layer, and 3) upsampling operation (ConvTranspose default) if incoming layer is lower thatn terminal layer. Subsequently, a convolution is applied to squeeze the feature map channels, followed by the chosen normalization and activation operations (BatchNorm and ReLU by default). All operations applied are defined in TUNet __init__, including: self.conv_kernel, self.normalization, self.activation, self.kernel_down, and self.kernel_down.

Parameters:

chart – the skip connection sizing chart; contains all information for correctly upsampling and downsampling.
from_depth – The incoming layer size.
to_depth – The terminal layer size.
squeeze_channels – The number of outgoing channels for each convolution application

build_unet_layer(in_channels, in_between_channels, out_channels)

Build a sequence of convolutions with activations functions and normalization layers

Parameters:

in_channels – input channels
in_between_channels – the in between channels
out_channels – the output channels

Returns:

forward(x)

Default forward operator.

Parameters:: x – input tensor.
Returns:: output of neural network

save_network_parameters(name=None): Save the network parameters :param name: The filename :type name: str :return: None :rtype: None

topology_dict()

Get all parameters needed to build this network

Returns:: An orderdict with all parameters needed
Return type:: OrderedDict

training: bool

dlsia.core.networks.tunet3plus.TUNetwork3Plus_from_file(filename)

Construct an MSDNet from a file with network parameters

Parameters:: filename (str) – the filename
Returns:: An SMSNet
Return type:: SMSNet

dlsia.core.networks.tunet3plus.build_down_operator(chart, from_depth, to_depth, maxpool_kernel, key='Pool_Settings')

Build a down sampling operator

Parameters:

chart – Array of sizing charts (one for each dimension)
from_depth – we start at this depth
to_depth – and go here
maxpool_kernel – the max pooling kernel we want to use (MaxPool2D or MaxPool3D)
key – a key we can use - default is fine

Returns:

An operator with given specs

dlsia.core.networks.tunet3plus.build_up_operator(chart, from_depth, to_depth, in_channels, out_channels, conv_kernel, key='convT_settings')

Build an up sampling operator

Parameters:

chart – An array of sizing charts (one for each dimension)
from_depth – The sizing is done at this depth
to_depth – and goes to this depth
in_channels – number of input channels
out_channels – number of output channels
conv_kernel – the convolutional kernel we want to use
key – a key we can use - default is fine

Returns:

returns an operator

dlsia.core.networks.tunet3plus.max_pool_size_result(Nin, kernel, stride, dilation=1, padding=0)

Determine the spatial dimension size after a max pooling operation

Parameters:

Nin – dimension of 1d array
kernel – kernel size
stride – stride; might need to match kernel size
dilation – dilation factor
padding – padding parameter

Returns:

the resulting array length

dlsia.core.networks.tunet3plus.skip_connection_sizing_chart(dim, chart, kernel_size, stride, encoder_channels, decoder_channels, carryover_channels, up_down_padding=0, dilation=1)

Build a sizing chart for the (depth * (depth - 1)) skip connections, including the required contraction/expansion/concatenations operators and corresponding operator parameters. In total, each decoder will receive a single connection from each layer.

Parameters:

dim – x or y (or z) spatial dimension index
chart – the skip connection sizing chart; for extracting the outgoing and incoming spatial dimensions at each layer.
kernel_size – The size of the upsampling and downsampling kernel used to step up and down.
stride – The stride we want to use. Typically defaulted to the kernel_size
encoder_channels – TBD
decoder_channels – TBD
carryover_channels – TBD
up_down_padding – upsampling and downsampling padding, Default is 0
dilation – the dilation factor. Default is 1

Returns:

A dictionary with information regarding the appropriate up/down operations or concatenation and required parameters.

dlsia.core.networks.tunet3plus.tst()

dlsia.core.networks.tunet3plus.unet_sizing_chart(N, depth, stride, maxpool_kernel_size, up_down_padding=0, dilation=1)

Build a set of dictionaries that are useful to make sure that we can map arrays back to the right sizes for each downsampling and upsampling operation.

Parameters:

N – dimension of array
depth – the total depth of the unet
stride – the stride - we fix this for a single UNet
maxpool_kernel_size – the max pooling kernel size
up_down_padding – max pooling and convT padding, Default is 0
dilation – the dilation factor. default is 1

Returns:

a dictionary with information regarding each upsampling and downsampling operation

The data associated with key “Sizes” provides images size per depth The data associated with key “Pool Setting” provides info needed to construct a MaxPool operator The data associated with key “convT Setting” provides info need to construct transposed convolutions such that the image of a the right size is constructed.

dlsia.core.networks.graph_utils module

dlsia.core.networks.graph_utils.assign_size(G, powers, p_power=None)

dlsia.core.networks.graph_utils.random_LL_graph(N, alpha, gamma, min_degree=None, max_degree=None, return_graph=False)

class dlsia.core.networks.graph_utils.random_graph_MT(N, alpha=1.0, min_degree=None, max_degree=None, gamma=2.5)

Bases: object

node_model(ii)

random_graph()

dlsia.core.networks.graph_utils.tst()

dlsia.core.networks.scale_up_down module

Some tools to compute padding and output_pad sizes.

dlsia.core.networks.scale_up_down.conv_padding(dil, kernel): Do we need a function for this? :param dil: Dilation :param kernel: Stride :return: needed padding value

dlsia.core.networks.scale_up_down.get_outpadding_convT(Nsmall, Nbig, ker, stride, dil, padding)

Compute the padding and output padding values neccessary for matching Nsmall to Nbig dimensionality after an application of nn.ConvTranspose

Parameters:

Nsmall – small array dimensions (start)
Nbig – big array dimension (end)
ker – kernel size
stride – stride
dil – dilation
padding – padding

Returns:

the padding and output_padding

dlsia.core.networks.scale_up_down.get_outpadding_upsampling(Nsmall, Nbig, factor)

Computes the extra padding value necessary for matching Nsmall to Nbig dimensionality after an application of nn.Upsample

Parameters:

Nsmall – small array dimensions (start)
Nbig – big array dimension (end)
factor – the upsampling sizing factor

Returns:

the padding and output_padding

dlsia.core.networks.scale_up_down.resulting_convT_size(Hin, dil, pad, stride, ker, outp)

Computes the resulting size of a tensor dimension given convT input parameters

Parameters:

Hin (input dimension) –
dil (dilation) –
pad (padding) –
stride (stride) –
ker (kernel size) –
outp (the outp parameter) –

Return type:

the size of the resulting tensor

dlsia.core.networks.scale_up_down.resulting_conv_size(Hin, dil, pad, stride, ker)

Computes the resulting size of a tensor dimension given conv input parameters

Parameters:

Hin (input dimension) –
dil (dilation) –
pad (padding) –
stride (stride) –
ker (kernsel size) –

Return type:

the size of the resulting tensor

dlsia.core.networks.scale_up_down.scaling_table(input_size, stride_base, min_power, max_power, kernel): A generic scaling table for a variety of possible scale change options. :param input_size: input image size :param stride_base: the stride_base we want to use :param min_power: determines the minimum stride: stride = stride_base**min_power :param max_power: determines the maximum stride: stride = stride_base**min_power :param kernel: kernel size :return: A dict with various settings #TODO: DEBUG THIS for stride_base!=2

Module contents

All dlsia models