Model Layers¶

This module contains many layer classes that we might be interested in using in our models. These layers complement the default Pytorch layers which we can also use as predefined layers.

In [1]:

from fastai.vision import *
from fastai.gen_doc.nbdoc import *

Custom fastai modules¶

In [2]:

show_doc(AdaptiveConcatPool2d, title_level=3)

`class` `AdaptiveConcatPool2d`[source][test]

AdaptiveConcatPool2d(sz:Optional[int]=*None*) :: PrePostInitMeta :: Module

No tests found for AdaptiveConcatPool2d. To contribute a test please refer to this guide and this discussion.

Layer that concats AdaptiveAvgPool2d and AdaptiveMaxPool2d.

In [3]:

from fastai.gen_doc.nbdoc import *
from fastai.layers import * 

The output will be 2*sz, or just 2 if sz is None.

The AdaptiveConcatPool2d object uses adaptive average pooling and adaptive max pooling and concatenates them both. We use this because it provides the model with the information of both methods and improves performance. This technique is called adaptive because it allows us to decide on what output dimensions we want, instead of choosing the input's dimensions to fit a desired output size.

Let's try training with Adaptive Average Pooling first, then with Adaptive Max Pooling and finally with the concatenation of them both to see how they fare in performance.

We will first define a simple_cnn using Adaptive Max Pooling by changing the source code a bit.

In [ ]:

path = untar_data(URLs.MNIST_SAMPLE)
data = ImageDataBunch.from_folder(path)

In [ ]:

def simple_cnn_max(actns:Collection[int], kernel_szs:Collection[int]=None,
               strides:Collection[int]=None) -> nn.Sequential:
    "CNN with `conv2d_relu` layers defined by `actns`, `kernel_szs` and `strides`"
    nl = len(actns)-1
    kernel_szs = ifnone(kernel_szs, [3]*nl)
    strides    = ifnone(strides   , [2]*nl)
    layers = [conv_layer(actns[i], actns[i+1], kernel_szs[i], stride=strides[i])
        for i in range(len(strides))]
    layers.append(nn.Sequential(nn.AdaptiveMaxPool2d(1), Flatten()))
    return nn.Sequential(*layers)

In [ ]:

model = simple_cnn_max((3,16,16,2))
learner = Learner(data, model, metrics=[accuracy])
learner.fit(1)

Total time: 00:02

epoch	train_loss	valid_loss	accuracy
1	0.102758	0.064676	0.984298

Now let's try with Adaptive Average Pooling now.

In [ ]:

def simple_cnn_avg(actns:Collection[int], kernel_szs:Collection[int]=None,
               strides:Collection[int]=None) -> nn.Sequential:
    "CNN with `conv2d_relu` layers defined by `actns`, `kernel_szs` and `strides`"
    nl = len(actns)-1
    kernel_szs = ifnone(kernel_szs, [3]*nl)
    strides    = ifnone(strides   , [2]*nl)
    layers = [conv_layer(actns[i], actns[i+1], kernel_szs[i], stride=strides[i])
        for i in range(len(strides))]
    layers.append(nn.Sequential(nn.AdaptiveAvgPool2d(1), Flatten()))
    return nn.Sequential(*layers)

In [ ]:

model = simple_cnn_avg((3,16,16,2))
learner = Learner(data, model, metrics=[accuracy])
learner.fit(1)

Total time: 00:02

epoch	train_loss	valid_loss	accuracy
1	0.241485	0.201116	0.973994

Finally we will try with the concatenation of them both AdaptiveConcatPool2d. We will see that, in fact, it increases our accuracy and decreases our loss considerably!

In [ ]:

def simple_cnn(actns:Collection[int], kernel_szs:Collection[int]=None,
               strides:Collection[int]=None) -> nn.Sequential:
    "CNN with `conv2d_relu` layers defined by `actns`, `kernel_szs` and `strides`"
    nl = len(actns)-1
    kernel_szs = ifnone(kernel_szs, [3]*nl)
    strides    = ifnone(strides   , [2]*nl)
    layers = [conv_layer(actns[i], actns[i+1], kernel_szs[i], stride=strides[i])
        for i in range(len(strides))]
    layers.append(nn.Sequential(AdaptiveConcatPool2d(1), Flatten()))
    return nn.Sequential(*layers)

In [ ]:

model = simple_cnn((3,16,16,2))
learner = Learner(data, model, metrics=[accuracy])
learner.fit(1)

Total time: 00:02

epoch	train_loss	valid_loss	accuracy
1	0.203015	0.122094	0.988224

In [4]:

show_doc(Lambda, title_level=3)

`class` `Lambda`[source][test]

Lambda(func:LambdaFunc) :: PrePostInitMeta :: Module

No tests found for Lambda. To contribute a test please refer to this guide and this discussion.

Create a layer that simply calls func with x

This is very useful to use functions as layers in our networks inside a Sequential object. So, for example, say we want to apply a log_softmax loss and we need to change the shape of our output batches to be able to use this loss. We can add a layer that applies the necessary change in shape by calling:

Lambda(lambda x: x.view(x.size(0),-1))

Let's see an example of how the shape of our output can change when we add this layer.

In [ ]:

model = nn.Sequential(
    nn.Conv2d(3,  16, kernel_size=3, stride=2, padding=1), nn.ReLU(),
    nn.Conv2d(16, 16, kernel_size=3, stride=2, padding=1), nn.ReLU(),
    nn.Conv2d(16, 10, kernel_size=3, stride=2, padding=1), nn.ReLU(),
    nn.AdaptiveAvgPool2d(1),
)

model.cuda()

for xb, yb in data.train_dl:
    out = (model(*[xb]))
    print(out.size())
    break

torch.Size([64, 10, 1, 1])

In [ ]:

model = nn.Sequential(
    nn.Conv2d(3,  16, kernel_size=3, stride=2, padding=1), nn.ReLU(),
    nn.Conv2d(16, 16, kernel_size=3, stride=2, padding=1), nn.ReLU(),
    nn.Conv2d(16, 10, kernel_size=3, stride=2, padding=1), nn.ReLU(),
    nn.AdaptiveAvgPool2d(1),
    Lambda(lambda x: x.view(x.size(0),-1))
)

model.cuda()

for xb, yb in data.train_dl:
    out = (model(*[xb]))
    print(out.size())
    break

torch.Size([64, 10])

In [5]:

show_doc(Flatten)

`class` `Flatten`[source][test]

Flatten(full:bool=*False*) :: PrePostInitMeta :: Module

No tests found for Flatten. To contribute a test please refer to this guide and this discussion.

Flatten x to a single dimension, often used at the end of a model. full for rank-1 tensor

The function we build above is actually implemented in our library as Flatten. We can see that it returns the same size when we run it.

In [ ]:

model = nn.Sequential(
    nn.Conv2d(3,  16, kernel_size=3, stride=2, padding=1), nn.ReLU(),
    nn.Conv2d(16, 16, kernel_size=3, stride=2, padding=1), nn.ReLU(),
    nn.Conv2d(16, 10, kernel_size=3, stride=2, padding=1), nn.ReLU(),
    nn.AdaptiveAvgPool2d(1),
    Flatten(),
)

model.cuda()

for xb, yb in data.train_dl:
    out = (model(*[xb]))
    print(out.size())
    break

torch.Size([64, 10])

In [6]:

show_doc(PoolFlatten)

`PoolFlatten`[source][test]

PoolFlatten() → Sequential

No tests found for PoolFlatten. To contribute a test please refer to this guide and this discussion.

Apply nn.AdaptiveAvgPool2d to x and then flatten the result.

We can combine these two final layers (AdaptiveAvgPool2d and Flatten) by using PoolFlatten.

In [ ]:

model = nn.Sequential(
    nn.Conv2d(3,  16, kernel_size=3, stride=2, padding=1), nn.ReLU(),
    nn.Conv2d(16, 16, kernel_size=3, stride=2, padding=1), nn.ReLU(),
    nn.Conv2d(16, 10, kernel_size=3, stride=2, padding=1), nn.ReLU(),
    PoolFlatten()
)

model.cuda()

for xb, yb in data.train_dl:
    out = (model(*[xb]))
    print(out.size())
    break

torch.Size([64, 10])

Another use we give to the Lambda function is to resize batches with ResizeBatch when we have a layer that expects a different input than what comes from the previous one.

In [7]:

show_doc(ResizeBatch)

`class` `ResizeBatch`[source][test]

ResizeBatch(***size**:int) :: PrePostInitMeta :: Module

No tests found for ResizeBatch. To contribute a test please refer to this guide and this discussion.

Reshape x to size, keeping batch dim the same size

In [ ]:

a = torch.tensor([[1., -1.], [1., -1.]])[None]
print(a)

tensor([[[ 1., -1.],
         [ 1., -1.]]])

In [ ]:

out = ResizeBatch(4)
print(out(a))

tensor([[ 1., -1.,  1., -1.]])

In [8]:

show_doc(Debugger, title_level=3)

`class` `Debugger`[source][test]

Debugger() :: PrePostInitMeta :: Module

No tests found for Debugger. To contribute a test please refer to this guide and this discussion.

A module to debug inside a model.

The debugger module allows us to peek inside a network while its training and see in detail what is going on. We can see inputs, outputs and sizes at any point in the network.

For instance, if you run the following:

model = nn.Sequential(
    nn.Conv2d(3,  16, kernel_size=3, stride=2, padding=1), nn.ReLU(),
    Debugger(),
    nn.Conv2d(16, 16, kernel_size=3, stride=2, padding=1), nn.ReLU(),
    nn.Conv2d(16, 10, kernel_size=3, stride=2, padding=1), nn.ReLU(),
)

model.cuda()

learner = Learner(data, model, metrics=[accuracy])
learner.fit(5)

... you'll see something like this:

/home/ubuntu/fastai/fastai/layers.py(74)forward()
     72     def forward(self,x:Tensor) -> Tensor:
     73         set_trace()
---> 74         return x
     75 
     76 class StdUpsample(nn.Module):

ipdb>

In [9]:

show_doc(PixelShuffle_ICNR, title_level=3)

`class` `PixelShuffle_ICNR`[source][test]

PixelShuffle_ICNR(ni:int, nf:int=*None, scale:int=2, blur:bool=False, norm_type=<NormType.Weight: 3>, leaky:float=None*) :: PrePostInitMeta :: Module

No tests found for PixelShuffle_ICNR. To contribute a test please refer to this guide and this discussion.

Upsample by scale from ni filters to nf (default ni), using nn.PixelShuffle, icnr init, and weight_norm.

In [10]:

show_doc(MergeLayer, title_level=3)

`class` `MergeLayer`[source][test]

MergeLayer(dense:bool=*False*) :: PrePostInitMeta :: Module

No tests found for MergeLayer. To contribute a test please refer to this guide and this discussion.

Merge a shortcut with the result of the module by adding them or concatenating them if dense=True.

In [11]:

show_doc(PartialLayer, title_level=3)

`class` `PartialLayer`[source][test]

PartialLayer(func, ****kwargs**) :: PrePostInitMeta :: Module

No tests found for PartialLayer. To contribute a test please refer to this guide and this discussion.

Layer that applies partial(func, **kwargs).

In [12]:

show_doc(SigmoidRange, title_level=3)

`class` `SigmoidRange`[source][test]

SigmoidRange(low, high) :: PrePostInitMeta :: Module

No tests found for SigmoidRange. To contribute a test please refer to this guide and this discussion.

Sigmoid module with range (low,x_max)

In [13]:

show_doc(SequentialEx, title_level=3)

`class` `SequentialEx`[source][test]

SequentialEx(***layers**) :: PrePostInitMeta :: Module

No tests found for SequentialEx. To contribute a test please refer to this guide and this discussion.

Like nn.Sequential, but with ModuleList semantics, and can access module input

In [14]:

show_doc(SelfAttention, title_level=3)

`class` `SelfAttention`[source][test]

SelfAttention(n_channels:int) :: PrePostInitMeta :: Module

Tests found for SelfAttention:

pytest -sv tests/test_torch_core.py::test_keep_parameter [source]

To run tests please refer to this guide.

Self attention layer for nd.

In [15]:

show_doc(BatchNorm1dFlat, title_level=3)

`class` `BatchNorm1dFlat`[source][test]

BatchNorm1dFlat(num_features, eps=*1e-05, momentum=0.1, affine=True, track_running_stats=True*) :: BatchNorm1d

No tests found for BatchNorm1dFlat. To contribute a test please refer to this guide and this discussion.

nn.BatchNorm1d, but first flattens leading dimensions

Loss functions¶

In [16]:

show_doc(FlattenedLoss, title_level=3)

`class` `FlattenedLoss`[source][test]

FlattenedLoss(func, ***args, axis:int=-1, floatify:bool=False, is_2d:bool=True, **kwargs**)

No tests found for FlattenedLoss. To contribute a test please refer to this guide and this discussion.

Same as func, but flattens input and target.

Create an instance of func with args and kwargs. When passing an output and target, it

puts axis first in output and target with a transpose
casts the target to float if floatify=True
squeezes the output to two dimensions if is_2d, otherwise one dimension, squeezes the target to one dimension
applies the instance of func.

In [17]:

show_doc(BCEFlat)

`BCEFlat`[source][test]

BCEFlat(***args, axis:int=-1, floatify:bool=True, **kwargs**)

No tests found for BCEFlat. To contribute a test please refer to this guide and this discussion.

Same as nn.BCELoss, but flattens input and target.

In [18]:

show_doc(BCEWithLogitsFlat)

`BCEWithLogitsFlat`[source][test]

BCEWithLogitsFlat(***args, axis:int=-1, floatify:bool=True, **kwargs**)

No tests found for BCEWithLogitsFlat. To contribute a test please refer to this guide and this discussion.

Same as nn.BCEWithLogitsLoss, but flattens input and target.

In [19]:

show_doc(CrossEntropyFlat)

`CrossEntropyFlat`[source][test]

CrossEntropyFlat(***args, axis:int=-1, **kwargs**)

No tests found for CrossEntropyFlat. To contribute a test please refer to this guide and this discussion.

Same as nn.CrossEntropyLoss, but flattens input and target.

In [20]:

show_doc(MSELossFlat)

`MSELossFlat`[source][test]

MSELossFlat(***args, axis:int=-1, floatify:bool=True, **kwargs**)

No tests found for MSELossFlat. To contribute a test please refer to this guide and this discussion.

Same as nn.MSELoss, but flattens input and target.

In [21]:

show_doc(NoopLoss)

`class` `NoopLoss`[source][test]

NoopLoss() :: PrePostInitMeta :: Module

No tests found for NoopLoss. To contribute a test please refer to this guide and this discussion.

Just returns the mean of the output.

In [22]:

show_doc(WassersteinLoss)

`class` `WassersteinLoss`[source][test]

WassersteinLoss() :: PrePostInitMeta :: Module

No tests found for WassersteinLoss. To contribute a test please refer to this guide and this discussion.

For WGAN.

Helper functions to create modules¶

In [23]:

show_doc(bn_drop_lin, doc_string=False)

`bn_drop_lin`[source][test]

bn_drop_lin(n_in:int, n_out:int, bn:bool=*True, p:float=0.0, actn:Optional[Module]=None*)

No tests found for bn_drop_lin. To contribute a test please refer to this guide and this discussion.

The bn_drop_lin function returns a sequence of batch normalization, dropout and a linear layer. This custom layer is usually used at the end of a model.

n_in represents the size of the input, n_out the size of the output, bn whether we want batch norm or not, p how much dropout, and actn (optional parameter) adds an activation function at the end.

In [24]:

show_doc(conv2d)

`conv2d`[source][test]

conv2d(ni:int, nf:int, ks:int=*3, stride:int=1, padding:int=None, bias=False, init:LayerFunc='kaiming_normal_'*) → Conv2d

No tests found for conv2d. To contribute a test please refer to this guide and this discussion.

Create and initialize nn.Conv2d layer. padding defaults to ks//2.

In [25]:

show_doc(conv2d_trans)

`conv2d_trans`[source][test]

conv2d_trans(ni:int, nf:int, ks:int=*2, stride:int=2, padding:int=0, bias=False*) → ConvTranspose2d

No tests found for conv2d_trans. To contribute a test please refer to this guide and this discussion.

Create nn.ConvTranspose2d layer.

In [26]:

show_doc(conv_layer, doc_string=False)

`conv_layer`[source][test]

conv_layer(ni:int, nf:int, ks:int=*3, stride:int=1, padding:int=None, bias:bool=None, is_1d:bool=False, norm_type:Optional[NormType]=<NormType.Batch: 1>, use_activ:bool=True, leaky:float=None, transpose:bool=False, init:Callable='kaiming_normal_', self_attention:bool=False*)

No tests found for conv_layer. To contribute a test please refer to this guide and this discussion.

The conv_layer function returns a sequence of nn.Conv2D, BatchNorm and a ReLU or leaky RELU activation function.

n_in represents the size of the input, n_out the size of the output, ks the kernel size, stride the stride with which we want to apply the convolutions. bias will decide if they have bias or not (if None, defaults to True unless using batchnorm). norm_type selects the type of normalization (or None). If leaky is None, the activation is a standard ReLU, otherwise it's a LeakyReLU of slope leaky. Finally if transpose=True, the convolution is replaced by a ConvTranspose2D.

In [27]:

show_doc(embedding, doc_string=False)

`embedding`[source][test]

embedding(ni:int, nf:int) → Module

No tests found for embedding. To contribute a test please refer to this guide and this discussion.

Create an embedding layer with input size ni and output size nf.

In [28]:

show_doc(relu)

`relu`[source][test]

relu(inplace:bool=*False, leaky:float=None*)

No tests found for relu. To contribute a test please refer to this guide and this discussion.

Return a relu activation, maybe leaky and inplace.

In [29]:

show_doc(res_block)

`res_block`[source][test]

res_block(nf, dense:bool=*False, norm_type:Optional[NormType]=<NormType.Batch: 1>, bottle:bool=False, ***conv_kwargs**)

No tests found for res_block. To contribute a test please refer to this guide and this discussion.

Resnet block of nf features. conv_kwargs are passed to conv_layer.

In [30]:

show_doc(sigmoid_range)

`sigmoid_range`[source][test]

sigmoid_range(x, low, high)

No tests found for sigmoid_range. To contribute a test please refer to this guide and this discussion.

Sigmoid function with range (low, high)

In [31]:

show_doc(simple_cnn)

`simple_cnn`[source][test]

simple_cnn(actns:Collection[int], kernel_szs:Collection[int]=*None, strides:Collection[int]=None, bn=False*) → Sequential

No tests found for simple_cnn. To contribute a test please refer to this guide and this discussion.

CNN with conv_layer defined by actns, kernel_szs and strides, plus batchnorm if bn.

Initialization of modules¶

In [32]:

show_doc(batchnorm_2d)

`batchnorm_2d`[source][test]

batchnorm_2d(nf:int, norm_type:NormType=*<NormType.Batch: 1>*)

No tests found for batchnorm_2d. To contribute a test please refer to this guide and this discussion.

A batchnorm2d layer with nf features initialized depending on norm_type.

In [33]:

show_doc(icnr)

`icnr`[source][test]

icnr(x, scale=*2, init='kaiming_normal_'*)

No tests found for icnr. To contribute a test please refer to this guide and this discussion.

ICNR init of x, with scale and init function.

In [34]:

show_doc(trunc_normal_)

`trunc_normal_`[source][test]

trunc_normal_(x:Tensor, mean:float=*0.0, std:float=1.0*) → Tensor

No tests found for trunc_normal_. To contribute a test please refer to this guide and this discussion.

Truncated normal initialization.

In [35]:

show_doc(icnr)

`icnr`[source][test]

icnr(x, scale=*2, init='kaiming_normal_'*)

No tests found for icnr. To contribute a test please refer to this guide and this discussion.

ICNR init of x, with scale and init function.

In [36]:

show_doc(NormType)

`NormType`[test]

Enum = [Batch, BatchZero, Weight, Spectral]

No tests found for NormType. To contribute a test please refer to this guide and this discussion.

An enumeration.

Undocumented Methods - Methods moved below this line will intentionally be hidden¶

In [37]:

show_doc(Debugger.forward)

`forward`[source][test]

forward(x:Tensor) → Tensor

No tests found for forward. To contribute a test please refer to this guide and this discussion.

In [38]:

show_doc(Lambda.forward)

`forward`[source][test]

forward(x)

No tests found for forward. To contribute a test please refer to this guide and this discussion.

In [39]:

show_doc(AdaptiveConcatPool2d.forward)

`forward`[source][test]

forward(x)

No tests found for forward. To contribute a test please refer to this guide and this discussion.

In [40]:

show_doc(NoopLoss.forward)

`forward`[source][test]

forward(output, ***args**)

No tests found for forward. To contribute a test please refer to this guide and this discussion.

In [41]:

show_doc(PixelShuffle_ICNR.forward)

`forward`[source][test]

forward(x)

No tests found for forward. To contribute a test please refer to this guide and this discussion.

In [42]:

show_doc(WassersteinLoss.forward)

`forward`[source][test]

forward(real, fake)

No tests found for forward. To contribute a test please refer to this guide and this discussion.

In [43]:

show_doc(MergeLayer.forward)

`forward`[source][test]

forward(x)

No tests found for forward. To contribute a test please refer to this guide and this discussion.

In [44]:

show_doc(SigmoidRange.forward)

`forward`[source][test]

forward(x)

No tests found for forward. To contribute a test please refer to this guide and this discussion.

In [45]:

show_doc(MergeLayer.forward)

`forward`[source][test]

forward(x)

No tests found for forward. To contribute a test please refer to this guide and this discussion.

In [46]:

show_doc(SelfAttention.forward)

`forward`[source][test]

forward(x)

No tests found for forward. To contribute a test please refer to this guide and this discussion.

In [47]:

show_doc(SequentialEx.forward)

`forward`[source][test]

forward(x)

No tests found for forward. To contribute a test please refer to this guide and this discussion.

In [48]:

show_doc(SequentialEx.append)

`append`[source][test]

append(l)

No tests found for append. To contribute a test please refer to this guide and this discussion.

In [49]:

show_doc(SequentialEx.extend)

`extend`[source][test]

extend(l)

No tests found for extend. To contribute a test please refer to this guide and this discussion.

In [50]:

show_doc(SequentialEx.insert)

`insert`[source][test]

insert(i, l)

No tests found for insert. To contribute a test please refer to this guide and this discussion.

In [51]:

show_doc(PartialLayer.forward)

`forward`[source][test]

forward(x)

No tests found for forward. To contribute a test please refer to this guide and this discussion.

In [52]:

show_doc(BatchNorm1dFlat.forward)

`forward`[source][test]

forward(x)

No tests found for forward. To contribute a test please refer to this guide and this discussion.

In [53]:

show_doc(Flatten.forward)

`forward`[source][test]

forward(x)

No tests found for forward. To contribute a test please refer to this guide and this discussion.

New Methods - Please document or move to the undocumented section¶

In [54]:

show_doc(View)

`class` `View`[source][test]

View(***size**:int) :: PrePostInitMeta :: Module

No tests found for View. To contribute a test please refer to this guide and this discussion.

Reshape x to size

In [55]:

show_doc(ResizeBatch.forward)

`forward`[source][test]

forward(x)

No tests found for forward. To contribute a test please refer to this guide and this discussion.

In [56]:

show_doc(View.forward)

`forward`[source][test]

forward(x)

No tests found for forward. To contribute a test please refer to this guide and this discussion.

Model Layers¶

Custom fastai modules¶

class AdaptiveConcatPool2d[source][test]

class Lambda[source][test]

class Flatten[source][test]

PoolFlatten[source][test]

class ResizeBatch[source][test]

class Debugger[source][test]

class PixelShuffle_ICNR[source][test]

class MergeLayer[source][test]

class PartialLayer[source][test]

class SigmoidRange[source][test]

class SequentialEx[source][test]

class SelfAttention[source][test]

class BatchNorm1dFlat[source][test]

Loss functions¶

class FlattenedLoss[source][test]

BCEFlat[source][test]

BCEWithLogitsFlat[source][test]

CrossEntropyFlat[source][test]

MSELossFlat[source][test]

class NoopLoss[source][test]

class WassersteinLoss[source][test]

Helper functions to create modules¶

bn_drop_lin[source][test]

conv2d[source][test]

conv2d_trans[source][test]

conv_layer[source][test]

embedding[source][test]

relu[source][test]

res_block[source][test]

sigmoid_range[source][test]

simple_cnn[source][test]

Initialization of modules¶

batchnorm_2d[source][test]

icnr[source][test]

trunc_normal_[source][test]

icnr[source][test]

`NormType`[test]

Undocumented Methods - Methods moved below this line will intentionally be hidden¶

forward[source][test]

forward[source][test]

forward[source][test]

forward[source][test]

forward[source][test]

forward[source][test]

forward[source][test]

forward[source][test]

forward[source][test]

forward[source][test]

forward[source][test]

append[source][test]

extend[source][test]

insert[source][test]

forward[source][test]

forward[source][test]

forward[source][test]

New Methods - Please document or move to the undocumented section¶

class View[source][test]

forward[source][test]

forward[source][test]

`class` `AdaptiveConcatPool2d`[source][test]

`class` `Lambda`[source][test]

`class` `Flatten`[source][test]

`PoolFlatten`[source][test]

`class` `ResizeBatch`[source][test]

`class` `Debugger`[source][test]

`class` `PixelShuffle_ICNR`[source][test]

`class` `MergeLayer`[source][test]

`class` `PartialLayer`[source][test]

`class` `SigmoidRange`[source][test]

`class` `SequentialEx`[source][test]

`class` `SelfAttention`[source][test]

`class` `BatchNorm1dFlat`[source][test]

`class` `FlattenedLoss`[source][test]

`BCEFlat`[source][test]

`BCEWithLogitsFlat`[source][test]

`CrossEntropyFlat`[source][test]

`MSELossFlat`[source][test]

`class` `NoopLoss`[source][test]

`class` `WassersteinLoss`[source][test]

`bn_drop_lin`[source][test]

`conv2d`[source][test]

`conv2d_trans`[source][test]

`conv_layer`[source][test]

`embedding`[source][test]

`relu`[source][test]

`res_block`[source][test]

`sigmoid_range`[source][test]

`simple_cnn`[source][test]

`batchnorm_2d`[source][test]

`icnr`[source][test]

`trunc_normal_`[source][test]

`icnr`[source][test]

`forward`[source][test]

`forward`[source][test]

`forward`[source][test]

`forward`[source][test]

`forward`[source][test]

`forward`[source][test]

`forward`[source][test]

`forward`[source][test]

`forward`[source][test]

`forward`[source][test]

`forward`[source][test]

`append`[source][test]

`extend`[source][test]

`insert`[source][test]

`forward`[source][test]

`forward`[source][test]

`forward`[source][test]

`class` `View`[source][test]

`forward`[source][test]

`forward`[source][test]