Prediction on a pre-trained neural network

I would like to define my own neural network and do predictions on it without performing any training.

Something like:

1. Define layers
2. Set the weights and biases of each layer (based on a trained model from another context)
3. Perform prediction

An example of 1) is below

layers = [
    imageInputLayer([32 32 1])   
    convolution2dLayer(5,6)
    batchNormalizationLayer
    reluLayer       
    maxPooling2dLayer(2,'Stride',2)    
    convolution2dLayer(5,6)
    batchNormalizationLayer
    reluLayer    
    maxPooling2dLayer(2,'Stride',2)    
    fullyConnectedLayer(80)
    batchNormalizationLayer
    reluLayer          
    fullyConnectedLayer(6)
    batchNormalizationLayer
    ];

Any pointers on 2) and 3) would be appreciated.

Answer 1

[Srivardhan Gadila]

Using weights of different network to the current network:

for 2 networks net1 & net2, copy the layers to some variable

layers = net1.Layers;
layers(net1layerindex).Weights(net1index) = net2.Layers(net2layerindex).Weights(net2index);

Let's say you want to set weights of a channel of a convolution2dLayer to zeros(channelDimension)

layers(convLayerIndex).Weights(:,:,channelIndex) = zeros(channelDimension)
%also
layers(convLayerIndex).Weights(:,:,channelIndex,Index) = zeros(respectiveDimension)

In case of batchNormalizationLayer it has TrainedMean, TrainedVariance, Scale & Offset.

Use the above layers to create a DAG or a Series network. You can also refer to the documentation of the assembleNetwork function which helps creating deep learning networks from layers without training.

You can also set the Learning factor for a layer as follows:

layer.WeightLearnRateFactor = 0;

[Srivardhan Gadila]

The similar thing applies to bias also