Train hidden networks example

Hello,

I try to understand the AutoencoderDigitsexample on Matlab 2016a. There is 2 hidden layers , ones composed of 100 neurons and the second composed of 50 layer and the outpout composed of 10 neurons with softmax function.

I don't understand, as it is written it seems there are no output functions in the hidden layer:

autoenc1 = trainAutoencoder(xTrainImages,hiddenSize1, ...
    'MaxEpochs',400, ...
    'L2WeightRegularization',0.004, ...
    'SparsityRegularization',4, ...
    'SparsityProportion',0.15, ...
    'ScaleData', false);

My first question is : are there output functions on the hidden layers?

My second question is about the trainAutoencoder function. What are the goals of that function? What tries to train that function?

[Xiling Sheng]

same ask

Answer 1

[Tejas]

Hello Xavier,

I am assuming the example mentioned in the question, is the one provided in this documentation:

web(fullfile(docroot, "deeplearning/ug/train-stacked-autoencoders-for-image-classification.html"));

Hidden layers have associated output functions known as activation functions. Commonly used activation functions include:

• Sigmoid:

    web(fullfile(docroot, "deeplearning/ref/dlarray.sigmoid.html"));  

• ReLU:

    web(fullfile(docroot, "deeplearning/ref/dlarray.relu.html")); 

The 'trainAutoencoder' function is responsible for training the model on input data. In the example above, this function trains the model to identify patterns in input images, ultimately classifying them into digits from 0 to 9. This function is not trained by any other function, instead when called, it is the one that trains the model.