Can I use "softmaxLayer" for regression network?

Can I use "softmaxLayer" for regression network?

Answer 1

[MathWorks Support Team]

A softmax layer normalizes the input to the layer such that its elements sum up to 1. Therefore, it is useful when you want the network to compute a probability of a classification problem, since it ensures that the sum of the scores over the classes is 1, as requested for a probability measure.

However, it can be used also in combination with other layers as long as it makes sense to normalize the inputs to sum up to 1.

Thus, you can use "softmaxLayer" in MATLAB for your regression problem (although assessing its effectiveness would depend on the problem you are trying to solve at hand).

For example, you can create the following deep learning network:

>> layers = [ ...
>>    imageInputLayer([28 28 1]), ...
>>    convolution2dLayer(12,25), ...
>>    softmaxLayer, ...
>>    fullyConnectedLayer(1), ...
>>    regressionLayer]

Currently, a"softmaxLayer" cannot directly precede a "regressionLayer", which is why a "fullyConnectedLayer" is between the two in the above example. However, another workaround would be to define a custom regression output layer and a custom softmax layer. If you use a custom layers instead of the layers provided in the Deep Learning Toolbox, you can have a softmax layer immediately before a regression layer.

Here is what the "layers" array in the above example would look like using custom layers called "mySoftmaxLayer" and "myRegressionLayer":

>> layers = [ ...
>>    imageInputLayer([28 28 1]), ...
>>    convolution2dLayer(12,25), ...
>>    mySoftmaxLayer, ...
>>    myRegressionLayer]

For instructions and an example on how to define a custom regression output layer, please view the documentation page linked here:

https://www.mathworks.com/help/deeplearning/ug/define-custom-regression-output-layer.html

For instructions and an example on how to define a custom layer with learnable parameters, please refer to the documentation page linked here:

https://www.mathworks.com/help/deeplearning/ug/define-custom-deep-learning-layer.html