How can I train a regression layer using the autoencoder approach

I am trying to adapt example provided here

https://www.mathworks.com/help/deeplearning/ug/create-simple-deep-learning-network-for-classification.html

Except that I want to replace 

softnet = trainSoftmaxLayer(feat2,tTrain,'MaxEpochs',400);

With something similar to this 

options = trainingOptions('sgdm','MaxEpochs',20,...'InitialLearnRate',0.0001);

routputlayer = regressionLayer('Name','routput');

trainedROL = trainNetwork(feat2,yTrain,routputlayer,options);

However, I receive the following error:

ERROR: Error using trainNetwork>iAssertXAndYHaveSameNumberOfObservations (line 604)
X and Y must have the same number of observations.

Error in trainNetwork>iParseInput (line 336)
    iAssertXAndYHaveSameNumberOfObservations( X, Y );

Error in trainNetwork (line 68)
[layers, opts, X, Y] = iParseInput(varargin{:});

Error in test (line 176)
trainedROL = trainNetwork(feat2,tTrain,routputlayer,options);

How can I modify the algorithm to do regression rather than classification?

Answer 1

[MathWorks Support Team]

Regression is not natively supported within the autoencoder framework. There is no equivalent to the trainSoftmaxLayer function which accepts a feature input matrix of dimensions featureSize-by-numObs.

The trainNetwork function in MATLAB R2017a is designed for image learning problems – i.e. when the input data has dimensions height-by-width-by-channels-by-numObs. So the autoencoder output is not natively supported by trainNetwork.

However, you can manipulate the dimensions of the autoencoded features to make it compatible with the regressionLayer in trainNetwork. See below an example script which demonstrates this, using the feat2 output from the second autoencoder from the example in "Train Stacked Autoencoders for Image Classification". Again, keep in mind this is not quite the intended workflow for either autoencoders or SeriesNetworks from trainNetwork.

The key here is to reshape the data into image format, and to include an input layer and fully connected layer alongside the regressionLayer in the output.

%% Layers
layers = [ imageInputLayer([1 50])
    fullyConnectedLayer(10)
    regressionLayer() ];

%% Manipulate feature data
% Reshape to image format ([H x W x C x N])
feat2ImageFormat = reshape( feat2, [1 50 1 5000] );

%% Train
trainReg = trainNetwork( feat2ImageFormat, tTrain', layers, trainingOptions('sgdm') );