Layer 'conv_1': Input size mismatch. Size of input to this layer is different from the expected input size.

Hi I am trying to run some data through the video classification network from MATLAB found here: https://uk.mathworks.com/help/deeplearning/examples/classify-videos-using-deep-learning.html

I have used the principles of the ECG tutorial (https://uk.mathworks.com/help/signal/examples/classify-ecg-signals-using-long-short-term-memory-networks.html) to create the code that prepares the data (everything up to the video classification part which is noted as a comment).

My train data is a cell array (4000 x 1), each cell contains a matrix of data (3 features x variable length, 2048 is longest length). The network has been created using the deep network designer.

My problem is that when I try to train the network, I get this error:-

>> ECG_Modified3

Error using trainNetwork (line 165)

Invalid network.

Error in ECG_Modified3 (line 125)

net = trainNetwork(XTrain,YTrain,lgraph,options);

Caused by:

Layer 'conv_1': Input size mismatch. Size of input to this layer is different from the expected input size.

Inputs to this layer:

from layer 'seqfold' output 'out' (3×1×1×4000 output)

I assume the issue is the inputsize is incorrect however I have tried many different combinations of scalar and 3 and 4 element vectors (e.g 4000, 3, 1, [4000 3 1], [1 1 4000], etc). I did manage to run the network once using an inputsize of [1 3 1] but it stopped working the next day for apparently no reason, with a different error which was:-

>> ECG_Modified3

Error using trainNetwork (line 165)

Invalid training data. Predictors must be a N-by-1 cell array of sequences, where N is the number of sequences. All sequences must have the

same feature dimension and at least one time step.

Error in ECG_Modified3 (line 125)

net = trainNetwork(XTrain,YTrain,lgraph,options);

The only other thing I can think of is that the conv_1 settings are incorrect but I am not knowledgeable enough on this subject to correct the situation.

If anyone could give some insight on this I would greatly appreciate it.

Thank you.

HFI = sequence{1}; 
S = sequence{4};
OF = sequence{21};
VS = sequence{63};
UF = sequence{139};

HFIX = sequence(label_array=='High Frequency Impulse');
HFIY = label_array(label_array=='High Frequency Impulse');
SX = sequence(label_array=='Snapshot');
SY = label_array(label_array=='Snapshot');
OFX = sequence(label_array=='Over-frequency');
OFY = label_array(label_array=='Over-frequency');
VSX = sequence(label_array=='Voltage Sag');
VSY = label_array(label_array=='Voltage Sag');
UFX = sequence(label_array=='Under-frequency');
UFY = label_array(label_array=='Under-frequency');

[trainIndHFI,~,testIndHFI] = dividerand(117,0.9,0.0,0.1); 
[trainIndS,~,testIndS] = dividerand(976,0.9,0.0,0.1);
[trainIndOF,~,testIndOF] = dividerand(26,0.9,0.0,0.1);
[trainIndVS,~,testIndVS] = dividerand(59,0.9,0.0,0.1);
[trainIndUF,~,testIndUF] = dividerand(30,0.9,0.0,0.1);

XTrainHFI = HFIX(trainIndHFI);
YTrainHFI = HFIY(trainIndHFI);
XTrainS = SX(trainIndS);
YTrainS = SY(trainIndS);
XTrainOF = OFX(trainIndOF);
YTrainOF = OFY(trainIndOF);
XTrainVS = VSX(trainIndVS);
YTrainVS = VSY(trainIndVS);
XTrainUF = UFX(trainIndUF);
YTrainUF = UFY(trainIndUF);

XTestHFI = HFIX(testIndHFI);
YTestHFI = HFIY(testIndHFI);
XTestS = SX(testIndS);
YTestS = SY(testIndS);
XTestOF = OFX(testIndOF);
YTestOF = OFY(testIndOF);
XTestVS = VSX(testIndVS);
YTestVS = VSY(testIndVS);
XTestUF = UFX(testIndUF);
YTestUF = UFY(testIndUF); 

XTrain = [repmat(XTrainHFI(1:100),8,1); XTrainS(1:800); repmat(XTrainOF(1:20),40,1); repmat(XTrainVS(1:50),16,1); repmat(XTrainUF(1:20),40,1)]; %4000 total
YTrain = [repmat(YTrainHFI(1:100),8,1); YTrainS(1:800); repmat(YTrainOF(1:20),40,1); repmat(YTrainVS(1:50),16,1); repmat(YTrainUF(1:20),40,1)]; %4000 total
XTest = [repmat(XTestHFI(1:12),8,1); XTestS(1:96); repmat(XTestOF(1:3),32,1); repmat(XTestVS(1:6),16,1); repmat(XTestUF(1:3),32,1)]; %480 total
YTest = [repmat(YTestHFI(1:12),8,1); YTestS(1:96); repmat(YTestOF(1:3),32,1); repmat(YTestVS(1:6),16,1); repmat(YTestUF(1:3),32,1);]; %480 total


%ECG tutorial bilstm network:-
%     layers = [ ...
%         sequenceInputLayer(3)
%         bilstmLayer(100,'OutputMode','last') 
%         fullyConnectedLayer(2)
%         softmaxLayer
%         classificationLayer
%         ];
%  
% 
 

%video classification network example:-
  lgraph = layerGraph();
  tempLayers = [
      sequenceInputLayer([3 1 1 4000],"Name","sequence")
      sequenceFoldingLayer("Name","seqfold")];
  lgraph = addLayers(lgraph,tempLayers);
  
  tempLayers = [
      convolution2dLayer([3 3],3,"Name","conv_1","Padding","same")
      maxPooling2dLayer([3 3],"Name","maxpool_1","Padding","same")
      convolution2dLayer([3 3],3,"Name","conv_2","Padding","same")
      maxPooling2dLayer([3 3],"Name","maxpool_2","Padding","same")];
  lgraph = addLayers(lgraph,tempLayers);
  
  tempLayers = [
      sequenceUnfoldingLayer("Name","sequnfold")
      flattenLayer("Name","flatten")
      lstmLayer(128,"Name","lstm")
      softmaxLayer("Name","softmax")
      classificationLayer("Name","classoutput")];
  lgraph = addLayers(lgraph,tempLayers);
  lgraph = connectLayers(lgraph,"seqfold/out","conv_1");
  lgraph = connectLayers(lgraph,"seqfold/miniBatchSize","sequnfold/miniBatchSize");
  lgraph = connectLayers(lgraph,"maxpool_2","sequnfold/in");
  clear tempLayers;
 % plot(lgraph);

 options = trainingOptions('adam', ...
     'ExecutionEnvironment','gpu', ... 
     'Shuffle','every-epoch', ...
     'MaxEpochs',1, ...
     'MiniBatchSize', 1, ...
     'InitialLearnRate', 0.01, ...
     'SequenceLength', 'longest', ...
     'GradientThreshold', 1, ...
     'plots','training-progress', ...
     'Verbose',false);


  net = trainNetwork(XTrain,YTrain,lgraph,options);
  trainPred = classify(net,XTrain,'SequenceLength',1000); 
  plotconfusion(YTrain',trainPred','Training Accuracy')
  figure
  testPred = classify(net,XTest,'SequenceLength',1000);
  plotconfusion(YTest',testPred','Testing Accuracy')

Answer 1

[Asvin Kumar]

The sequenceInputLayer accepts as ‘inputSize’ the number of features of the input data. Setting the ‘inputSize’ to [3 1 1] should resolve the error. There is no necessity to provide the size of the dataset.

The help page for the sequenceInputLayer is:

https://www.mathworks.com/help/deeplearning/ref/nnet.cnn.layer.sequenceinputlayer.html

Answer 2

[Sabbir Ahmed]

This error arises when you use a 4D input layer(in your case it is sequenceInputLayer) but you don't use a 3D convolutional or max pooling layer. You can use 2D convolutional or max pooling layer when you use 3D input layer. Moreover make sure that you changed the filter size and stride accordingly unless don't want to see another error.

I have faced the same problem while working with 4D matrix.