Converting training inputs to 4D array

Dear all,

I need to convert my training data which comprise real and imaginary values into a 4D matrix and I am getting trouble.

Here is the line of code I did so far with the data:

Train_inputX1 =All_data_segments; %5000 segments x 400 samples
Train_outputX1 =Modulation_Schemes ;% 5000 segments x 1
Test_X1 =ceil(0.1 *size(All_data_segments,1)); % 10% is catered for testing
idx1_t= randperm(size(All_data_segments,1),Test_X1);%This  will generate the 10% of the random test numbers from 1 to 5000
% 
% % Now I'm gonna partition the data accordingly...
testX1_input = All_data_segments(idx1_t',:); %Phase test input
testX1_output = Modulation_Schemes(idx1_t); %Phase test output
Train_inputX1(idx1_t',:) = []; % assigning '[]' to matrix entries deletes them.
Train_outputX1(idx1_t',:) =[]; % Deletes the outputs that was used to test input
% 
% %Now we need to repeat to find the validation data
Train1_inputX1 = Train_inputX1; %5000 segments x 400 samples
Train1_outputX1 =Train_outputX1;% 5000 segments x 1
Val_X1 =ceil(0.1 *size(Train_outputX1,1)); % 10% is catered for Validation
idx1_v= randperm(size(Train_outputX1,1),Val_X1);%This  will generate the 10% of the random test numbers from 1 to 5000
% 
 % Now I'm gonna partition the data accordingly...
 ValX1_input =Train1_inputX1(idx1_v',:); %Val input
 ValX1_output =Train_outputX1(idx1_v); % Val output
 Train1_inputX1(idx1_v',:) = []; % assigning '[]' to matrix entries deletes them.
 Train1_outputX1(idx1_v',:) =[]; % Deletes the outputs that was used to validation input


RealTrain_data = real(cell2mat(Train_inputX1));
ImTrain_data = Imag(cell2mat(Train_inputX1));

RealTest_data = real(cell2mat(testX1_input));
ImTest_data = Imag(cell2mat(testX1_inputX1));

RealVal_data = real(cell2mat(ValX1_input));
ImVal_data = Imag(cell2mat(ValX1_input));

Train_output = categorical(Train1_outputX1); %4050 segments
Test_output = categorical(testX1_output); % 500 segments
Val_output =categorical(ValX1_output);%450 segments

could you assist in converting the training, val and test input data into 4D arrays which comprise both real and imaginary parts

I am trying to build a sequence to 1 CNN to match the inputs to the corresponding modulation scheme

Thanks in advance

[Shivam Singh]

Hello tyler,

As per my understanding, you have a dataset with size = [5000, 400], which has complex values. You want to create a training dataset matrix with real values having real as well as complex part. So, you can convert it to size =[5000, 40, 2], where last dimesion can give you the real part information and complex part information separately.

Can you explain why do you want to go for 4D matrix? Can you provide more information about your model?

[tyler seudath]

Hi Shivam,

To train a CNN network, isnt the input vector supposed to be a 4D array?

What I have is 5000 segments which each contains 400 samples of complex data which is the input data. The output data is just the 5000 names of each modulation scheme so 1250 QAM, 1250 QPSK, 1250 16QAM , 1250 64 QAM. I converted to categorical vector ( This in total is a 5000 x 1 catergorical data).

This is how I divide my data in trainining, validation and testing into a 4D input array.

DataParts = zeros(size(Train_inputX1,1), size(Train_inputX1,2),1,2);
DataParts(:,:,:,1) = real(cell2mat(Train_inputX1));
DataParts(:,:,:,2) = imag(cell2mat(Train_inputX1)) ;
XTrain=num2cell(reshape(DataParts, [400,1,2,4500]));


DataParts1 = zeros(size(testX1_input,1), size(testX1_input,2),1, 2);
DataParts1(:,:,:,1) = real(cell2mat(testX1_input));
DataParts1(:,:,:,2) = imag(cell2mat(testX1_input)) ;
Ttrain=num2cell(reshape(DataParts1,[400,1,2,500]));

DataParts2 = zeros(size(ValX1_input,1), size(ValX1_input,2),1, 2);
DataParts2(:,:,:,1) = real(cell2mat(ValX1_input));
DataParts2(:,:,:,2) = imag(cell2mat(ValX1_input));
Vtrain =num2cell(reshape(DataParts2,[400,1,2,450]));


%VTrain=reshape(dataparts2_, [400,1,2,450]);
 Valoutfinal= categorical(ValX1_output);
 testoutfinal = categorical(testX1_output);
 Trainoutfinal= categorical(Train_outputX1);










%% NETWORK ARCHITECTURE
layers = [imageInputLayer([400 1 2])  % Creating the image layer
    %imageInputLayer([400 1 2])
    convolution2dLayer([102 1],3,'Stride',1)
    maxPooling2dLayer(2,'Stride',2,'Padding',[0 0 0 1])
    convolution2dLayer([24 1],10,'Stride',1)
    maxPooling2dLayer(2,'Stride',2,'Padding',[0 0 0 1])
    convolution2dLayer([11 1],10,'Stride',1)
    maxPooling2dLayer(2,'Stride',2,'Padding',[0 0 0 1])
    convolution2dLayer([9 1],10,'Stride',1)
    maxPooling2dLayer(2,'Stride',2,'Padding',[0 0 0 1])
    fullyConnectedLayer(30)
    fullyConnectedLayer(10)
    fullyConnectedLayer(2)
    softmaxLayer
    classificationLayer];

% Specify training options.  
opts = trainingOptions('sgdm', ...
    'MaxEpochs',40, ...
    'Shuffle','every-epoch', ...
    'Plots','training-progress', ...
    'Verbose',false, ...
    'ValidationData',{VTrain,Valoutfinal},...
    'ValidationPatience',Inf);

%% Train network
% Same comment as in the previous section for training options.
net = trainNetwork(XTrain,Trainoutfinal,layers,opts);

However, I am getting Invalid training data. Predictors and responses must have the same number of observations.

Thanks in advance.

Answer 1

[Shivam Singh]

Hello tyler,

You should keep the "XTrain", "Ttrain" and "Vtrain" as double array only, and don’t convert it into cell array using “num2cell” function. So, the "XTrain" will be a double array of size= [400, 1, 2, 4500], "Ttrain" will be a double array of size= [400, 1, 2,500] and "Vtrain" will be a double array of size= [400, 1, 2, 450]

Also, you should keep the last layer of your network architecture as :

fullyConnectedLayer (number of categories in “Trainoutfinal”)

[tyler seudath]

Thanks a mil Shivam!! The network did trained!