"Invalid training data" for multiclass classification

I am trying to use classification, to classify three classes, using a simple fully connected NN.

My training data consists of 69979 examples with 12 input data 'X* (real numbers), and I wish to predict the classes '0','1', and '2', stored in 'Y'

>> whos X Y
  Name          Size              Bytes  Class          Attributes

  X         69979x12            6717984  double                   
  Y         69979x1               70299  categorical              

>> summary(Y)
     0      27469 
     1      28078 
     2      14432 

My network is like this,

     layers = 

  9×1 Layer array with layers:

     1   'fcIN'           Feature Input           12 features
     2   ''               ReLU                    ReLU
     3   'hl1'            Fully Connected         40 fully connected layer
     4   ''               ReLU                    ReLU
     5   'hl2'            Fully Connected         40 fully connected layer
     6   ''               ReLU                    ReLU
     7   'fcOUT'          Fully Connected         3 fully connected layer
     8   'SoftMax'        Softmax                 softmax
     9   'class_output'   Classification Output   crossentropyex

and training works fine using

[ml.net,ml.info] = trainNetwork(X,D,layers,options);

But what I really would like to do is multiclass classification. I want to estimayte the probability of classes '0' '1' and '2' for 125 parameters.

My 'raw' in and out data have sizes

>> whos X Y
  Name          Size               Bytes  Class          Attributes

  X         69979x12             6717984  double                   
  Y         69979x125            8747695  categorical     

So, for each 'x' in the input I have 12 data points, and I know the class of 125 (and not just 1) parameters.

I use the same network as above, but this time with 3*125 nodes in the last fully connected layer, as I have 3 classes for each of the 125 parameters.

But, when I try to train the network I get the error "Invalid training data":

>> [ml.net,ml.info] = trainNetwork(X,Y,ml.layers,ml.options)
Error using trainNetwork (line 184)
Invalid training data. For classification tasks, responses must be a vector of categorical responses. For regression
tasks, responses must be a vector, a matrix, or a 4-D array of numeric responses which must not contain NaNs.

IN order to get a "vector of categorical responses" I try to use

Y = num2cell(categorical(Y),2);
whos Y
  Name          Size               Bytes  Class    Attributes

  Y         69979x1             38418471  cell 

But I get exactly the same error message.

So how to I format the Y data so I can train this type of data to predict the class of the 125 parameters for a set of input data (of size 1x12)?

Is it possible?

Answer 1

[Srivardhan Gadila]

In order to implement the above problem the output of the softmax layer should be 125x3 or 125x1x3 etc., you can create a custom layer to reshape the output 375 (= 3*125) from fullyconnected layer and insert it before the softmax layer. Make sure that the dimension of 125 corresponds to the dimension label 'S' and that of 3 corresponds to 'C'. You can refer to Define Custom Deep Learning Layers and addLayers for implementing the said functionality. But again later you may encounter issues with figuring out what should be the data format to give to the trainNetwork function. Hence I would suggest to go with dlnetwork & Deep Learning Custom Training Loops based approach instead of layerGraph and trainNetwork based workflow.

Apart from the above suggestions I think in general this may not be a good idea of predicting the class labels of higher dimensional data (125) from lower dimensional data (12), it may or may not give the good results.