how to train deep learning network with multi-inputs

Hi I need to train a CNN with two different type of images, so I used the following code

inputSize = [64 64 64 4];

inputLayer = image3dInputLayer(inputSize,'Normalization','none','Name','input');

inputLayer2 = image3dInputLayer(inputSize,'Normalization','none','Name','input2');

when I used analyzeNetwork(lgraph), I got error that should only one input layer be used. So please is there any way I can use multi inputs in the matlab?

Also, if multi inputes is supported in matlab, how can define the two different inputs in the training process ( trainNetwork(dsTrain,lgraph,options);)

I appreciate your advice!

Thanks

[Kenta]

You can use multi-input CNN with "custom training loop" as it is started to be supported from 2019b. The below is an example. I hope it can help you.

https://jp.mathworks.com/matlabcentral/fileexchange/74760-image-classification-using-cnn-with-multi-input

[Hon Wah Yeung]

There is an easier way to by-pass it as the inputSize for the 2 inputs are the same.

Just simply stack the 2 inputs channel-wise and use grouped-convolution with number of groups set as 2. Or if you want the learning to be done differently for the 2 inputs, you can create your own layer to split the stacked input into 2 outputs.

Answer 1

[Sai Bhargav Avula]

Hi,

You can either use the CombinedDatastore or define a custom layer that can take multiple inputs. For defining a custom layer with multiple inputs you can follow the following link

https://www.mathworks.com/help/deeplearning/ug/define-custom-layer-with-multiple-inputs.html

Additionally the transformedDatastore will also serve the purpose.

Hope this helps !