Multi input convolutional neural network

How to implement three stream convolutional neural network. I was tried to use the following file exchange which is the implementation of two stream CNN using digit database.

https://jp.mathworks.com/matlabcentral/fileexchange/74760-image-classification-using-cnn-with-multi-input

But my database in folder format. Above file exchange use the digitTrain4Darray as the input in which the images are stored in the table and corresponding label is stored in the table in array format. But i don't know how to map my database to the code provided in the file exchange.

my database contains 50 subfolders represents the 50 classes. Each class contains the 6 image. I have to use 4 images for training and 2 images for testing.

Similarly i have to 2 more database in same format.

I need to train my CNN with these three database separately. Finally i have to concatenate the results as in the image.

Kindly suggest the ways to do this task.

Thanks and regards,

Ramasenthil.

Answer 1

[Ameer Hamza]

It is possible to design such a network in MATLAB if you have the deep learning toolbox. The easiest way is to open the Deep Network Designer app and use the GUI to create the network you want.

deepNetworkDesigner

You can export the network directly to the workspace, or you can also generate code to create the network programmatically.

[Rd]

Dear ameer Hamza,

I was tired to use deepNetworkDesigner. But i can't give 3 separate input folders to. because it accpet only one folder as input.

I need to train cnn with three input database each one for each input image (imagefolder1 -> inputlayer1, imagefolder2 -> inputlayer2, imagefolder3 -> inputlayer3).

Kindly suggest some idea.

Thanks.

[Ameer Hamza]

Ok, I misunderstood the question. The question was actually about how to input the data to the network. Since I don't have the dataset, I cannot suggest an exact code, but the following hints will help in finding the solution. Also, I am not sure if it is possible to do this in the app, so you will need to do this programmatically.

You first need to create three imageDatastores for each folder

ds1 = imageDatastore('folder1', 'IncludeSubfolders', 1)
ds2 = imageDatastore('folder2', 'IncludeSubfolders', 1)
ds3 = imageDatastore('folder3', 'IncludeSubfolders', 1)

Then use transform() to combine these datastores for the trainNetwork() function. In this section: https://www.mathworks.com/help/deeplearning/ref/trainnetwork.html#mw_6a0ead40-d0f3-4af8-be23-37b407b8e923 of the documentation, it is described what type of values MATLAB expects when network has multiple inputs layes. For example

ds = transform(ds1, ds2, ds3, @(ds1, ds2, ds3) {ds1, ds2, ds3})

However, note that this command is still not correct. You will need to figure out how to add the labels as the 4th element of the cell array. These are very crude hints, but it should help you to get started.

[Rd]

ok.

Thanks for your reply.

[merve kesim]

Hello, could you please help me? What method did you develop for the problem? I have a similar problem as well.