Split neural network input into different branches of a network

I am attempting to build and train a neural network that takes as its input two surcfaces, and predicts another surface as the response. I would ideally like to have the two input surfafces go through convolutional layers separately and then be recombined later to go through more convolutional layers and finally a regression output. I am not very well versed in image processing, however I feel that there is somehting to be gained by learning the two input surfaces individually before learning the information together. I am using MATLAB R2019a, deep learning toolbox.

Currently I am useing the image3dinput to input the two surfaces as a [170, 134, 2, 1, M] (M is the number of samples) array. I have trained models which use 3d convolutions to try and skip the separate convolution step and they train properly. I want to either replace this with two image2dinput layers of size [170, 134, 1, M], or I would like to have a layer that splits the input along the third dimension to separate branches of the network.

I have seen that the depthConcatenationLayer can be used to concatenate inputs along the third dimension. I want to know if there is an equivalent inverse of this layer that un-concatenates along the third layer, as well as how to set up the network to have different branches. I have started looking into makeing a custom layer to do this, but I wanted to know if there was an easier way to approach this, or if anyone had suggestions...

Answer 1

[Srivardhan Gadila]

As you want to perform separate convolutions for the 2 surfaces of your input, you might use the groupedConvolution2dLayer . You can refer to it at https://www.mathworks.com/help/deeplearning/ref/nnet.cnn.layer.groupedconvolution2dlayer.html