Creating a neural network with hidden layer that has more than one activation function.

Hello.

I'm trying to create a simple neural network to fit the data that I have. it's a simple forward network with one hidden layer and one output layer. I would like to make the hidden layer in a way that it has a linear part and a non-linear part. The activation function of linear part is 'purelin' and activation function of non-linear part is 'tansig'. For example half of neurans use 'purelin' and the other half use 'tansig'.

Is it even possible to do this without programming the network?

This is an example from "Due Truong Pham and Liu Xing, Neural Networks for Identification,Prediction and Control" and I'm trying to recreate the result using MATLAB.

Answer 1

[Udit06]

Hi,

As per my knowledge, this is not possible without programming the network.

Deep Learning Toolbox of MATLAB allows to customize the network by defining the custom layers for any deep learning model. You can refer to the following MathWorks documentation to understand more about the same:

https://www.mathworks.com/help/releases/R2022b/deeplearning/ug/define-custom-deep-learning-intermediate-layers.html#:~:text=parameter%20W.-,Intermediate%20Layer%20Template,-To%20define%20a

In the template present in the above link, you can modify the "predict" and "forward" function to create a hidden layer with more than one activation function.