Artificial neural network - how to use in solving of constrained problem

Let's assume that I want to use ANN to predict mixture on the output of the apparatus. Let the composition be described in percentage by volume. Thus the total sum of A+B+C must be equal to 100%. In very simplified case I can run:

rand('seed',1) %For 100% repordability
input=[80 70 55 60; 10 20 25 20; 10 10 20 20]
target=[42 33 25 27; 30 40 42 23; 30 27 33 50]
net = feedforwardnet(10,'trainlm');
net=  train(net,input,target);

But in some cases I can achieve prediction with A+B+C sum > 100%, for example:

output=net([20; 60; 20])
total_p=sum(output) %this is equal to 123.4275

How to solve this issue? In conventional optimization algorithm I would set simple linear constraint but how to do this in ANN?

Answer 1

[Shreeya]

Hi @Karol P.

I understand that you want to train a custom ANN such that it meets the desired constraints.

In order to achieve desirable predictions from a neural network, a sufficiently large training dataset and target labels adhered to the specified constraints can help train the model such that the predictions naturally tend to be bound by the constraints.

However, in the given example, the training data is limited, which could be a contributing factor to the failure in meeting the constraints.

To address this issue and ensure that the network is trained while enforcing the specified conditions, the following strategies can be employed:

1. Incorporate an additional expression into the cost function. This expression should be a differentiable function that evaluates to zero when the constraints are satisfied, and a large positive value otherwise. By including this expression, the network is encouraged to prioritize meeting the constraints during the training process.
2. Select an appropriate activation function that enforces the allowed range of predictions. For instance, if the sigmoid function is utilized, the output can be scaled to fall within the desired range [a, b] by applying the formula: output * (b - a) + a.

For further help, kindly refer to the below answer:

https://www.mathworks.com/matlabcentral/answers/96329-how-can-i-train-a-neural-network-such-that-the-output-of-the-network-satisfies-a-certain-constraint

I hope this helps!

[Karol P.]

Thank you for your answer. I already solved my problem in the way you described in point 1. However the second option looks interesting. Is there any way to select directly [a,b] without creating custom activation function?

[Shreeya]

Hi, you can use the "functionlayer" function to apply the scaled activation function to the neural network layers. Refer to the documentation linked below:

https://www.mathworks.com/help/deeplearning/ref/nnet.cnn.layer.functionlayer.html#mw_1040fdee-a6db-45e9-bfb0-c34660e4f72d