How can I include prior information like a known relationship between my output variables of a regression problem into neural network design (fitnet)?

Hi

How can I include prior information like a known relationship between my output variables into neural networks design?

In my case, I have many input variables that are physically independant from one another, and 4 output variables that have to sum up to 1 (respectively 100 %). The neural network per se tries to to satisfy this criterium, since all the training targets fulfill it, but it never quite gets 100 %, but rather somewhere between 98 and 102 %, which is physically not possible.

I was wondering if anybody knows a way to constrain the outputs by a function that needs to be fulfilled, or similar.

Thanks in advance, FS

Answer 1

[Greg Heath]

It depends on the relationship.

For [ 0, 1 ] targets that sum to 1, just use the crossentropy function in the output layer and choose the output with the maximum value.

This is standard approach for classifiers where the outputs are interpreted as posterior probabilities conditional on the input.

Hope this helps.

Thank you for formally accepting my answer

Greg

[F S]

Hi Greg

Thank you for your answer! I do not quite understand why the crossentropy function would be preferable over the mse function in such a case. It is not that I have to deal with large inaccuracies, but rather with small ones, since the neural network quite accurately predicts every single one of my 4 outputs, but neglects the physical relationship between them (in my case that all 4 sum up to 1 or 100 % respectiveley) which is known to me, but not to the network. It does that in a fairly small amount (e.g. 98 % or 102 %), but still, the result is not physically possible anymore, so I was hoping for a way to implement that relationship of the outputs in the model.

Also, I wasn't very specific about which kind of problem I am trying to solve, I am sorry! I am dealing with a regression problem, not with a classification problem. I will comply with that and edit my question.

Thank you anyway for your answer!

[Greg Heath]

If targets are 0 or 1 and the output sum is 1, crossentropy will yield what you want.

Although you are not designing a classifier, the conditions you have stated are, exactly, what defines a classier.

Hope this helps.

Greg