Noisy results from Neural Network

I've trained a feed-forward neural network with 4 inputs and 2 targets with 10000 sample data. When I use this network for testing on a set of data, the average of data seems acceptable, but it is very noisy (see attached figure). Any suggestion to resolve this issue? I have tried many different layer and neuron numbers, as well as training methods, but no major improvement.

Answer 1

[Greg Heath]

Try preprocessing with a lowpass filter.

The cheapest one I can think of is

 x(i) = mean([x0(i-1),x0(i),x0(i+1)])  % 3-point moving average

I don't necessarily recommend it, it is just an example. Better to find a good LPF reference.

Hope this helps.

Thank you for formally accepting my answer

Greg

[Sam136]

Thanks for your answer Greg. Is there any way to get noise-free results from the NN itself without using lowpass filter. As I am new in NN area, I have no idea if this noisy result is a normal thing or not. I haven't seen this issue in many examples that I've reviewed.

[Image Analyst]

Or you can do Greg's method without a loop doing

movingAverageX = conv(x, [1,1,1]/3, 'same');

For a little less smoothing you can use sgolayfilt() (in the Signal Processing Toolbox) which uses polynomials of higher order than 1 (which is what a moving average is).

[Greg Heath]

In order to understand noisy results, you have to first analyze the noise of the input and target used in training.

If the training data is noiseless, then I would not expect the output from noisy nontraining input data to be noise free.

If you want the net to be robust w.r.t. noise you can either filter the data before inputting or train the net with noisy inputs and smooth outputs.

I am assuming you have a static net with no feedback or delays.

Greg

[Sam136]

Actually,both set of data (training and testing) are noise-free, but the response is noisy. I use static net. Do you think that it is the problem?

[Greg Heath]

If the data is static why not just sort the data with respect to the input value?

[Sam136]

Thanks Greg for your comment. Can you explain more about this.

[Greg Heath]

Sorry, my response assumed the input data was one-dimensional.

I think you can better understand your results if you do the following for each input

[sortx1 ind1] = sort(input(1,:));

figure, hold on plot(sortx1,target(ind1),'k--') plot(sortx1,output(ind1),'b')

Hope this helps,

Greg

[Greg Heath]

It doesn't appear to me that the output is significantly more noisy than the target.

Maybe you can see it better with other plotting options or plotting error (target-output).

Answer 2

[Sam136]

I changed my net to dynamic(narxnet) with zero delay for input and 1:2 delay for feedback, but I still have the problem. Below figures show NN response to the input data (same data that I used for training the network).

[Greg Heath]

If you are going to switch to narxnet. See my posts on how to choose delays

 greg nncorr

how to choose number of hidden nodes

 greg Hub

and how to normalize results

 greg NMSE
 greg R2
 greg R2a

[Sam136]

Thanks Greg. The network worked, and the results are pretty good. I have a question though. I have a difficulty to use the trained network (narxnet) for new set of input data. I get dimension error and it seems that I should include the target values. However, there is no target values when we want to use the network. Can you help me in this regard?