Neural Network forecasting Issue

Hello,

I've created a neural network to fit a certain (simple) input-output relationship. When I look at the time-series responses plot using the nntrain gui the predictions seem quite adequate, however, when I try to do out of sample prediction the results are nowhere close to the function being modelled.

I've googled this problem extensively and messed around with my code to no avail, I'd really appreciate a little insight into what I've been doing wrong.

I've included a minimal working example below.

 A = 1:1000;  B = 10000*sin(A); C = A.^2  +B;
Set = [A' B' C'];
input = Set(:,1:end-1);
target = Set(:,end);
inputSeries = tonndata(input(1:700,:),false,false);
targetSeries = tonndata(target(1:700,:),false,false);
%
inputSeriesVal = tonndata(input(701:end,:),false,false);
targetSeriesVal = tonndata(target(701:end,:),false,false);
%
inputDelays = 1:2;
feedbackDelays = 1:2;
hiddenLayerSize = 5;
net = narxnet(inputDelays,feedbackDelays,hiddenLayerSize);
%
[inputs,inputStates,layerStates,targets] = preparets(net,inputSeries,{},targetSeries);
net.divideFcn = 'divideblock';  % Divide data in blocks
net.divideMode = 'time';  % Divide up every value

 % Train the Network
[net,tr] = train(net,inputs,targets,inputStates,layerStates);
Y = net(inputs,inputStates,layerStates); 

 % Prediction Attempt
delay=length(inputDelays); N=300;
inputSeriesPred  = [inputSeries(end-delay+1:end),inputSeriesVal];
targetSeriesPred = [targetSeries(end-delay+1:end), con2seq(nan(1,N))];
netc = closeloop(net);
[Xs,Xi,Ai,Ts] = preparets(netc,inputSeriesPred,{},targetSeriesPred);
yPred = netc(Xs,Xi,Ai);
perf = perform(net,yPred,targetSeriesVal);

 figure;
plot([cell2mat(targetSeries),nan(1,N);
      nan(1,length(targetSeries)),cell2mat(yPred);
      nan(1,length(targetSeries)),cell2mat(targetSeriesVal)]')
legend('Original Targets','Network Predictions','Expected Outputs')
  end

I'd appreciate any insight into what I'm doing wrong,

Regards, James

[Greg Heath]

Plot A, A.^2, B and C.

 1. Terms differ by many orders of magnitude

 2. The dominant term A.^2 is not stationary.

 3. There are no explicit lags in your equation

 4. Find more appropriate time series data in the documentation.

Greg

Answer 1

[Shashank Prasanna]

You are using the wrong type of network for your problem.

Essentially you are trying to fit a function which is completely uncorrelated with itself, and you have no reason to use an Autoregressive network at all. You can choose a classical feedforward network to fit a function and forecast, it will do a much better job.

If you have a function where

C(n) = f(C(n-1),C(n-2),...)

NARX will make more sense.

[James Donovan]

Hi Benji,

Thanks for your answer. I believe that the narx net uses lagged targets as well as the lagged inputs in this example, so c(t) = c(t-1) + c(t-2) + a(t-1) + a(t-2) + b(t-1) + b(t-2). I realise that this is not an appropriate specification for the problem but I intend on using this as a base model for a pure time series problem.

I'm still not clear on why the forecasting ability is so weak when there seems to be a decent fit when it's being trained and in validation and testing. Is it fact that the forcast is using a closed loop network that's causing poor out of sample performance?

[Shashank Prasanna]

James, there are several concerns. Firstly this is not an appropriate netowrk for you data as I mentioned, secondly with a choice of network you HAVE TO play around with the number of nodes and delays - this is more art than science.

Ex1: Make the following change for descent prediction for atleast 30steps

rng(0)
inputDelays = 1:10;
feedbackDelays = 1:10;
hiddenLayerSize = 30;

Note: I mention again, you have NO serial autocorrelation, there is NO reason what so ever for the network to keep predicting perfectly for ever. Try an open loop architecture such as FITNET. Its just a model, and it has limitations.

Ex2: I have concerns about your your data. It is not stationary and that sometimes poses problems. See how the network give you near perfect prediction here: Change

 C = B;

and

rng(0);
inputDelays = 1:10;
feedbackDelays = 1:10;
hiddenLayerSize = 20;

Again I can emphasize enough, the tools are only good if you use them appropriately. Only the right tool with the right configuration may give you desired results. Please used the rng(0) so that you can reproduce my results. It sets the random seeds.

[James Donovan]

Thanks Benji,

You've made things much clearer. I think I lost sight of the basics, this modelling technique is quite new to me!