Problem with NARX open-loop training - Correlated with a shift - How to improve NARX training / testing

Hello everybody

I am new to statistics and ANN. I am using NARX (open-loop) to train a set of data (a time series as input X and a time series as target T, with more than 4000 data points). In the data, most of the time, X and T are constant, but sometimes X increases suddenly and after a short period of time T also increases, then X decreases to the constant value and T also decreases to its constant value. Please see the attached figure which shows a small part of the data.(in the figure, the green lines show the start of increase in X). After training the data I want to use the model to predict T at one-step ahead. But I have problem in training the data.

I use the following commands to create the net, train it, and then test it.

net = narxnet(inputDelays,feedbackDelays,hiddenLayerSize);
[inputs,inputStates,layerStates,targets] = ...
    preparets(net,inputSeries(train_period),{},targetSeries(train_period));
[net,tr] = train(net,inputs,targets,inputStates,layerStates);
outputs = net(inputs,inputStates,layerStates);

The output Y correlates to the target T, but, as can be seen in the figure, with a shift of about one step size!. In the lower plot, blue line is the target and red line shows the output. This is only a short part of the data, and this problem is seen in the whole data (even more serious when the bump in T is lower). My question is that why is there a shift of one step size between target T and output Y? How can I improve the model to have a good correlation between those two? I use 10 hidden layers, 3 input and feedback delays, and "divideint" for dividing the data. I also used other values for these parameters but no success.

I tried to use this trained model to predict one-step ahead of T, but as the output is shifted one step ahead in training, it also has a similar shift in the testing. Therefore, the result is not useful because when we are at time = n, the model gives the value of T at the same time as the output at time = n+1.

Any advice would be much appreciated. After solving this problem, I am going to use closed-loop to make multi-step ahead predictions.

Thank you!

Answer 1

[Greg Heath]

 1. Openloop(OL) NAR and NARX configurations are NOT DEPLOYABLE! 
 2. They cannot estimate outputs when only inputs are known. 
 3. They are only stages in the design of deployable closeloop (CL) 
    configurations where past output signal estimates are fed back 
    to the input.
 4. Therefore, if you need a NAR or NARX timeseries net to estimate 
    unknown outputs from known inputs, you have to use a deployable 
    CL configuration.
 5. BOTTOM LINE: OL configurations are only useful for designing 
    deployable CL configurations!

Hope this helps.

Thank you for formally accepting my answer

Greg

Answer 2

[Greg Heath]

1. There is not much information in your plotted data.

     a.  There seems to be 5 points within intervals of length 
          0.005 that define your input peaks. 
     b.  Only 3 of the points defining input peaks are are elevated.
     c. The 1st target and output peaks lag the input peak by one point.
     d. There doesn't seem to be any lag between the second input and 
         target peaks; whereas the output peak occurs one timestep later.

2. You probably need many more features for better predictions.

3. One possibility is to try to add more points to your input and target data via interpolation.

Hope this helps.

Thank you for formally accepting my answer

Greg

Answer 3

[EK_47]

Greg

Thank you for taking the time to answer this question.

1)

a. Yes, it is a bit less than 5 points. 
b. Yes
c. Yes
d. Yes

Actually, I have more data points to be used. The original data is collected every 5 minutes, but here I averaged the data every 2 hours, just to have a higher forecast horizon. I trained the model with the original data as well, but I have the same problem, i.e. there is a lag/shift between target and output profiles which is about one step size. Obviously, when I use the data with higher resolution, the step size is smaller so the lag/shift would be smaller. But the forecast horizon would also be smaller. This figure shows only a small period of the data; the lag/shift between output and target is seen all over the data. My question is how to remove this lag/shift in training.

2) What sort of features do you mean?

3) As mentioned above, the original data has more points, but same issue.

Thanks again for your answer and any other comments would be really appreciated.

Thanks

Answer 4

[EK_47]

Greg

Thanks a lot for your answer. It was very useful. I trained the model with open-loop configuration, then I closed it with closed-loop configuration and then removed a delay by removedelay. The code is summarised as below:

net = narxnet(inputDelays,feedbackDelays,hiddenLayerSize,'open',trainFcn);
[x,xi,ai,t] = preparets(net,X,{},T);
[net,tr] = train(net,x,t,xi,ai);
y = net(x,xi,ai);

netc = closeloop(net);
[xc,xic,aic,tc] = preparets(netc,X,{},T);
yc = netc(xc,xic,aic);

nets = removedelay(net);
[xs,xis,ais,ts] = preparets(nets,X,{},T);
ys = nets(xs,xis,ais);

Please see the figures. The first one shows all data (inputs and targets); the second one shows the data used for training; the third one shows the profiles within 2.32*10^5 to 2.35*10^5 minutes that is around the end of the training data where one-step prediction has been implemented. In this figure, green dot is located at the end of the training period (time = t), black dot shows the target at time = t+1, and the red dot shows the predicted output at time = t+1. The result seems good.

Red line is the output from "nets" network which is after removedelay, and green line shows the output from "netc" network which is the closed-loop network.

The last figure shows a part of the data within training period (between 0.5*10^5 and 0.6*10^5 min). There seems to be a lag/shift between target (black line) and "nets" output (red line). Besides the correlation of the "netc" output (green line) to the target (black line) looks very poor.

I have two questions:

1- Is is possible to improve the performance of the model with regard to the correlation between output (red line) and target (black line), i.e. to remove the lag/shift?

2- Is it a serious problem that there is no correlation between the output of the CL network (green line) and the target?

Next, I am going to make multi-step prediction, as I need to extend the forecast horizon. Any advice would be appreciated in this regard.

Thank you in advance for your time. -----------------------------------------------

Fig. (1):

Fig. (2):

Fig. (3):

Fig. (4):