Train one NARX network to predict an output, but using multiple sets of time series data gathered in separate runs (each one consists of adjusting a different variable and observing the system output)

I've got a system I am trying to model, and the data I have consists of separate runs where specific parameters are changed over the course of each run (e.g. variable 1 is slowly changed throughout run 1, variable 2 is slowly changed throughout run 2, etc).

I am trying to predict the output for one variable based on the changes seen in variable 1, 2, etc. over time, but because train automatically reinitializes the weights, it appears as though I cannot simply train the network on run1, then on run 2, etc. However, because there are significant time delays in the system, I also cannot really just append run 2 onto run 1 etc (I am currently using many lagged values as input to account for the transport delays in the system). Is there some way to train the network on all of the separate data runs I have?

I am also not sure whether I should really be using train here or adapt. I understand that train disregards order and thus seems unsuitable for time-series prediction, but then I am confused as to why it is the default for narxnet.

Anyway, any help/insight would be much appreciated.

[Greg Heath]

 [ I N ] = size(input) = ?
 [ O N ] = size(target) = ?
  FD = ? % Significant delays of the target autocorrelation function.
  ID = ? % Significant delays of the input/target cross-correlation functions?

 H = ? How many hidden layer nodes?

 How many series do you have to work with ?

In the latest versions of MATLAB

 1. Train only reinitializes weights when all weights are zero.
 2. Train does not disregard order for time-series functions. 
    However the default data division function is dividerand 
    which yields nonconstant differences between time points.           
    Therefore, use divideblock or one of the other dividefunctions.
 3. I do not recommend using ADAPT 
 4. Have you tried appending one series to another?

Hope this helps.

Greg

Answer 1

[Greg Heath]

Sorry, but you have to train nets with input/output pairs similar to the ones that they will encounter when deployed.

[Lee]

In this case, the data are all representative of probable input/output pairs and behavior which could be encountered during deployment. With the separate experimental runs, I was trying to fully cover the region of the parameter space most likely to be encountered (this could not be all done in one run due to time constraints during limited windows of operation).

So, then I guess the broader question I have (which is likely very naive on my part) is how to train on more than one single time series sequentially (e.g. what if I have five separate sets of experimental results from the same system operating under similar conditions, and together these cover normal operation in a way which fully captures the parameter space likely to be seen during deployment).

[Apoorv Reddy]

Hi, I am facing the some problem. Did you arrive at a solution to train an NARX on multiple time series.

[Fernando García-García]

Same problem here. I want to train a NARX net with a collection of time series all representing the phenomenon I want to predict (just different realizations). If anyone achieves it, please post. We would appreciate it so much!

[Greg Heath]

The logistics of MIMO Narx doesn't present an insurmountable problem ... assuming the inputs occur concurrently and similarly with the targets.

However, when you have I simultaneous input series and O simultaneous output target series, you have to determine a manageable subset of the I +I*O sets of significant correlation lags. To be clear, the same set of input lags will be applied to all of the inputs and similarly with the target/output feedback lags.

Is that the difficulty you are concerned with; or is there something else?

Greg

[christttttttophe]

Anybody have any progress on this issue regarding multiple datasets? This is definitely not answered.

[Torsten K]

Hello everybody,

I have a similar problem: NARX network, 3 inputs, 1 output, 120 time series of different lengths (200 to 2500 minutes) in minute resolution. The 120 time series were packed together with the help of "catsamples", so that the network is trained with all time series at once. For days now I've been pondering how to calculate the output autocorrelation and the input-output cross-correlation to determine a reasonable number of TSPs. In addition, I still don't know how to estimate a reasonable number of neurons in the hidden layer. Anyone have an idea? I would be grateful for every tip, no matter how small!

Best regards

Torsten

[Imola Fodor]

catsamples with the 'pad' for some reason doesnt work

Data argument dimensions are not consistent.

[Federico Alberto Gorrini]

@Torsten K can you tell me exactly in which format did you introduced each time serie? Because I have a similar problem, with 15 time series, and my problem has 3 inputs and 4 outputs...