"polyest" gives a different MSE of the found model than "compare" and "sim"

I've used polyest to estimate a model for time series data. When finished, it gives a report where it is stated that:

Fit to estimation data: 64.13% (prediction focus)

FPE: 0.1117, MSE: 0.111

However, when I then simulate this model (using sim) with the same time series data, the Mean Squared error is 0.8032. When using the function compare, the error is a little bit smaller but still much bigger than 0.111.

Does polyest use a different way of simulating data or calculating the error? As far as I can tell, it uses the full time-series data.

Answer 1

[Vatsal]

Hi,

The Mean Squared Error (MSE) reported by the "polyest" function is based on the fit of the model to the estimation data. This error is calculated based on the differences between the observed outputs and the outputs predicted by the model for the same inputs.

When you simulate the model using the "sim" function with the same time series data, the MSE might differ. This is because the simulation performs a forward-prediction, where each output is predicted using both past inputs and previously predicted outputs, rather than the actual observed outputs. As a result, any error in predicting an output at a certain time step can affect the prediction at subsequent time steps, potentially leading to an increased MSE.

Conversely, the "compare" function in MATLAB computes the one-step-ahead prediction error, where each output is predicted using both past inputs and the actual observed outputs from previous time steps. Since this approach does not allow errors to accumulate over time, it often results in a smaller MSE compared to the simulation.

It is also important to note that the MSE can be affected by the choice of initial conditions for the simulation or prediction process.