Basic Regression with missing data question

I have a matrix of predictors size (M x N), where N is the number of predictors (cols) and M is the number of observations (rows). I have a vector of responses 1x M. I wish to somehow calculate the betas. The data is timeseries data, so alignment of values is critical.

Some of the predictor data is NaN. Matlab deals with this by:

% Remove missing values, if any
wasnan = (isnan(y) | any(isnan(X),2));
havenans = any(wasnan);
if havenans
   y(wasnan) = [];
   X(wasnan,:) = [];
   n = length(y);
end

If there is NaNs on any one row it will remove the entire row. The problem is every row contains a Nan for one of the predictors. hence regress sets the entire dataset to empty. Somehow I would like my regression function to just ignore Nans and so for the point when there is a nan, the total number of predictors will just change. I see some people on here have tried to deal with this by a sort of rolling regression.

I cant set the NaN to zero or to the last valid value, as this would mean something for my results.I should point out that strictly my data isnt missing (ie lost) but actually doesnt exist for these points in time.

Does anyone know of a more intelligent way that Matlab can be used to deal with missing data? Thank you

Answer 1

[mathworks2011]

Thank you for the reply.

I try and do:

Data = X; Design = y; [Parameters, Covariance, Resid, Info] = ecmmvnrmle(Data, Design);

(ie all optionals set to default). This throws an error in ecmpart.m as it passes in a single mean value of zero and then tries to index that mean with the non-nan positions (more than one)and hence it fails.

obviously Im (not) doing the pre-processing correctly. can you advise? thank you

[Andrew Newell]

I have just realized that I read the documentation wrong. NaN's are only allowed in the responses, not the predictors. Sorry about that. You may have to interpolate your predictor matrix to get the missing values.