correlation shows NaN values?

Hi all,

I would like to calculate the correlation between two vectors (1*744) for each. but the correlation output gives a matrix (744*744) contains NaNs with no number in the matrix.

Any help will be appreciated.

thanks

Answer 1

[Birdman]

Try corrcoef

corrcoef(A,B)

[Lilya]

the obtained result is:

NaN NaN NaN 1

[Birdman]

Can you share your vectors in a mat file?

[Lilya]

Find the attachment, please. thank you

[Birdman]

If we want to omit NaN values from your data, the two array becomes differently sized. Therefore, a solution could be replacing NaN values with the mean of the array. Only JulTotcurr include NaN values.

JulTotcurr(isnan(JulTotcurr))=mean(JulTotcurr,'omitnan');
corrcoef(JulTotcurr,JulTotwnd)

[Lilya]

Great! it works But can we obtain it as a vector? to plot the data?

[Birdman]

What do you want to plot?

[Lilya]

the values of the correlation

[Birdman]

res=corrcoef(JulTotcurr,JulTotwnd);
plot(res(:,1),res(:,2),'o')

[Lilya]

Totally appreciate it thank you

[Jos (10584)]

Statistically, it is not recommended to replace the missing values by the mean ("mean imputing"). There are other techniques that are (often) more reliable.

Statistical analyses with missing data is quite common, and there are many pitfalls to be aware of, in order to avoid drawing false conclusions!

[Lilya]

Thank you for your clarification Jos. Do you recommend one of them?

[Jos (10584)]

My pleasure. You can use random sampling, for instance. Replace the NaNs by values sampled from the rest of the data vector. The advantage is that it, in the long run, preserves both the mean and the variance of the original data (whereas mean imputing only preserves the mean). You also might consider bootstrapping this procedure to get a better estimate of the correlation with another vector. If X is your original vector:

Xcopy = X ;
tf = isnan(X) ;
Xcopy(tf) = randsample(X(~tf), sum(tf), true)

You also should analyse whether your missing data is (completely) random or not, as this would influence the inferences you can make.