Eigenvalues (eig) and singular values (svd)

Hello,

My understaing is that the relationship between eigenvalues () > 0, and singular values () is the following one:

Where;

are the singular values of a matrix X, size(X)=MxN

are the eigenvalues of the square matrix X'X (where ' is the complex conj transpose)

In my case, I have a matrix size(X)=1000x5

its singular values

sv=svd(X)

are

196942.326781670

30136.1778043317

23562.4701314061

5.85220605708913e-10

2.64092030198871e-12

wherease the square root of the eigenvalues of the matrix X'X

eigenvalues_sqrt=eig((X'*X)^.5)

are

196942.32678167

23562.470131406

30136.177804331

0.00120366636426888

0.00276213718421482

As you can see, the first 3 values are the same.

Here my question, why the last two are different since the eigenvalues are not < 0? is this because they are close to 0?

Any help would be highly appreciated!

Gabri

Answer 1

[Marco Riani]

Hello Gabri,

the answer in short is: "the difference is due to numerical errors".

For example in the normal case

% Normal case no singularities
rng('default')
X=randn(1000,5);
sv=svd(X);
svchk=sort(eig((X'*X)^.5),'descend');
disp([sv svchk])

the output of the two methods virtually coincides

32.805793753218964 32.805793753218971

32.596688238949213 32.596688238949206

31.830948559876663 31.830948559876632

30.587981509539315 30.587981509539343

29.375388175424334 29.375388175424348

In the case below with two eigenvalues equal to 0 (similar to your case)

because columns 4 and 5 are linear combinations of the first 3 columns, here is what happens

rng('default')
rng(10)
n=1000;
X13=1000*randn(n,3);
X4=X13(:,1)*100+X13(:,3)*200;
X5=X13(:,2)+11200*X13(:,3); 
X=[X13 X4 X5];
sv=svd(X);
svchk=sort(eig((X'*X)^.5),'descend');
disp([sv svchk])

1.0e+08 *

3.649002137098263 3.649002137098261

0.031607898246415 0.031607898246415

0.000320576287875 0.000320576287875

0.000000000000000 0.000000000023097

0.000000000000000 0.000000000002413

The singular values not equal to 0 are virtually the same using the two methods.

The singular values close to 0 computed with svd (as it happens in your case) can be slightly different from those computed with eig.

In any case the singluar values computed with svd seem to be more reliable.

One final remark: it is always better to start standardizing the data in order to avoid too small or too large numbers and potential numerical errors.

Hope it helps

Marco

[Gabriele Galli]

Carissimo Marco,

Thank you so much for your exhaustive reply!

Cordiali saluti, Gabri