PCA by optimization

I'd like to find the principal components of a data matrix X by solving the optimization problem min||X-XBB'||, where the norm is the Frobenius norm, and B is an orthonormal matrix. I'm wondering if anyone could tell me how to do that. Ideally, I'd like to be able to do this using the optimization toolbox. I know how to find the principal components using other methods. My goal is to understand how to set up and solve an optimization problem which has a matrix as the answer. I'd very much appreciate any suggestions or comments.

Thanks! MJ

Answer 1

[Steve Grikschat]

Hi MJ,

Optimization toolbox functions will minimize over "X" where X can be a matrix or vector. Internally, matrices get flattened out into vectors (i.e. x = X(:)). When calling your function, the optimization solvers will call with a matrix of the correct size.

You can pass your constraints in a similar way, although beware about the sizes of the linear constraint matrices. Their sizes should match with the vector length (size(A,2) == numel(X)).

Try it out:

% Define B and initial guess at solution X0
[Xsol,fval] = fminunc(@(X) norm(X-X*B*B','fro'),X0);

[MJ]

Hi Steve,

Thank you for your response. I was somewhat sloppy in my formulation of the problem. The unknown is the matrix B, with the constraint B'B = I, and the function to be minimized is f(B) = ||X - X*B*B'||. So, it's really a constrained optimization, with a nonlinear constraint. I'm wondering how that would be set up in Matlab. Also, looking at the documentation for the minimization routines, I couldn't find any mention of matrix responses. Where would be the best place for me to read about that?

Many thanks again!