Question about A\b

Hi everyone,

I really like A\b (just 3 letters and it can do magic). However, I run into this problem.

>> A = [1 1; 1 1]

A =
     1     1
     1     1

>> b = [1; 1]

b =
     1
     1

>> A\b

Warning: Matrix is singular to working precision.
(Type "warning off MATLAB:singularMatrix" to suppress this warning.)

ans =

     NaN
     NaN

I understand that A is not invertible. However there is a solution and I kind of expect MATLAB will return [1; 0].

Or maybe I have the wrong expectation.

[Star Strider]

See the discussion in: Need help with MATLAB question (link)

[John D'Errico]

It looks like everyone in that same class will be asking this virtually identical question.

[Stephen Wang]

Wow. What a coincidence. I am not in that class. Actually my question is more on MATLAB A\b than on linear algebra itself.

Answer 1

[Matt J]

It's really all a question of which linear solver you use. Each has its own idea of which of the infinite solutions to choose from. One alternative is,

    >> pinv(A)*b

    ans =

        0.5000
        0.5000

Another is,

    >> lscov(A,b)
    Warning: A is rank deficient to within machine precision. 
    > In lscov (line 200) 

    ans =

        1.0000
             0

[Matt J]

Also, if you get rid of the redundant equations, A\b will still work

   >> [1,1]\1

ans =

       1
       0

as will linsolve

   >> linsolve([1,1],1)

ans =

       1
       0

[Stephen Wang]

Hi Matt,

Thanks for your suggestions. Thanks for introducing the matlab function lscov().

My apology. In my original post, I should have stated my question more clearly. So here it goes again.

I would like to solve the good old (you can even call it boring) least squares equation in MATLAB,

     A*x = b

    where A can be any matrix, i.e. square, skinny (eg. 5-by-3) or 
    fat (eg. 3-by-5 matrix). Both x and b are just column vectors 
    having the same number of rows as matrix A.

Since this is a least squares problem, this means that there is always a solution. If there is a unique solution for x, then it is great and we are done. However, if there are infinite number of solutions for x, then I want x to have maximum number of elements to be zero.

Based on what I want, I always think that (until now) the answer is the magic 3 letters in MATLAB:

A\b

And I thought that A\b always works regardless of A. So it caught me by surprise that in my toy example, A\b breaks down.

>> A = [1 1; 1 1], b = [1; 1], x = A\b
A =
     1     1
     1     1
b =
     1
     1
Warning: Matrix is singular to working precision.
(Type "warning off MATLAB:singularMatrix" to suppress this warning.)

x =
   NaN
   NaN

So I guess I should not use A\b anymore (this is what I have been memorizing so far).

So let's try lscov. It does not quite always work either. Acutally it is worst. LSCOV errors out in the following example

>> A = [1 1], b = [1], x = lscov(A, b)
A =
     1     1
b =
     1
Warning: A is rank deficient to within machine precision.
(Type "warning off MATLAB:lscov:RankDefDesignMat" to suppress this warning.)

> In lscov (line 200) 
The logical indices in position 1 contain a true value outside of the array bounds.
Error in lscov (line 201)
        R = R(keepCols,keepCols);

Any other suggestion?

[Matt J]

You may have found a corner case, but LSCOV should always return the solution with a maximum number of zeros. From the documentation:

x = lscov(A,B) returns the ordinary least squares solution to the linear system of equations A*x = B, i.e., x is the n-by-1 vector that minimizes the sum of squared errors (B - A*x)'*(B - A*x), where A is m-by-n, and B is m-by-1. B can also be an m-by-k matrix, and lscov returns one solution for each column of B. When rank(A) < n, lscov sets the maximum possible number of elements of x to zero to obtain a "basic solution".