Generalized Least Squares Error & Weights

Hi Guys,

Hope everyone is doing good. My question is related to using GLS (by function lscov). I have set of independent variables (302 x 6) and dependent variable.

I must mention that this is the first time I am trying GLS (or WLS). I would like to use weights such that I would emphasize those observations that closer to an "average" and penalizing (by less weight) for those which seem to be far or outlier. (I would like to know your input in this approach too).

In order to do that, I devised a weight scheme such the weights add up to 1. And then I ran simpy lscov(A, b, w). Is this right approach? I get warning:

Warning: A is rank deficient to within machine precision. > In lscov at 197

Please let me know if there is something wrong am I doing?

Looking forward to get some serious inputs. Thank you very much in advance.

Kaushal

Answer 1

[Matt J]

Yes, if you apply small enough weights to informative data, you can make your system look rank deficient, leading to bad results. Example:

    >>A=rand(3); b=sum(A,2); w=[1 0 0];

    >> rank(A)

    ans =

         3

    >> lscov(A,b)

    ans =

        1.0000
        1.0000
        1.0000

but

    >> lscov(A,b,w)
    Warning: A is rank deficient to within machine precision. 
    > In lscov at 197 

    ans =

             0
        2.0486
             0

[Matt J]

Also, if A is rank deficient even when unweighted, similar problems arise. Example:

    >> A=rand(3); b=sum(A,2); w=[1 0 0]; A(:,end)=0;
    >> rank(A)

    ans =

         2

    >> lscov(A,b)
    Warning: A is rank deficient to within machine precision. 
    > In lscov at 197 

    ans =

        0.9731
        3.2323
             0

[KKR]

Great thank you very much for your response. That means, I should come up with a better weighting scheme, correct? I know the classic literature suggest to use the inverse of variances as weights, however the variances are usually not known for the observations. Any idea to get proper weights?

[Matt J]

If you don't suspect your A matrix of being rank deficient, then yes, the weights would be the culprit.

As for how to get the weights, it depends on whether you have multiple observations. If you do, maybe you could use the inverse of the sample variances.