Hypothesis Tests for Constrained Linear Regression

Hi, I would like to do the hypothesis test for the following linear regression with equality constrants to exam the significance of each parameters

min_s || y - Xs ||^2

s.t. A s = b

is there any MATLAB build in function that can do the test?

[Torsten]

What are the sizes of X and A ?

[Star Strider]

I doubt this is possible. Constraining the parmeters likely invalidates any calculations of their uncertainties.

[Torsten]

Maybe if A has less rows than columns, one could just eliminate (columns-rows) parameters in s and fit the remaining ones in the usual way (with the possibility to test for their significance) ?

I'm not that much experienced in statistics ...

[Jeff Miller]

Assuming that the error scores associated with the y values are mutually independent, then you should be able to do this if you can formulate the hypothesis of interest as a comparison between two linear models--specifically, a smaller one nested in a larger one. This would be done using the outputs of the glm's for the two models.

Examining "the significance of each predictor" usually involves testing a smaller model without the predictor against a larger model with the predictor, so that part sounds possible. It will probably come down to what constraints you want to enforce on the parameter values and whether you can formulate nested linear models obeying those constraints.

Answer 1

[the cyclist]

One can often estimate the uncertainty in statistical parameters by bootstrapping.

Here is an example of doing this with an ordinary least squares regression. I compare

• standard error of the coefficient (of x1 term)
• bootstrap estimate of the same standard error

% Set random seed, for reproducibility
rng default

% Number of observations
N = 100;

% Explanatory variables
x1 = rand(N,1);
x2 = rand(N,1);
X = [x1, x2];

% Response variable, generated from explanatory plus some noise
y = 2 + 3*x1 + 5*x2 + 0.07*randn(N,1);

% Fit the model
mdl = fitlm(X,y);

% To estimate the parameter uncertainty, run the same fit with resampled data

% Number of resamples
NRESAMPLE = 5000;

% Preallocate the beta for x1, for all the resamples
beta_x1_R = zeros(NRESAMPLE,1);

% Loop over the models
for nr = 1:NRESAMPLE
    
    % "Bootstrap" sample of the data 
    idx = randsample(N,N,true);
    XR = X(idx,:);
    yR = y(idx,:);
    
    % Fit the model on the resampled data
    mdlR = fitlm(XR,yR);
    
    % Store the beta of x1, for the resampled fit
    beta_x1_R(nr) = mdlR.Coefficients.Estimate(2); 

end

% Standard error of the original fit
x1_standard_error_original_fit = mdl.Coefficients.SE(2)

x1_standard_error_original_fit = 0.0232

% The bootstrap estimate of the standard error is the standard
% deviation of coefficients from the resampled fits.
% Notice that it is comparable to the above estimate.
x1_standard_error_resample = std(beta_x1_R)

x1_standard_error_resample = 0.0223

I think you ought to be able to do the same with your constrained fit.