LinearModel.stepwise uses interactions

Hello, LinearModel.stepwise produces a model with some interactions, even when set to 'linear'. It does not result in the same model as when it IS set to 'interactions'. Can someone please explain this, and perhaps the difference between these? Many thanks, JEL

Answer 1

[the cyclist]

I can't fully answer your question, because I don't fully understand it. But let me try to say some helpful things.

It is important to understand that the word "linear" in the phrase "linear model" means that the model is linear in the coefficients. It has nothing to do with the powers of the explanatory variables. So, for example,

y(x) = alpha + beta1*x + beta2*x^2

is a linear model (assuming y is the response variable, and x is the explanatory variable).

y(x) = alpha * sin(beta * x)

is not, because of the non-linear dependence on the coefficient beta.

A linear model can have interaction terms:

y(x1,x2) = A + B*x1 + C*x2 + D*x1*x2

is a linear model with interactions.

When describing models that have interactions, you will sometimes refer to the linear terms. In the above, B*x1 and C*x2 are the linear terms, and D*x1*x2 is the interaction term. (The terminology can definitely be confusing.)

Regarding different calls to LinearModel.stepwise giving different results ... I think it would be best if you gave a couple distilled examples. Otherwise, we're just guessing at exactly what you mean.

[John Layne]

THanks for the very propmt response! I was referring to the modelspec for the function LinearModel.stepwise. If the modelspec is set to 'linear', the starting model contains an intercept and linear terms for each predictor. If the modelspec is set to 'interactions', the starting model contains an intercept, linear terms, and all products of pairs of distinct predictors (no squared terms). However, when I set the modelspec to 'linear', the final output includes interaction terms, i.e., the product of some pairs of predictors. I guess it's a difference in the starting model. Is the difference due to the fact that if modelspec is 'linear', interactions are introduced until a threshold fit is achieved, but if set to 'interactions', interactions (and individual predictors) are removed until a threshold fit is achieved?

[the cyclist]

Ah, now I understand better.

I expect your guess is right. The stepwise process is bringing in an interaction term before a linear term, and then the linear term does not improve the model.

I believe there is output from that function that will tell you the order that the terms were added, and stats on the improvement of the model. (I know that was true for the older stepwise and stepwisefit functions. I have not yet used LinearModel.stepwise.)

If you want to post your code here, I could look. (Won't be as rapid a response as these were, though.)

[John Layne]

Many thanks, Cyclist! Can you tell me how I can gain access to the contents of a model object? As in, the output from LinearModel.stepwise. I can't even tell what it contains.

[the cyclist]

The documentation for the Linear Model class discusses Properties and Methods. I took only a quick glance, but it looks like you are most interested in

mdl.Steps.History

where mdl is the model object.

Here's an example from the documentation of stepwiselm:

load hald
mdl = stepwiselm(ingredients,heat,'PEnter',0.06)
mdl.Steps.History