# [R] Regression query

Peter Flom flom at ndri.org
Sun Jun 13 17:03:12 CEST 2004

```If variables are colinear, then looking at interactions among them
doesn't make much sense.  High collinearity means that one variable is
nearly a linear combination of others.  IOW, that variable is not adding
much information.  So, if you look at the interaction, you are ALMOST
looking at a quadratic (e.g., if the collinearity involves only 2
variables, then one is very similar to the other, so X1*X2 is almost
X1*X1).  The output will be confusing, to say the least.

Worse, when you include collinear variables, the resulting equation is
highly sensitive to small (sometimes very small) changes in the data.
Belsley gives an example where changes in the third decimal place result
in totally different equations.

For details see Belsley's book titled something like "collinearity and
weak data in regression" (sorry, the book and my files are at the
office, but this should let you find it

HTH

Peter L. Flom, PhD
Assistant Director, Statistics and Data Analysis Core
Center for Drug Use and HIV Research
National Development and Research Institutes
71 W. 23rd St
www.peterflom.com
New York, NY 10010
(212) 845-4485 (voice)
(917) 438-0894 (fax)

>>> "Devshruti Pahuja" <devshruti at hotmail.com> 06/11/04 5:35 AM >>>
Hi

I have a set of data with both quantitative and categorical predictors.
After scaling of response variable, i looked for multicollinearity (VIF
values) among the predictors and removed the predictors who were hinding
some of the
other significant predictors. I'm curious to know whether the predictors
(who are not significant) while doing simple 'lm' will be involved in
interactions. How do i take into
account  interactions of those predictors whom i removed just on the
basis
of  multicollinearity ?

I'll appreciate if someone can throw some light on this matter and how
to
use R to detect the interactions effectively .

Thanks

Regards
Dev

> ------Final 'lm model'--------------------
> > logmodelfull_minus_run_hr_walk_batting <- lm(log(salary) ~ hit+rbi +
walk
> + obp +
strike.out+free.agent.eligible+free.agent.1991+arbitr.elgible.)
> > summary(logmodelfull_minus_run_hr_walk_batting)
>
> Call:
> lm(formula = log(salary) ~ hit + rbi + walk + obp + strike.out +
>     free.agent.eligible + free.agent.1991 + arbitr.elgible.)
>
> Residuals:
>      Min       1Q   Median       3Q      Max
> -2.41786 -0.28911 -0.02814  0.31890  1.49007
>
> Coefficients:
>                       Estimate Std. Error t value Pr(>|t|)
> (Intercept)           5.340782   0.251218  21.260  < 2e-16 ***
> hit                   0.004479   0.001158   3.867 0.000133 ***
> rbi                   0.011102   0.002195   5.059 7.05e-07 ***
> walk                  0.005421   0.002206   2.457 0.014533 *
> obp                  -1.385584   0.824105  -1.681 0.093653 .
> strike.out           -0.005399   0.001438  -3.755 0.000205 ***
> free.agent.eligible1  1.611521   0.080657  19.980  < 2e-16 ***
> free.agent.19911     -0.301243   0.103481  -2.911 0.003848 **
> arbitr.elgible.1      1.293059   0.086696  14.915  < 2e-16 ***
> ---
> Signif. codes:  0 `***' 0.001 `**' 0.01 `*' 0.05 `.' 0.1 ` ' 1
>
> Residual standard error: 0.5351 on 328 degrees of freedom
> Multiple R-Squared: 0.7981,     Adjusted R-squared: 0.7932
> F-statistic: 162.1 on 8 and 328 DF,  p-value: < 2.2e-16
>
>
--------------------------------------------------------------------------
--
> ----------------------------------------------------
>
>
> --------------with
>
interactions----------------------------------------------------------------
> ---------------------------
>
> >
> > summary(baseball.lgmodel_with_interactions_ALL_arbid)
>
> Call:
> lm(formula = log(salary) ~ hit + rbi + strike.out +
free.agent.eligible +
>     free.agent.1991 + arbitr.elgible. + hit * free.agent.1991 +
>     hit * arbitr.elgible. + hit * rbi + rbi * free.agent.eligible +
>     rbi * arbitr.elgible. + rbi * arbitr.1991 + hit * strike.out +
>     strike.out * free.agent.eligible + strike.out * arbitr.elgible. +
>     strike.out * run + strike.out * hr + hit * free.agent.eligible +
>     free.agent.eligible * run + hit * free.agent.1991 + strike.out *
>     free.agent.1991 + free.agent.1991 * batting + free.agent.1991 *
>     obp + arbitr.elgible. * run + batting * double + obp * run +
>     obp * hr + walk * stolen.base + hit * arbitr.1991 +
free.agent.eligible
> *
>     double + arbitr.elgible. * double + strike.out * triple +
>     triple * batting + triple * walk + triple * walk + hit *
>     hr + rbi * hr + free.agent.eligible * hr + free.agent.1991 *
>     hr + arbitr.elgible. * hr + hr * arbitr.1991 + hit * walk +
>     free.agent.eligible * walk + walk * rbi + rbi * stolen.base +
>     strike.out * stolen.base + stolen.base * batting + stolen.base *
>     walk + stolen.base * rbi + stolen.base * walk + arbitr.elgible. *
>     error)
>
> Residuals:
>      Min       1Q   Median       3Q      Max
> -2.29352 -0.28287 -0.03748  0.29790  1.31590
>
> Coefficients:
>                                   Estimate Std. Error t value Pr(>|t|)
> (Intercept)                      5.217e+00  3.467e-01  15.048  < 2e-16
***
> hit                              6.927e-03  6.226e-03   1.112 0.266889
> rbi                              1.908e-02  1.150e-02   1.658 0.098350
.
> strike.out                      -5.692e-03  4.586e-03  -1.241 0.215517
> free.agent.eligible1             1.287e+00  2.259e-01   5.699 3.05e-08
***
> free.agent.19911                 3.828e-01  6.575e-01   0.582 0.560914
> arbitr.elgible.1                 1.038e+00  2.195e-01   4.726 3.63e-06
***
> arbitr.19911                    -1.024e+00  4.392e-01  -2.331 0.020443
*
> run                              4.932e-02  2.905e-02   1.698 0.090682
.
> hr                              -1.093e-01  7.208e-02  -1.516 0.130543
> batting                         -1.814e-01  2.558e+00  -0.071 0.943522
> obp                             -1.375e+00  2.253e+00  -0.610 0.542099
> double                          -5.259e-02  4.489e-02  -1.172 0.242349
> walk                             1.395e-02  9.757e-03   1.430 0.153808
> stolen.base                     -1.685e-02  4.299e-02  -0.392 0.695372
> triple                          -1.367e-01  1.600e-01  -0.854 0.393807
> error                           -4.097e-03  6.879e-03  -0.595 0.552007
> hit:free.agent.19911             8.248e-04  4.611e-03   0.179 0.858174
> hit:arbitr.elgible.1             4.873e-03  6.448e-03   0.756 0.450395
> hit:rbi                         -1.382e-04  7.709e-05  -1.792 0.074184
.
> rbi:free.agent.eligible1         5.352e-03  9.555e-03   0.560 0.575855
> rbi:arbitr.elgible.1            -3.384e-03  1.136e-02  -0.298 0.766072
> rbi:arbitr.19911                 3.596e-02  2.179e-02   1.650 0.100046
> hit:strike.out                   5.480e-06  5.446e-05   0.101 0.919917
> strike.out:free.agent.eligible1 -2.570e-03  4.282e-03  -0.600 0.548890
> strike.out:arbitr.elgible.1     -9.703e-04  5.234e-03  -0.185 0.853068
> strike.out:run                   1.685e-04  1.246e-04   1.352 0.177345
> strike.out:hr                   -3.088e-04  2.277e-04  -1.356 0.176229
> hit:free.agent.eligible1        -1.359e-03  6.224e-03  -0.218 0.827363
> free.agent.eligible1:run         1.248e-02  9.109e-03   1.370 0.171917
> strike.out:free.agent.19911     -1.851e-02  5.974e-03  -3.099 0.002140
**
> free.agent.19911:batting         7.076e-01  6.200e+00   0.114 0.909215
> free.agent.19911:obp            -1.421e+00  3.952e+00  -0.360 0.719394
> arbitr.elgible.1:run            -8.541e-03  8.773e-03  -0.974 0.331100
> batting:double                   2.346e-01  1.609e-01   1.458 0.145884
> run:obp                         -1.825e-01  7.492e-02  -2.436 0.015462
*
> hr:obp                           3.687e-01  2.116e-01   1.742 0.082608
.
> walk:stolen.base                -6.789e-05  1.557e-04  -0.436 0.663083
> hit:arbitr.19911                -5.835e-03  7.084e-03  -0.824 0.410808
> free.agent.eligible1:double     -1.151e-02  1.663e-02  -0.692 0.489334
> arbitr.elgible.1:double          2.169e-03  1.938e-02   0.112 0.910985
> strike.out:triple               -8.106e-04  6.023e-04  -1.346 0.179475
> batting:triple                   5.179e-01  5.599e-01   0.925 0.355841
> walk:triple                      8.755e-04  9.262e-04   0.945 0.345349
> hit:hr                          -3.320e-04  2.626e-04  -1.264 0.207180
> rbi:hr                           4.748e-04  3.015e-04   1.575 0.116414
> free.agent.eligible1:hr          1.840e-02  2.313e-02   0.796 0.426972
> free.agent.19911:hr              7.216e-02  1.889e-02   3.819 0.000165
***
> arbitr.elgible.1:hr              4.111e-02  2.803e-02   1.467 0.143564
> arbitr.19911:hr                 -2.368e-02  4.647e-02  -0.510 0.610723
> hit:walk                         3.173e-05  7.826e-05   0.405 0.685442
> free.agent.eligible1:walk       -5.423e-03  4.984e-03  -1.088 0.277472
> rbi:walk                        -7.569e-05  1.313e-04  -0.577 0.564598
> rbi:stolen.base                  3.980e-05  1.605e-04   0.248 0.804409
> strike.out:stolen.base          -2.611e-04  1.615e-04  -1.617 0.107004
> batting:stolen.base              1.552e-01  1.434e-01   1.082 0.280020
> arbitr.elgible.1:error           3.930e-03  1.390e-02   0.283 0.777495
> ---
> Signif. codes:  0 `***' 0.001 `**' 0.01 `*' 0.05 `.' 0.1 ` ' 1
>
> Residual standard error: 0.4925 on 280 degrees of freedom
> Multiple R-Squared: 0.854,      Adjusted R-squared: 0.8248
> F-statistic: 29.24 on 56 and 280 DF,  p-value: < 2.2e-16
>

______________________________________________
R-help at stat.math.ethz.ch mailing list
https://www.stat.math.ethz.ch/mailman/listinfo/r-help