[R] weighted regression

[rex_bryan@urscorp.com]

Paul
Thanks for your quick response.  Yes I have
x and y backward.  Thomas Blackwell pointed that out to
me. Oops.  And yes I will be considering more than 3 points
for any regression.  This little data set was first presented by
a math wizard in a MathCAD forum explaining the mathematics
behind weighted regression.  When I tried Statistica on the data set
I  found that their definition of weights was a replication counter NOT in
the spirit of 1/var.
It put me into a blue funk ... I hate it when my state-of-the-art tools turn
out to be a sham.
I then turned to R and all appears to be mathemagically working well.
As I walk the dogs I'm thinking (amittedly confusedly) on how prediction and
confidence
intervals could be generated around a weighted regression line.  My thoughts
range from purloining concepts from kriging, or bootstapping ... in each
case
I come to the conclusion that this is more complex topic than I thought....
Oh well...it keeps me interested in talking to smart people like you.
Have a Merry Christmas
REX
----- Original Message -----
From: "Paul E. Johnson" <pauljohn at ku.edu>
To: "rex_bryan at urscorp.com" <Rex_Bryan at urscorp.com>
Sent: Friday, December 19, 2003 2:43 PM
Subject: Re: [R] weighted regression


>
> I keep new user tips at http://www.ku.edu/~pauljohn/R/Rtips.html
>
> I have some observations on your example:
>
> 1. You have y and x backward in your lm code
>
> 2. You get rid of the intercept by lm(y~x-1) but it is not generally
> advisable.  it will bias the slope parameter if your assumption is wrong.
>
> 3. You are right that Weighted Least Squares can be accomplished by
> weights in lm.  Please read the description of weight in ?lm
>
> weights: an optional vector of weights to be used in the fitting
>           process. If specified, weighted least squares is used with
>           weights 'weights' (that is, minimizing 'sum(w*e^2)');
>           otherwise ordinary least squares is used.
>
> If you understand WLS, then you will see how to enter the variable w.
>
> 4. I hope you aren't really doing regression with 3 observations.
>
> pj
>
>
> rex_bryan at urscorp.com wrote:
>
> >To all
> >
> >I have some simple questions pertaining to weights used in regression.
> >If the variability of the dependent variable (y) is a function of the
magnitude of predictor
> >variable (x), can the use of weights give an appropriate answer to the
regression parameters
> >and the std errors?
> >
> >Assume that y at x=1 and 6 has a standard deviation of 0.1 and at x=11 it
is 0.4
> >Then according to a web page on weighted regression for a calibration
curve at
> >http://member.nifty.ne.jp/mniwa/rev006.htm, I should use 1/(std^2) for
each weight.
> >
> >i.e. for x=1 and 6, w = 100 and x=11, w = 6.25
> >
> >In R the run is:
> >
> >
> >
> >>y<-c(1,6,11)
> >>x<-c(6.7,6.7,6.6)
> >>w<-c(100,100,6.25)
> >>reg <-lm(x~y, weight=w)
> >>summary(reg)
> >>
> >>
> >
> >Call:
> >lm(formula = x ~ y, weights = w)
> >
> >Residuals:
> >       1        2        3
> >-0.04762  0.09524 -0.19048
> >
> >Coefficients:
> >             Estimate Std. Error t value Pr(>|t|)
> >(Intercept)  6.707619   0.025431 263.762  0.00241 **
> >y           -0.002857   0.005471  -0.522  0.69361
> >---
> >Signif. codes:  0 `***' 0.001 `**' 0.01 `*' 0.05 `.' 0.1 ` ' 1
> >
> >Residual standard error: 0.2182 on 1 degrees of freedom
> >Multiple R-Squared: 0.2143,     Adjusted R-squared: -0.5714
> >F-statistic: 0.2727 on 1 and 1 DF,  p-value: 0.6936
> >
> >Am I using the weight method correctly?
> >And if so does the Estimated Std. Error for the Intercept and slope make
sense?
> >
> >On another note.  How does one do a regression with the origin fixed at
0?
> >
> >Merry Christmas
> >
> >REX
> >
> >
> >
> >
> >
> > [[alternative HTML version deleted]]
> >
> >______________________________________________
> >R-help at stat.math.ethz.ch mailing list
> >https://www.stat.math.ethz.ch/mailman/listinfo/r-help
> >
> >
>
>
> --
> Paul E. Johnson                       email: pauljohn at ukans.edu
> Dept. of Political Science            http://lark.cc.ukans.edu/~pauljohn
> University of Kansas                  Office: (785) 864-9086
> Lawrence, Kansas 66045                FAX: (785) 864-5700
>