[R] Standard error of coefficient in linear regression

[Dimitrios Rizopoulos]

these standard errors and other quantities are calculated as by  
products of the QR decomposition used in lm.fit(). A simple way (but  
not efficient) to obtain them is:

exped <- c(4.2, 6.1, 3.9, 5.7, 7.3, 5.9)
sales <- c(27.1, 30.4, 25.0, 29.7, 40.1, 28.8)
S <- data.frame(exped, sales)
lmfit <- lm(sales ~ exped, data = S)

X <- model.matrix(lmfit)
sigma2 <- sum((sales - fitted(lmfit))^2) / (nrow(X) - ncol(X))

sqrt(sigma2)
sqrt(diag(solve(crossprod(X))) * sigma2)

I hope it helps.

Best,
Dimitris


----
Dimitris Rizopoulos
Ph.D. Student
Biostatistical Centre
School of Public Health
Catholic University of Leuven

Address: Kapucijnenvoer 35, Leuven, Belgium
Tel: +32/(0)16/336899
Fax: +32/(0)16/337015
Web: http://med.kuleuven.be/biostat/
      http://www.student.kuleuven.be/~m0390867/dimitris.htm


Quoting Maciej Bliziński <m.blizinski at wit.edu.pl>:

> Hello R users,
>
> I have a substantial question about statistics, not about R itself, but
> I would love to have an answer from an R user, in form of an example in
> R syntax. I have spent whole Sunday searching in Google and browsing the
> books. I've been really close to the answer but there are at least three
> standard errors you can talk about in the linear regression and I'm
> really confused. The question is:
>
> How exactly are standard errors of coefficients calculated in the linear
> regression?
>
> Here's an example from a website I've read [1]. A company wants to know
> if there is a relationship between its advertising expenditures and its
> sales volume.
>
> ========================================================
>> exped <- c(4.2, 6.1, 3.9, 5.7, 7.3, 5.9)
>> sales <- c(27.1, 30.4, 25.0, 29.7, 40.1, 28.8)
>> S <- data.frame(exped, sales)
>> summary(lm(sales ~ exped, data = S))
>
> Call:
> lm(formula = sales ~ exped, data = S)
>
> Residuals:
>       1       2       3       4       5       6
>  1.7643 -1.9310  0.7688 -1.1583  3.3509 -2.7947
>
> Coefficients:
>             Estimate Std. Error t value Pr(>|t|)
> (Intercept)   9.8725     5.2394   1.884   0.1326
> exped         3.6817     0.9295   3.961   0.0167 *
> ---
> Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
>
> Residual standard error: 2.637 on 4 degrees of freedom
> Multiple R-Squared: 0.7968,     Adjusted R-squared: 0.7461
> F-statistic: 15.69 on 1 and 4 DF,  p-value: 0.01666
> ========================================================
>
> I can calculate the standard error of the estimate, according to the
> equation [2]...
>
>> S.m <- lm(sales ~ exped, data = S)
>> S$pred <- predict(S.m)
>> S$ye <- S$sales - S$pred
>> S$ye2 <- S$ye ^ 2
>> Se <- sqrt(sum(S$ye2)/(length(S$sales) - 1 - 1))
>> Se
> [1] 2.636901
>
> ...which matches the "Residual standard error" and I'm on the right
> track. Next step would be to use the equation [3] to calculate the
> standard error of the regression coefficient (here: exped). The equation
> [3] uses two variables, meaning of which I can't really figure out. As
> the calculated value Sb is scalar, all the parameters need also to be
> scalars. I've already calculated Se, so I'm missing x and \bar{x}. The
> latter could be the estimated coefficient. What is x then?
>
> Regards,
> Maciej
>
> [1] http://www.statpac.com/statistics-calculator/correlation-regression.htm
> [2] http://www.answers.com/topic/standard-error-of-the-estimate
> [3] http://www.answers.com/topic/standard-error-of-the-regression-coefficient
>
> --
> Maciej Bliziński <m.blizinski at wit.edu.pl>
> http://automatthias.wordpress.com
>
> ______________________________________________
> R-help at stat.math.ethz.ch mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>



Disclaimer: http://www.kuleuven.be/cwis/email_disclaimer.htm