[R] Standard error of coefficient in linear regression
Dimitrios Rizopoulos
Dimitris.Rizopoulos at med.kuleuven.be
Sun Sep 17 21:13:02 CEST 2006
these standard errors and other quantities are calculated as by
products of the QR decomposition used in lm.fit(). A simple way (but
not efficient) to obtain them is:
exped <- c(4.2, 6.1, 3.9, 5.7, 7.3, 5.9)
sales <- c(27.1, 30.4, 25.0, 29.7, 40.1, 28.8)
S <- data.frame(exped, sales)
lmfit <- lm(sales ~ exped, data = S)
X <- model.matrix(lmfit)
sigma2 <- sum((sales - fitted(lmfit))^2) / (nrow(X) - ncol(X))
sqrt(sigma2)
sqrt(diag(solve(crossprod(X))) * sigma2)
I hope it helps.
Best,
Dimitris
----
Dimitris Rizopoulos
Ph.D. Student
Biostatistical Centre
School of Public Health
Catholic University of Leuven
Address: Kapucijnenvoer 35, Leuven, Belgium
Tel: +32/(0)16/336899
Fax: +32/(0)16/337015
Web: http://med.kuleuven.be/biostat/
http://www.student.kuleuven.be/~m0390867/dimitris.htm
Quoting Maciej Bliziński <m.blizinski at wit.edu.pl>:
> Hello R users,
>
> I have a substantial question about statistics, not about R itself, but
> I would love to have an answer from an R user, in form of an example in
> R syntax. I have spent whole Sunday searching in Google and browsing the
> books. I've been really close to the answer but there are at least three
> standard errors you can talk about in the linear regression and I'm
> really confused. The question is:
>
> How exactly are standard errors of coefficients calculated in the linear
> regression?
>
> Here's an example from a website I've read [1]. A company wants to know
> if there is a relationship between its advertising expenditures and its
> sales volume.
>
> ========================================================
>> exped <- c(4.2, 6.1, 3.9, 5.7, 7.3, 5.9)
>> sales <- c(27.1, 30.4, 25.0, 29.7, 40.1, 28.8)
>> S <- data.frame(exped, sales)
>> summary(lm(sales ~ exped, data = S))
>
> Call:
> lm(formula = sales ~ exped, data = S)
>
> Residuals:
> 1 2 3 4 5 6
> 1.7643 -1.9310 0.7688 -1.1583 3.3509 -2.7947
>
> Coefficients:
> Estimate Std. Error t value Pr(>|t|)
> (Intercept) 9.8725 5.2394 1.884 0.1326
> exped 3.6817 0.9295 3.961 0.0167 *
> ---
> Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
>
> Residual standard error: 2.637 on 4 degrees of freedom
> Multiple R-Squared: 0.7968, Adjusted R-squared: 0.7461
> F-statistic: 15.69 on 1 and 4 DF, p-value: 0.01666
> ========================================================
>
> I can calculate the standard error of the estimate, according to the
> equation [2]...
>
>> S.m <- lm(sales ~ exped, data = S)
>> S$pred <- predict(S.m)
>> S$ye <- S$sales - S$pred
>> S$ye2 <- S$ye ^ 2
>> Se <- sqrt(sum(S$ye2)/(length(S$sales) - 1 - 1))
>> Se
> [1] 2.636901
>
> ...which matches the "Residual standard error" and I'm on the right
> track. Next step would be to use the equation [3] to calculate the
> standard error of the regression coefficient (here: exped). The equation
> [3] uses two variables, meaning of which I can't really figure out. As
> the calculated value Sb is scalar, all the parameters need also to be
> scalars. I've already calculated Se, so I'm missing x and \bar{x}. The
> latter could be the estimated coefficient. What is x then?
>
> Regards,
> Maciej
>
> [1] http://www.statpac.com/statistics-calculator/correlation-regression.htm
> [2] http://www.answers.com/topic/standard-error-of-the-estimate
> [3] http://www.answers.com/topic/standard-error-of-the-regression-coefficient
>
> --
> Maciej Bliziński <m.blizinski at wit.edu.pl>
> http://automatthias.wordpress.com
>
> ______________________________________________
> R-help at stat.math.ethz.ch mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>
Disclaimer: http://www.kuleuven.be/cwis/email_disclaimer.htm
More information about the R-help
mailing list