Thank you!! Using lm.fit did the trick.

2012/2/23 R. Michael Weylandt <michael.weylandt@gmail.com>

> It looks like what you are doing is reasonably efficient: I do think
> there's a residuals element to the object returned by lm() so you
> could just call that directly (which will be just a little more
> efficient).
>
> The bulk of the time is probably being taken up in the lm() call,
> which has alot of overhead: you could use fastLm from the
> RcppArmadillo package or lm.fit() directly to cut alot of this out.
>
> Michael
>
> On Wed, Feb 22, 2012 at 9:10 PM, Martin <misenial@gmail.com> wrote:
> > Hello,
> > I'm very new to R so my apologies if I'm making an obvious mistake.
> >
> > I have a data frame with ~170k rows and 14 numeric variables. The first 2
> > of those variables (let's call them group1 and group2) are used to define
> > groups: each unique pair of (group1,group2) is a group. There are roughly
> > 50k such unique groups, with sizes varying from 1 through 40 rows each.
> >
> > My objective is to fit a linear regression within each group and get its
> > mean square error (MSE). So the final output needs to be a collection of
> > 50k MSE's.  Now, regardless of the size of the group, the regression
> needs
> > to be run on exactly 40 observations. If the group has less than 40
> > observations, then I need to add rows to get to 40, populating all
> > variables with 0's for those extra rows. Here's the function I wrote to
> do
> > this:
> >
> > get_MSE = function(x) {
> >  rownames(x) = x$ID  #'ID' can take on any value from 1 to 40.
> >  x = x[as.character(1:40), ]
> >  x[is.na(x)] = 0
> >  regressionResult = lm(A ~ B + C + D + E, data=x)  #A-E are some
> variables
> > in the data frame.
> >  MSE = mean((regressionResult$fitted.values - A)^2)
> >  return(MSE)
> > }
> >
> > library(plyr)
> > output = ddply(dataset, list(dataset$group1, dataset$group2), get_MSE)
> >
> > The above code takes about 10 minutes to run, but I'd really need it to
> be
> > much faster, if at all possible. Is there anything I can do to speed up
> the
> > code?
> >
> > Thank you very much in advance.
> >
> > Jose
> >
> >        [[alternative HTML version deleted]]
> >
> > ______________________________________________
> > R-help@r-project.org mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-help
> > PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> > and provide commented, minimal, self-contained, reproducible code.
>

	[[alternative HTML version deleted]]

