[R] Working with large matrix

[Jonathan Baron]

Another possible interpretation is that each row contains several
vectors, possibly laid end to end.  For example, if the row J were
length 30 (which it isn't), the first 10 might be Y, the next 10 X1,
and the next N2.  The function might be lm(J[1:10] ~ J[11:20] +
J[21:30]).

If this is what is going on, then one possible approach is to
transform this matrix into the "long form", with one observation
(value of Y) and two predictors per row instead of (here) 10 values of
Y, then use lmList() in the lme4 package.  I suspect that this is
faster than using apply() with lm(), but I'm not sure.  I don't think
this will help load the file.

Jon

On 09/12/09 08:32, Douglas Bates wrote:
> On Fri, Sep 11, 2009 at 12:15 PM, A Ezhil <ezhil02 at yahoo.com> wrote:
> > Dear All,
> 
> > I have large matrix (46000 x 11250). I would like to do the linear regression for 
> each row. I wrote a simple function that has lm() and used apply(mat,1,func). The 
> issue is that it takes ages to load the file and also to finish the lm. I am using 
> LINUX 64 bit with 32G mem. Is there an elegant and fast way of completing this task?
> 
> I'm not quite sure what you mean by "do the linear regression for each
> row" but you may find it convenient to use a matrix as the response in
> the call to lm.  In that case lm fits the same model, described by the
> right hand side of the formula, to each column of the matrix on the
> left hand side.
> 
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

-- 
Jonathan Baron, Professor of Psychology, University of Pennsylvania
Home page: http://www.sas.upenn.edu/~baron
Editor: Judgment and Decision Making (http://journal.sjdm.org)