[R] building a formula for glm() with 30,000 independent variables

[Christian Schulz]

....if you have really enough cases isn't it at first a technical challenge
;-)
( the statistical sense ?)
my suggestion is, try weka (www.cs.waikato.ac.nz/ml/weka/)
, a very big linux-machine? +  "data-base for large application"   & perhaps
the Attribute Evaluator
as Meta-Classifier (logistic regresssion is possible,too!) is a interesting
tool for you !?

 i didn't now weka have the possibilty and perhaps crash , too.
But in the past there where some topics (search the mailing list archive)
about data-mining purposes with ~ 2 GB data - good luck !

regards,christian




----- Original Message -----
From: "Ben Liblit" <liblit at eecs.berkeley.edu>
To: <r-help at stat.math.ethz.ch>
Sent: Sunday, November 10, 2002 3:28 PM
Subject: [R] building a formula for glm() with 30,000 independent variables


> I would like to use R to perform a logistic regression with about
> 30,000 independent variables.  That's right, thirty thousand.  Most
> will be irrelevant: the intent is to use the regression to identify
> the few that actually matter.
>
> Among other things, this calls for giving glm() a colossal "y ~ ..."
> formula with thirty thousand summed terms on its right hand side.  I
> build up the formula as a string and then call as.formula() to convert
> it.  Unfortunately, the conversion fails.  The parser reports that it
> has overflowed its stack.  :-(
>
> Is there any way to pull this off in R?  Can anyone suggest
> alternatives to glm() or to R itself that might be capable of handling
> a problem of this size?  Or am I insane to even be considering an
> analysis like this?
>
> Thanks!
>
> -.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.
-.-.-
> r-help mailing list -- Read
http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
> Send "info", "help", or "[un]subscribe"
> (in the "body", not the subject !)  To: r-help-request at stat.math.ethz.ch
>
_._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._.
_._


-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
Send "info", "help", or "[un]subscribe"
(in the "body", not the subject !)  To: r-help-request at stat.math.ethz.ch
_._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._