[R] classification for huge datasets: SVM yields memory troubles

[Berton Gunter]

" I have a matrix with 30 observations and roughly 30000 
variables, ... <snipped>" 

Comment: This is ** not ** a "huge" data set -- it is a tiny one with a
large number of covariates. The difference is: If it were truly huge, SVM
and/or LDA or ... might actually be able to produce useful results. With so
few data and so many variables, it is hard to see how any approach that one
uses is not simply a fancy random number generator.


-- Bert Gunter
Genentech Non-Clinical Statistics
South San Francisco, CA
 
"The business of the statistician is to catalyze the scientific learning
process."  - George E. P. Box
 
 

> -----Original Message-----
> From: r-help-bounces at stat.math.ethz.ch 
> [mailto:r-help-bounces at stat.math.ethz.ch] On Behalf Of Andreas
> Sent: Monday, December 13, 2004 12:56 PM
> To: r-help at stat.math.ethz.ch
> Subject: Re: [R] classification for huge datasets: SVM yields 
> memory troubles
> 
> Hi,
> 
> I'm a beginner in the SVM-module but I have seen there is a 
> parameter called
> :
> cachesize #cache memory in MB (default 40)
> 
> please let me know if this parameter solved your problem, I 
> might get the
> same number of samples in the near future.
> 
> regards Andreas
> 
> "Christoph Lehmann" <christoph.lehmann at gmx.ch> schrieb im Newsbeitrag
> news:41BD8A9F.4040509 at gmx.ch...
> > Hi
> > I have a matrix with 30 observations and roughly 30000 
> variables, each
> > obs belongs to one of two groups. With svm and slda I get 
> into memory
> > troubles ('cannot allocate vector of size' roughly 2G). PCA LDA runs
> > fine. Are there any way to use the memory issue withe 
> SVM's? Or can you
> > recommend any other classification method for such huge datasets?
> >
> >
> > P.S. I run suse 9.1 on a 2G RAM PIV machine.
> > thanks for a hint
> >
> > Christoph
> >
> > ______________________________________________
> > R-help at stat.math.ethz.ch mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-help
> > PLEASE do read the posting guide!
> http://www.R-project.org/posting-guide.html
> >
> 
> ______________________________________________
> R-help at stat.math.ethz.ch mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide! 
> http://www.R-project.org/posting-guide.html
>