[R] Re: large survey data

[Peter Dalgaard BSA]

Douglas Bates <bates at stat.wisc.edu> writes:

> Micha? Bojanowski <bojanr at wp.pl> writes:
> 
> > Recently I came across a problem. I have to analyze a large survey 
> > data - something about 600 columns and 10000 rows (tab-delimited file 
> > with names in the header). I was able do import the data into an 
> > object, but there is no more memory left.
> > 
> > Is there a way to import the data column by column? I have to analyze 
> > the whole data, but only two variables at a time.
> 
> You will probably need to do the data manipulation externally.
> Two possible solutions are to use a scripting language like python or
> perl or to store the data in a relational database like PostgreSQL or
> MySQL.  For data of this size I would recommend the relational
> database approach.
> 
> R has packages to connect to PostgreSQL or to MySQL.

We didn't see what OS this came from so it might well be Windows....

There, you have some possibilities of setting up an ODBC connection to
a text file (via Control Panel - slightly cryptic, but I managed to
get it to play at some point). You should be able to access the
table as a database using the RODBC package and that will allow you to
do the selection of cases/variables.

-- 
   O__  ---- Peter Dalgaard             Blegdamsvej 3  
  c/ /'_ --- Dept. of Biostatistics     2200 Cph. N   
 (*) \(*) -- University of Copenhagen   Denmark      Ph: (+45) 35327918
~~~~~~~~~~ - (p.dalgaard at biostat.ku.dk)             FAX: (+45) 35327907
-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
Send "info", "help", or "[un]subscribe"
(in the "body", not the subject !)  To: r-help-request at stat.math.ethz.ch
_._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._