[R] Reading SAS data into R

[Gilpin, Scott]

> -----Original Message-----
> From: r-help-bounces at stat.math.ethz.ch [mailto:r-help-
> bounces at stat.math.ethz.ch] On Behalf Of Søren Højsgaard
> Sent: Friday, August 27, 2004 11:46 AM
> To: r-help at stat.math.ethz.ch
> Cc: Søren Højsgaard
> Subject: [R] Reading SAS data into R
> 
> Dear all,
> One of my students (whom I am trying to convince to use R) wants to get a
> fairly large SAS dataset into R (about 150mB). An obvious and simple thing
> she tried was to write the dataset as a .csv-file and then read that into
> R, but that takes forever (or something close to that..). The dataset is
> so large, that exporting it as an Excel file from SAS is not feasible
> (more than 65000 lines). I am reluctant to ask her to go through all the
> data base steps (then she'll just stick to SAS...). Can anyone help me out
> on that one?

What platform are you on, and how much memory do you have?  150mb isn't *that* large - but it will depend on your system.  See the FAQs regarding memory issues, as well as ?mem.limits and ?gc

It also depends on the type of data you're working with (integers take half the space of numerics) and what type of analysis you want to do.  The CSV approach should work fine - but you'll want to use scan instead of read.table.  You can use scan to read data in chunks (using skip and nlines), do something useful with this chunk of data, run gc(), and then read in another chunk.

In general, users I've seen who try to go from SAS to R don't seem to realize that R is not a data manipulation language, and hence just try to shove their entire dataset into R and manipulate it (which is what they would do in SAS).  Perl, awk, cut, etc. (not to mention DBMSes) are all very useful for processing data before putting it into R.


> Thanks in advance
> Søren Højsgaard
> 
> 
> 	[[alternative HTML version deleted]]
> 
> ______________________________________________
> R-help at stat.math.ethz.ch mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide! http://www.R-project.org/posting-
> guide.html