[R] Reading SAS data into R

[Frank E Harrell Jr]

------- Begin quoted text
From: "Gilpin, Scott" <sgilpin at doubleclick.net>
 >-----Original Message-----
 >> From: r-help-bounces at stat.math.ethz.ch [mailto:r-help-
 >> bounces at stat.math.ethz.ch] On Behalf Of Søren Højsgaard
 >> Sent: Friday, August 27, 2004 11:46 AM
 >> To: r-help at stat.math.ethz.ch
 >> Cc: Søren Højsgaard
 >> Subject: [R] Reading SAS data into R
 >>
 >> Dear all,
 >> One of my students (whom I am trying to convince to use R) wants to 
get a
 >> fairly large SAS dataset into R (about 150mB). An obvious and simple 
thing
 >> she tried was to write the dataset as a .csv-file and then read that 
into
 >> R, but that takes forever (or something close to that..). The dataset is
 >> so large, that exporting it as an Excel file from SAS is not feasible
 >> (more than 65000 lines). I am reluctant to ask her to go through all the
 >> data base steps (then she'll just stick to SAS...). Can anyone help 
me out
 >> on that one?


What platform are you on, and how much memory do you have?  150mb isn't 
*that* large - but it will depend on your system.  See the FAQs 
regarding memory issues, as well as ?mem.limits and ?gc

It also depends on the type of data you're working with (integers take 
half the space of numerics) and what type of analysis you want to do. 
The CSV approach should work fine - but you'll want to use scan instead 
of read.table.  You can use scan to read data in chunks (using skip and 
nlines), do something useful with this chunk of data, run gc(), and then 
read in another chunk.

In general, users I've seen who try to go from SAS to R don't seem to 
realize that R is not a data manipulation language, and hence just try 
to shove their entire dataset into R and manipulate it (which is what 
they would do in SAS).  Perl, awk, cut, etc. (not to mention DBMSes) are 
all very useful for processing data before putting it into R.

-----------End quoted text

R is also a data manipulation language and once you get used to it it is 
better than SAS at data manipulation, for non-huge datasets.  We have 
many examples of data manipulation with R on our web site 
http://biostat.mc.vanderbilt.edu (see especially the Alzola and Harrell 
text).

What we do for importing SAS datasets is described at 
http://biostat.mc.vanderbilt.edu/twiki/bin/view/Main/SASexportHowto . 
This will preserve labels and value labels, handle dates, times, 
date/times, and get around problems we've faced in importing SAS V5 
transport files using the foreign package, by having SAS run PROC EXPORT 
to create csv files.

-- 
Frank E Harrell Jr   Professor and Chair           School of Medicine
                      Department of Biostatistics   Vanderbilt University