[R] Large Stata file Import in R

[Carlos J. Gil Bellosta]

Hello,

You are dealing with two different problems at the same time: importing
Stata data and importing a relatively big file.

Can you try to export your data to txt file first and try to import from
it directly?

Secondly, problems concerning reading big files with R occur quite often
and there are plenty of discussions and workarounds described in
previous posts. 

I am the author of a new package aimed at reading files column-wise. It
is quite frugal with memory as the data resides mostly on R dumped files
of the objects representing the rows of your data.

You can install and test it via

install.packages("colbycol",repos="http://R-Forge.R-project.org")

Comments and bug reports are more than welcome!

Best regards,

Carlos J. Gil Bellosta
http://www.datanalytics.com




On Mon, 2009-06-29 at 15:50 +0100, saurav pathak wrote:
> Hi
> 
> I am using Stata 10 and I need to import a data set in stata 10 to R, I have
> saved the dataset in lower versions of Stata as well by using saveold
> command in Stata.
> 
> My RAM is 4gb and the stata file is 600MB, I am getting an error message
> which says :
> 
> "Error: cannot allocate vector of size 3.4 Mb
> In addition: There were 50 or more warnings (use warnings() to see the first
> 50)"
> 
> Thus far I have already tried the following
> 
> 1. By right clicking on the R icon I have used --max-mem-size=1000M in the
> "target" under "properties of the R icon
> 2. I have used library(foreign) at teh command prompt
> 3. then I use trialfile <- read.dta("C:/filename.dta")
>  Here I get error for a Stata data file that is 600MB in size, however, with
> data set in Stata 10 and Stata 8 of the size of 200KB, I have successfully
> being able to import the stata file in R
> 
> I am therefor confused whteher there is problem with the version of my stata
> file (which should not eb the case as I the smaller file of both versions
> are working fine) or is it the size issue,
> 
> Its pretty important for me, kindly address this question
> Thanks
> Saurav
> 
>