[R] Stata file and R Interaction :File Size Problem in Import

[Thomas Lumley]

This is at least the fourth time you have asked this question, which is at least two more than the maximum excusable number of times.

The error message says that your computer doesn't have enough memory to load this data set. This pretty clearly suggests that the size of the file is the problem. This isn't something we can fix.

If you need the whole file for your analysis you are probably out of luck -- since Stata binary files use smaller data types than R uses in memory, a 600Mb file is quite likely over 1Gb in memory, and this isn't going to work on a 32 bit system [it looks as though you are using Windows, though you don't actually *say*].   The fact that you told R to use at most 1Gb of memory with --max-mem-size=1000M would have pretty much guaranteed that it would fail, but I think it is likely to be impossible even if you allow R to use all your available memory.

If you don't need the whole file at once, saving it as a text file will allow you to read just parts of the file.

 Alternatively, if I recall correctly, two-stage least squares just involves two ordinary least squares fits, so you could use the biglm package to fit each of these least squares fits to a data set that was too big to fit in memory.


      -thomas

Thomas Lumley			Assoc. Professor, Biostatistics
tlumley at u.washington.edu	University of Washington, Seattle