[R] write.table very slow

[Ott Toomet]

Hi,

I think the problem lies in the code of write.table().  It is essentially a
paste() function, which pastes all the data in the table into a long
character string and thereafter writes the string into file.  I was not able
to write a dataset of 7500 obs times 1200 variables at all, I had to split
it up into smaller units and write those separately.  In addition caused it
much swapping on my 128MB system.

I think (I have not tried) it could work faster in your case if you just
save the observatons separately into separate files and thereafter merge the
files (but it is worth of doing only if you have to write the table
repeatedly, of course).  In long run I think a rewrite of the write.table()
in C in such a way that it do not store the whole file in memory may be a
solution.

Regards,

Ott Toomet
-------------------------------
On Wed, 5 Dec 2001, Cole Harris wrote:

> When writing tables with a large number of columns, write.table() seems to take way too much time - e.g. a table with ~80 rows and ~6000 columns takes ~30 min cpu on my 900 MHz pc.
> I would appreciate any explainations or advice.
>
> Thanks,
> Cole
> -.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
> r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
> Send "info", "help", or "[un]subscribe"
> (in the "body", not the subject !)  To: r-help-request at stat.math.ethz.ch
> _._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._
>

-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
Send "info", "help", or "[un]subscribe"
(in the "body", not the subject !)  To: r-help-request at stat.math.ethz.ch
_._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._