[R] cbind alternate

[David Winsemius]

On Jan 6, 2012, at 12:43 PM, Mary Kindall wrote:

> I have two one dimensional list of elements and want to perform  
> cbind and
> then write into a file. The number of entries are more than a  
> million in
> both lists. R is taking a lot of time performing this operation.
>
> Is there any alternate way to perform cbind?
>
> x =
> y = table2[1:1000000,5]
>
> z = cbind(x,y)   //hanging the machine

You should have been able to bypass the intermediate steps with just:

z = cbind( table1[1:1000000,1] table2[1:1000000,5])

Whether you will have sufficient contiguous memory for that object at  
the moment or even after rm(x), rm(y) is in doubt, but had you not  
created the unneeded x and y, you _might_ have succeeded in your  
limited environment. (Real answer: Buy more RAM.)

I speculate that you are on Windows and so refer your to the R-Win FAQ  
for further reading about memory limits.


>
> write.table(z,'out.txt)

I do not know of a way to bypass the requirement of a named object to  
pass to write.table, but testing suggests that you could try:

write( t(cbind( table1[1:1000000,1] table2[1:1000000,5])).   
"test.txt", 2)

write()  does not require a named object but is less inquisitive than  
write table and will give you a transposed matrix with 5 columns by  
default which will really mess up things, so you need to transpose and  
specify the number of columns. (And that may not save any space over  
creating a "z" object.)

So there is another thread today to which master R programmer Bill  
Dunlap has offered this strategy (with minor modifications to your  
situation by me):

###
f1 <- function (n, fileName) {
      unlink(fileName)
      system.time({
          fileConn <- file(fileName, "wt")
          on.exit(close(fileConn))
          for (i in seq_len(n)) cat( table1[i, 1], " ",
                                     table2[i, 5],
                                   "\n", file = fileConn)
      })
  }

f1(1000000, 'out.txt')

#------------
-- 

David Winsemius, MD
West Hartford, CT