[R] Building a big.matrix using foreach

[Michael Knudsen]

On Mon, Jul 20, 2009 at 10:23 AM, Michael Knudsen<micknudsen at gmail.com> wrote:

> Thanks! The shared.big.matrix was exactly what I needed. It still
> remains for me, though, to check if I run into memory problems anyway.
> It doesn't seem as if there's a "don't return anything" option in the
> foreach function (also mentioned in my previous post in this thread).

Oops! The following code made R go bananas and left the system dead
for fifteen minutes. Trying a simple 'ls' in a terminal resulted in
something like "Segmentation fault. Too many files open".

distances = shared.big.matrix(nrow=length(x),ncol=length(x),type="double",init=0)
desc = describe(distances)
	
foreach (i=1:(length(x)-1)) %dopar%
{
   first_x = x[[i]]
   these_distances = numeric(length(x))
				
   for (j in (i+1):length(x))
   {
      second_x = x[[j]]
      these_distances[j] = as.numeric(ks.test(first_x,second_x)$statistic)
   }
		
   y = attach.big.matrix(desc)
   y[i,] = these_distances
}

The error message from R was:

*** caught segfault ***
address (nil), cause 'memory not mapped'

/Michael

-- 
Michael Knudsen
micknudsen at gmail.com
http://lifeofknudsen.blogspot.com/