I have a complex data-set to analyze and am trying to put mcapply to work
hastening computation time. The data is nested and I need to construct the
model at using various subsets of the data. Basically I've got 3 different
brain regions each with data for 5 different experiments. I need to model
each brain region including all experiments and then each experiment within
each region. So my initial construction was to loop i over the 3 brain
regions and then j over the 5 experiments. Each modeling exercise actually
models and does an ANOVA on 100000 genes.

I have 8 cores available to me on a coreI7 chip with 12 GB of memory. I
constructed a very simple example with an mcapply nested within another
mcapply and the example worked perfectly. All the appropriate processes were
spawned and the output just as I hoped. Unfortunately, my real world example
fails despite the inner mcapply working on its own and the outer mcapply
working if the inner mcapply is commented out. Way more R processes are
spawned that necessary and most only take up a few kB. Using 'top' I can see
that most are just idle.

I was pleasantly surprised when my simple example worked (I wasn't sure it
would) but now I am perplexed that a larger real-world example fails. I am
not familiar enough with Linux system calls to understand on my own what
might be going on. Could there be memory corruption using larger
data-structures? Should this in theory work or is it just generally a bad
idea on principle?

Mark

Mark W. Kimpel MD  ** Neuroinformatics ** Dept. of Psychiatry
Indiana University School of Medicine

15032 Hunter Court, Westfield, IN  46074

(317) 490-5129 Work, & Mobile & VoiceMail
(317) 399-1219 Skype No Voicemail please

	[[alternative HTML version deleted]]

