On Thu, Aug 26, 2010 at 7:30 PM, Norm Matloff <matloff@cs.ucdavis.edu>wrote:

> If your machine uses DMA for disk access, then your disk activity should
> not be reflected in CPU utilization.  Some systems allow you to
> enable/disable DMA, and you may want to look into enabling it.
>
> But even if the CPU is being used for disk actions, I don't see why it
> should always be one particular core that is handling that action.  It
> should be pretty much random across the cores.  Maybe I'm missing
> something?
>
>
I understood that the CPU utilization was changing throughout the
processing; perhaps I misunderstood things.

As for disk and CPU, I didn't mean that the CPU was eating cycles doing disk
IO but that some of the processes were waiting for data.  On our clusters,
this is the most common reason for non-linear performance gains when adding
cores to processing--the disk is limiting.  There are certainly other
explanations, though.

Sean


> Norm Matloff
>
> On Thu, Aug 26, 2010 at 06:35:01PM -0400, Sean Davis wrote:
> > On Thu, Aug 26, 2010 at 6:14 PM, Brian D Peyser PhD <bpeyser@jhmi.edu
> >wrote:
> >
> > > Hi everyone,
> > >
> > > I am trying to run code in parallel across 4 cores with foreach and
> > > doMC. The code runs fine, and it will spawn 4 R processes, but these
> > > processes aren't all on separate cores. Often, I will see one core
> maxed
> > > with each R process taking about 24% of the CPU. I also see something
> > > like 96%, 98%, 48%, 48% (3 of 4 cores being used) or 32%, 32%, 96%,
> 32%.
> > > (2 cores). This may change during the run, and I almost never see 4
> > > cores getting maxed.
> > >
> > > I am running like this (pseudo-code):
> > >
> > > > library("doMC")
> > > > registerDoMC(cores=4) # I have a 4-core processor
> > > >       # Read in data....
> > > > foreach(obj=gsub(".txt", "",
> > > files), .inorder=FALSE, .errorhandling="pass", .options.multicore =
> > > list(preschedule = FALSE)) %dopar% {
> > > >       # Do lots of computation with get(obj)
> > > >       # Output a PNG plot and some text
> > > > }
> > >
> > > I have also tried with preschedule=TRUE, which has the same problem,
> > > though then a process or two will end early since it has been going on
> > > its own core instead of sharing. This is on Linux (Ubuntu 10.04 64-bit
> > > desktop edition), and I am running R from Emacs with ESS (maybe I
> should
> > > try from the terminal instead--hadn't thought to do that). Any thoughts
> > > that could help?
> > >
> > >
> > Assuming there is nothing else running on the system?
> >
> > In any case, it appears that you are reading text files and writing to
> disk.
> >  Only one core can use the IO at a time, so that may explain this.  Try
> > running something entirely computational and see if you can get close to
> > 100% on all four cores.
> >
> > Sean
> >
> >       [[alternative HTML version deleted]]
> >
> > _______________________________________________
> > R-sig-hpc mailing list
> > R-sig-hpc@r-project.org
> > https://stat.ethz.ch/mailman/listinfo/r-sig-hpc
>
> _______________________________________________
> R-sig-hpc mailing list
> R-sig-hpc@r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-hpc
>

	[[alternative HTML version deleted]]

