[R-SIG-Mac] Multi-thread R processes performance on 8-core Mac-Pro

[Simon Urbanek]

On Apr 18, 2007, at 8:28 AM, elw at stderr.org wrote:

>
>> I am running R 2.4.1 on the new 8-core Mac Pro with the parSapply
>> function from the Snow package. Tests using 2,4, and 8 threads with
>> makeCluster() yield somewhat disappointing results. The 4 thread  
>> process
>> is fastest. With 8 threads, all the cores max out at about 70% power,
>> and even then it is slower than the 4 thread process which maxes  
>> out the
>> 4 threads at about 90-95%. This suggests the additional 4 cores on  
>> the
>> Mac Pro do not improve performance in an embarrassingly parallel R/ 
>> Snow
>> environment...
>
>> The function I tested in parSapply runs a regression (a call to  
>> "lm"). I
>> am using Snow/rpvm package combination. There is no issue of limited
>> memory. There are over 3 gigs of free RAM in the tests.
>
>
> There has been some discussion (elsewhere) of the mac pro  
> motherboard's inability to keep all eight cores supplied with data  
> from memory at an adequate rate.
>
> Your processor cores may simply be starved for data.  The amount of  
> memory available doesn't matter, much, if there's no IO bandwidth  
> left between the CPUs and the DIMMs....................
>

The Intel's SMP design is poor compared to its competitors (not just  
the MoBo, the CPUs as well), that is a well known fact, but I  
wouldn't be so sure that it is what hits you. Can you try to run some  
some big BLAS  operations like for example:

set.seed(1)
a<-matrix(rnorm(4000000),2000)
b<-matrix(rnorm(4000000),2000)
system.time(for (i in 1:20) a%*%b)

This takes ca 14.2s on a 2.66 quad Mac Pro (46.5s user time). I  
wonder what the 8-core does with this. If you can't feed all 8 cores  
then I'd say it's likely a bandwidth issue ...

Cheers,
S