[R] Performance degradation on multi-processor system

[Rajiv Garg]

Hi,

We are seeing performance degradation when running the same R script in
multiple instances of R on a multi-processor system.  We are a bit surprised
by this because we figured that each instance of R is running in its own
processor, and therefore running a second, third or fourth instance should
not affect the performance of the first instance.

Here's a test script that exhibits this behavior:

  library("nnls")  #load library

  set.seed(seed=100) #set random number generator  seeds
  X.mat=matrix(runif(2000*500,0,1),nrow=2000,ncol=500)
  y.vec=runif(2000,0,1)
  X.mat[1:1000,5]=X.mat[1:1000,5]+4*y.vec[1:1000]
  X.mat[1001:2000,15]=X.mat[1001:2000,15]+4*y.vec[1001:2000]

  for (j in 1:100)
  {
    for (i in 1:500) tmp.model=nnls(X.mat,y.vec)  #the inner loops (i from 1
to 500) takes about 30 seconds
    print(j)
    print(date())
  }

This script has two loops -- the inner loop outputs a timestamp each time it
finishes.  If it is running by itself, it takes 30 seconds for each inner
loop on our server.  If we run multiple instances in parallel, the time
dramatically increases.  We have run up to 6 of them in parallel, and got
the following results:

No of Scripts                     Time for Inner Loop to Execute in sec
          1                                30
          2                                34
          3                                48
          4                                51
          5                                3 @ 55 and 2 @ 70
          6                                3 @ 60 and 3 @ 90

The server is a Red Hat Enterprise Linux 5.2 server, running Linux kernel
2.6.18.-128.  It has 2 quad-core Xeon E5420 processors with 6 MB of L2 cache
for each processor.  The server has 32 GB of RAM.  While the tests were run,
the memory utilization was very low, and there was not much I/O.  The CPU
utlization was 100% for each core running an instance of R.

Our theory is that the degradation is due to cache and/or memory bus
contention between the processors.  We are going to test this using oprofile
to monitor for L2 cache misses, and possibly memory latency, but before
doing so wanted to run this by this list to see if anyone else has seen
similar behavior, or can shed some light on the possible cause.

Any comments are appreciated.

Rajiv

 

-- 
View this message in context: http://www.nabble.com/Performance-degradation-on-multi-processor-system-tp24356764p24356764.html
Sent from the R help mailing list archive at Nabble.com.