[R-SIG-Mac] Nehalem performance [Was: Is R more heavy on memory or processor?]

[Max Kuhn]

I've been doing consistent time benchmarks across a range of computers
(no Nehalems yet). A few of these are OS X related and might be of
interest.

The test systems:

A. MacPro: Dual Quad core 2.8 GHz, 32 GB RAM. Running either OS X
10.5.6, 10.6 or Slackware 12.2
B. MacBook Pro:  Intel Core 2 Duo 2.4 GHz, 4 GB RAM OS X 10.5.6

All using R 2.9.0 with at least 3 replicates per benchmark.

The benchmarks were:

1. RMA: RMA normalize 57 hgu133plus2 microarrays. The time quoted is
just to normalize (not to read the files in).

The summary in minutes:

                                    group mean   sd
              MacBook Pro 2.4 GHz Leopard 1.02 0.01
 MacBook Pro 2.4 GHz Leopard Snow Leopard 1.09 0.35
                     MacPro 2.8 GHz Linux 1.26 0.06
                   MacPro 2.8 GHz Leopard 0.89 0.13

There was one outlier with Snow Leopard. Without it:

                                    group mean   sd
              MacBook Pro 2.4 GHz Leopard 1.02 0.01
 MacBook Pro 2.4 GHz Leopard Snow Leopard 0.95 0.01
                     MacPro 2.8 GHz Linux 1.26 0.06
                   MacPro 2.8 GHz Leopard 0.89 0.13

2 .Kernels: fit a support vector machine and a relevance vector
machine to 1,500 samples with 1,000 variables. Times for each model
are separated out.

For ksvm:

                                    group mean   sd
              MacBook Pro 2.4 GHz Leopard 0.22 0.01
 MacBook Pro 2.4 GHz Leopard Snow Leopard 0.19 0.01
                     MacPro 2.8 GHz Linux 0.27 0.01
                   MacPro 2.8 GHz Leopard 0.26 0.05


For rvm:

                                    group mean   sd
              MacBook Pro 2.4 GHz Leopard 0.71 0.01
 MacBook Pro 2.4 GHz Leopard Snow Leopard 0.60 0.01
                     MacPro 2.8 GHz Linux 2.24 0.53
                   MacPro 2.8 GHz Leopard 0.46 0.05

I have no idea what happened with the Linux runs.

3. RandomForest: fit a randomForest model with 1,500 samples with
1,000 variables with the 5,000 trees and mtry = 100. The timings also
include the variable importance calculations.

The rows with >1 node was using a parallel randomforest via the nws package:

                                    group  mean   sd
              MacBook Pro 2.4 GHz Leopard 22.31 0.15
    MacBook Pro 2.4 GHz Leopard (2 nodes) 12.72 0.27
    MacBook Pro 2.4 GHz Leopard (5 nodes) 13.51   NA
 MacBook Pro 2.4 GHz Leopard Snow Leopard 21.47 0.81
                     MacPro 2.8 GHz Linux 17.28 0.05
           MacPro 2.8 GHz Linux (5 nodes)  4.61 0.26
                   MacPro 2.8 GHz Leopard 17.42 0.07
         MacPro 2.8 GHz Leopard (5 nodes)  3.87 0.03

So, basically, Snow Leopard does slightly better but not much (I think
this was expected). I'll be testing more on the Mac Pros soon. The
code is pretty basic, but I'll be happy to send it to anyone who wants
a head-on comparison.

Max