[R-SIG-Mac] Poor Performance of R for Mac as distributed on the developer's page
Yan Zhou
zhouyan at me.com
Mon Jan 10 18:25:19 CET 2011
Dear all,
I have been using the stable branch (i.e. 2.12-branch patched for now) downloaded from http://r.urbanek.info/ for quite a long time. The binary is compiled on Mac OS X 10.5 as said on the webpage. Just for curiousness I tried to build R on my laptop today and got some surprising result about the performance.
The benchmark is R-benchmark-25.R on http://r.urbanek.info/benchmarks/
For a complete log see the end of this email.
The laptop used for testing is a mid-2010 macbook pro 15", with core i7 2.66GHz, 8GB memory, Mac OS X 10.6
Two versions of R are tested, one is the binary downloaded from http://r.urbanek.info/, R 2.12.1 patched (2011-01-08).
The other is build from source. The source is checked out from SVN repository, 2.12-branch, revision 53948 (https://svn.r-project.org/R/branches/R-2-12-branch/). The building is almost the same as described on http://r.urbanek.info/building.html, except that gfortran is replaced by intel ifort and only x86_64 version is build rather than a universal binary. Blas and lapack are linked to vecLib (--with-blas='-framework vecLib' --with-lapack). The build passed "make check" successfully.
Comparing the benchmarks result. The binary distributed from http://r.urbanek.info/ is much slower than the one build from source for cross-product (10 times slower) and linear regression, determinant, Cholesky decomposition, inverse of matrix (roughly 5 times slower). Other things are almost the same or only slightly slower.
I thought all these tasks are performed by blas/lapack and both versions are linked to vecLib. So I cannot understand the difference in performance. I doubt that it has anything to do with the use of ifort. It would be highly appreciated if someone could explain this to me. In addition I think both R are dynamically linked to the veclib as installed on my mac, the fact the the binary distributed on web is build on mac os x 10.5 should have little to do with this. Correct me on this if I am wrong.
Now the complete log of the two benchmarks.
R as downloaded from http://r.urbanek.info/
=================================
R version 2.12.1 Patched (2011-01-08 r53945)
Copyright (C) 2011 The R Foundation for Statistical Computing
ISBN 3-900051-07-0
Platform: x86_64-apple-darwin9.8.0/x86_64 (64-bit)
# some text omitted
R Benchmark 2.5
===============
Number of times each test is run__________________________: 3
I. Matrix calculation
---------------------
Creation, transp., deformation of a 2500x2500 matrix (sec): 1.01866666666667
2400x2400 normal distributed random matrix ^1000____ (sec): 0.677
Sorting of 7,000,000 random values__________________ (sec): 0.870666666666667
2800x2800 cross-product matrix (b = a' * a)_________ (sec): 14.7716666666667
Linear regr. over a 3000x3000 matrix (c = a \ b')___ (sec): 6.79066666666667
--------------------------------------------
Trimmed geom. mean (2 extremes eliminated): 1.81941655450421
II. Matrix functions
--------------------
FFT over 2,400,000 random values____________________ (sec): 0.686666666666667
Eigenvalues of a 640x640 random matrix______________ (sec): 1.11166666666666
Determinant of a 2500x2500 random matrix____________ (sec): 5.63833333333333
Cholesky decomposition of a 3000x3000 matrix________ (sec): 4.45466666666667
Inverse of a 1600x1600 random matrix________________ (sec): 5.54233333333333
--------------------------------------------
Trimmed geom. mean (2 extremes eliminated): 3.01643621298111
III. Programmation
------------------
3,500,000 Fibonacci numbers calculation (vector calc)(sec): 0.73466666666667
Creation of a 3000x3000 Hilbert matrix (matrix calc) (sec): 0.51133333333334
Grand common divisors of 400,000 pairs (recursion)__ (sec): 0.974333333333334
Creation of a 500x500 Toeplitz matrix (loops)_______ (sec): 0.725666666666655
Escoufier's method on a 45x45 matrix (mixed)________ (sec): 0.543000000000006
--------------------------------------------
Trimmed geom. mean (2 extremes eliminated): 0.661519188946841
Total time for all 15 tests_________________________ (sec): 45.0513333333333
Overall mean (sum of I, II and III trimmed means/3)_ (sec): 1.53693763318728
--- End of test ---
==================================================================
R build from source
=================================
Yan-Zhous-MacBook-Pro:bin Zhou$ ./R
R version 2.12.1 Patched (2011-01-10 r53948)
Copyright (C) 2011 The R Foundation for Statistical Computing
ISBN 3-900051-07-0
Platform: x86_64-apple-darwin10.6.0 (64-bit)
# some text omitted
R Benchmark 2.5
===============
Number of times each test is run__________________________: 3
I. Matrix calculation
---------------------
Creation, transp., deformation of a 2500x2500 matrix (sec): 0.991666666666667
2400x2400 normal distributed random matrix ^1000____ (sec): 0.588
Sorting of 7,000,000 random values__________________ (sec): 0.867333333333336
2800x2800 cross-product matrix (b = a' * a)_________ (sec): 1.46866666666667
Linear regr. over a 3000x3000 matrix (c = a \ b')___ (sec): 0.953666666666668
--------------------------------------------
Trimmed geom. mean (2 extremes eliminated): 0.936086794519483
II. Matrix functions
--------------------
FFT over 2,400,000 random values____________________ (sec): 0.671666666666667
Eigenvalues of a 640x640 random matrix______________ (sec): 0.777333333333331
Determinant of a 2500x2500 random matrix____________ (sec): 0.94733333333333
Cholesky decomposition of a 3000x3000 matrix________ (sec): 0.835999999999999
Inverse of a 1600x1600 random matrix________________ (sec): 0.914666666666664
--------------------------------------------
Trimmed geom. mean (2 extremes eliminated): 0.840798910709667
III. Programmation
------------------
3,500,000 Fibonacci numbers calculation (vector calc)(sec): 0.502666666666665
Creation of a 3000x3000 Hilbert matrix (matrix calc) (sec): 0.511666666666665
Grand common divisors of 400,000 pairs (recursion)__ (sec): 0.928333333333332
Creation of a 500x500 Toeplitz matrix (loops)_______ (sec): 0.73733333333333
Escoufier's method on a 45x45 matrix (mixed)________ (sec): 0.468999999999994
--------------------------------------------
Trimmed geom. mean (2 extremes eliminated): 0.574526889787841
Total time for all 15 tests_________________________ (sec): 12.1653333333333
Overall mean (sum of I, II and III trimmed means/3)_ (sec): 0.767549169862442
--- End of test ---
Best regards
Yan Zhou
Department of Statistics
University of Warwick
More information about the R-SIG-Mac
mailing list