[R-SIG-Finance] Matlab vs. R performance/convenience benchmarking for quantitative analysis business
Dirk Eddelbuettel
edd at debian.org
Wed Oct 6 21:58:35 CEST 2010
On 6 October 2010 at 15:24, fjpcaballero at gmail.com wrote:
| I tested ATLAS and Goto on Linux 64 and I couldn't match the performance of
| Revolution-R. I guess credit shall go to Intel's compiler suite and Mkl,
| but that said,
I recently wrote a paper/package combination benchmarking different BLAS as
well as a GPU solution. Please see
http://dirk.eddelbuettel.com/blog/code/gcbd/
for two posts on this alongside with summary charts and links to the paper.
In short, Goto is pretty and about the same as MKL. The newest Atlas is also
pretty good when built multithreaded.
[ And the benchmark that Eric quotes looked surprisingly snake-oilish: wasn't
that merely single-threaded versus multi-threaded BLAS? You can easily do
get that by swapping BLAS libraries as my paper argues. Moreover, for tests
that do more realistic R work outside of core linear algebra, the effects
will be nowhere near as drastic -- Amdahl's Law and all that. ]
| I once tried to compile R using Intel's suite and I gave up out of boredom
| after a day trying to get compiler suite installing correctly. I can easily
I have been meaning to get back to testing Intel's 'icc' compiler but have
not gotten around to it as there are only so many hours in the day...
| imagine that budget-insensitive people may be more than willing to pay for
| the support.
Well, you do get multithreaded BLAS for free in several places, and easy
solutions for parallel computing (such as the excellent multicore) are also
easy to deploy.
As for the earlier Matlab comparison: I would second Tobias' points about
ease of deployment and lack of license hazzles, unsurpassed richness of CRAN
libraries, possibility of client/server computing (eg via Rapache by Jeff
Horner), and easy extensibility -- I am somewhat partial to Rcpp there.
Dirk
| On Oct 6, 2010, at 2:28 PM, Tobias Verbeke <tobias.verbeke at openanalytics.eu> wrote:
|
| > L.S.
| >
| > 1) For a good IDE one can use Eclipse with the StatET plug-ins
| > for R development (which, moreover, is cross-platform);
| >
| > 2) It is perfectly possible to use an optimized open source BLAS with
| > R such as Atlas, so there are many more Optimized R versions than the
| > Revolution marketing wants one to believe (and it is not at all
| > necessary to get trapped in closed source environments to have
| > equivalent tools).
| >
| > Best,
| > Tobias
| >
| > On 10/06/2010 08:02 PM, Eric Zivot wrote:
| >> Something in between free R and Matlab to consider is RevolutionR (I don't
| >> work for them, please don't flame me). It uses optimized math libraries, has
| >> a very nice IDE and is set-up for parallel computing out-of-the box. I don't
| >> know what their corporate pricing is but it has to be cheaper than Matlab.
| >> Below is a comparison (taken from the Revolution documentation) between the
| >> optimized math libraries and the standard R libraries for certain
| >> computations:
| >>
| >> Calculation Regular R Optimized
| >> libraries
| >> Eigen 11.45 3.63
| >> Svd 7.34 1.19
| >> Qr 1.17 0.93
| >> Lm 1.47 1.23
| >> Matrix mult 1.75 0.09
| >>
| >> Note: Mean times for various calculations below
| >>
| >> set.seed(14)
| >> x<- matrix(rnorm(1000000),nrow=1000)
| >> xout<- numeric(20)
| >> for (i in 1:20) xout[i]<- system.time(eigen(x))[3]
| >> xout2<- numeric(20)
| >> for (i in 1:20) xout2[i]<- system.time(svd(x))[3]
| >> xout3<- numeric(20)
| >> for (i in 1:20) xout3[i]<- system.time(qr(x))[3]
| >> xout4<- numeric(20)
| >> for (i in 1:20) xout4[i]<- system.time(lm(x[,i]~x[,-i]))[3]
| >> xout5<- numeric(20)
| >> for (i in 1:20) xout5[i]<- system.time(t(x)%*%x)[3]
| >>
| >>
| >>
| >> Eric Zivot
| >> Robert Richards Chaired Professor of Economics
| >> Adjunct Professor of Finance
| >> Adjunct Professor of Statistics
| >> Department of Economics
| >> Box 353330 email: ezivot at u.washington.edu
| >> University of Washington phone: 206-543-6715
| >> Seattle, WA 98195-3330
| >> www: http://faculty.washington.edu/ezivot
| >>
| >>
| >> -----Original Message-----
| >> From: r-sig-finance-bounces at stat.math.ethz.ch
| >> [mailto:r-sig-finance-bounces at stat.math.ethz.ch] On Behalf Of
| >> fjpcaballero at gmail.com
| >> Sent: Wednesday, October 06, 2010 9:38 AM
| >> To:<Samuel.Meichtry at bkw-fmb.ch>
| >> Cc:<r-sig-finance at stat.math.ethz.ch>
| >> Subject: Re: [R-SIG-Finance] Matlab vs. R performance/convenience
| >> benchmarking for quantitative analysis business
| >>
| >> Note that this benchmark appears to be using non-optimized BLAS libraries.
| >> Speed of matrix operations should be comparable to Matlab when using ATLAS,
| >> Intel mkl, vecLib, or Goto. Loops are a different story. Even with Ra and
| >> JIT, R is a far cry from Matlab.
| >>
| >> My personal take:
| >>
| >> Pro Matlab: compact and clean syntax, good IDE and tooling, more efficient
| >> (e.g. pass by reference, loop optimization), some very good toolboxes not
| >> available elsewhere (eg CVX, Murphy's BBN toolbox)
| >>
| >> Against Matlab: it is a pain to work with anything that doesn't look like a
| >> matrix; lags behind R in methods available, licensing costs
| >>
| >> Pro R: (much) broader selection of methods -- at least in machine learning,
| >> survival analysis, and finance; named arguments in functions, easier to work
| >> with mixed data (eg dataframes), better statistics-oriented graphics (e.g.
| >> Lattice, ggplot), better interactive graphics [for my taste] (rggobi,
| >> iplots), licensing costs
| >>
| >> Against R: performance issues previously discussed, pretty ugly code, lack
| >> of a good IDE -- all the ones I have tested lack an integrated debugger and
| >> profiler, steep learning curve.
| >>
| >>
| >>
| >> On Oct 6, 2010, at 9:08 AM,<Samuel.Meichtry at bkw-fmb.ch> wrote:
| >>
| >>> Hello everyone,
| >>>
| >>> At the moment we are trying to decide which software to use for
| >> quantitative analysis and the following Link about performance doesn't
| >> really vote for R (http://mlg.eng.cam.ac.uk/dave/rmbenchmark.php).
| >>>
| >>> I am not against Matlab and cannot say anything about its convenience
| >> because I haven't worked with it for a long time. On the other hand Matlab
| >> is already heavily used in other departments in our company and we have to
| >> tell our project committee why we would like to use R instead.
| >>>
| >>> Now I am very interested to know why you are using R in your business and
| >> not Matlab?
| >>> Or do you use both of them?
| >>>
| >>> Thank you very much for your feedback!
| >>>
| >>>
| >>> Kind regards,
| >>> Samuel Meichtry
| >>>
| >>>
| >>> ____________________________________________
| >>> BKW FMB Energie AG
| >>> Energy Trading
| >>> Samuel Meichtry
| >>> Analyst Energy Trading
| >>> Tel +41 31 330 53 99
| >>> Fax +41 31 330 56 16
| >>> e-mail samuel.meichtry at bkw-fmb.ch
| >>>
| >>> * Sie drucken dieses E-Mail nicht aus? Die Umwelt dankt!
| >>>
| >>>
| >>>
| >>>
| >>> [[alternative HTML version deleted]]
| >>>
| >>> _______________________________________________
| >>> R-SIG-Finance at stat.math.ethz.ch mailing list
| >>> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
| >>> -- Subscriber-posting only. If you want to post, subscribe first.
| >>> -- Also note that this is not the r-help list where general R questions
| >> should go.
| >>
| >> _______________________________________________
| >> R-SIG-Finance at stat.math.ethz.ch mailing list
| >> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
| >> -- Subscriber-posting only. If you want to post, subscribe first.
| >> -- Also note that this is not the r-help list where general R questions
| >> should go.
| >>
| >> _______________________________________________
| >> R-SIG-Finance at stat.math.ethz.ch mailing list
| >> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
| >> -- Subscriber-posting only. If you want to post, subscribe first.
| >> -- Also note that this is not the r-help list where general R questions should go.
| >>
| >
|
| _______________________________________________
| R-SIG-Finance at stat.math.ethz.ch mailing list
| https://stat.ethz.ch/mailman/listinfo/r-sig-finance
| -- Subscriber-posting only. If you want to post, subscribe first.
| -- Also note that this is not the r-help list where general R questions should go.
--
Dirk Eddelbuettel | edd at debian.org | http://dirk.eddelbuettel.com
More information about the R-SIG-Finance
mailing list