[R-SIG-Finance] Matlab vs. R performance/convenience benchmarking for quantitative analysis business

Tobias Verbeke tobias.verbeke at openanalytics.eu
Wed Oct 6 21:52:02 CEST 2010


Hi,

I agree with most of your comments, but

1) my point is certainly not against paying for commercial
    services (I manage a consulting firm, by the way), but
    against losing the benefits open source and cross-platform
    software has in commercial environments;

2) according to the benchmarks of Dirk Eddelbuettel (slides 46
    to 49 of)

    http://dirk.eddelbuettel.com/papers/useR2010hpcTutorial.pdf

    differences between Goto and Intel seem to be really small

3) even if Atlas or Goto would not match the performance of
    the Intel BLAS, linear algebra routines are just a part
    of statistical routines and not always _the_ performance
    bottleneck in real life scenarios. Requests to speed up
    R code have so far only been a tiny, tiny part of requests
    we got.

Just wanted to put the marketing in perspective.

Best,
Tobias


On 10/06/2010 09:24 PM, fjpcaballero at gmail.com wrote:
> Eclipse+StatEt is probably the best one, but unfortunately it is still missing a debugger, profiler, and even object inspection inside functions.
>
> I tested ATLAS and Goto on Linux 64 and I couldn't match the performance of Revolution-R.  I guess credit shall go to Intel's compiler suite and Mkl, but that said, I once tried to compile R using Intel's suite and I gave up out of boredom after a day trying to get compiler suite installing correctly. I can easily imagine that budget-insensitive people may be more than willing to pay for the support.
>
>
>
> On Oct 6, 2010, at 2:28 PM, Tobias Verbeke<tobias.verbeke at openanalytics.eu>  wrote:
>
>> L.S.
>>
>> 1) For a good IDE one can use Eclipse with the StatET plug-ins
>> for R development (which, moreover, is cross-platform);
>>
>> 2) It is perfectly possible to use an optimized open source BLAS with
>> R such as Atlas, so there are many more Optimized R versions than the
>> Revolution marketing wants one to believe (and it is not at all
>> necessary to get trapped in closed source environments to have
>> equivalent tools).
>>
>> Best,
>> Tobias
>>
>> On 10/06/2010 08:02 PM, Eric Zivot wrote:
>>> Something in between free R and Matlab to consider is RevolutionR (I don't
>>> work for them, please don't flame me). It uses optimized math libraries, has
>>> a very nice IDE and is set-up for parallel computing out-of-the box. I don't
>>> know what their corporate pricing is but it has to be cheaper than Matlab.
>>> Below is a comparison (taken from the Revolution documentation) between the
>>> optimized math libraries and the standard R libraries for certain
>>> computations:
>>>
>>> Calculation            Regular R            Optimized
>>> libraries
>>> Eigen                11.45                3.63
>>> Svd                7.34                1.19
>>> Qr                1.17                0.93
>>> Lm                1.47                1.23
>>> Matrix mult            1.75                0.09
>>>
>>> Note: Mean times for various calculations below
>>>
>>> set.seed(14)
>>> x<- matrix(rnorm(1000000),nrow=1000)
>>> xout<- numeric(20)
>>> for (i in 1:20) xout[i]<- system.time(eigen(x))[3]
>>> xout2<- numeric(20)
>>> for (i in 1:20) xout2[i]<- system.time(svd(x))[3]
>>> xout3<- numeric(20)
>>> for (i in 1:20) xout3[i]<- system.time(qr(x))[3]
>>> xout4<- numeric(20)
>>> for (i in 1:20) xout4[i]<- system.time(lm(x[,i]~x[,-i]))[3]
>>> xout5<- numeric(20)
>>> for (i in 1:20) xout5[i]<- system.time(t(x)%*%x)[3]
>>>
>>>
>>>
>>> Eric Zivot
>>> Robert Richards Chaired Professor of Economics
>>> Adjunct Professor of Finance
>>> Adjunct Professor of Statistics
>>> Department of Economics
>>> Box 353330                  email:  ezivot at u.washington.edu
>>> University of Washington    phone:  206-543-6715
>>> Seattle, WA 98195-3330
>>> www:  http://faculty.washington.edu/ezivot
>>>
>>>
>>> -----Original Message-----
>>> From: r-sig-finance-bounces at stat.math.ethz.ch
>>> [mailto:r-sig-finance-bounces at stat.math.ethz.ch] On Behalf Of
>>> fjpcaballero at gmail.com
>>> Sent: Wednesday, October 06, 2010 9:38 AM
>>> To:<Samuel.Meichtry at bkw-fmb.ch>
>>> Cc:<r-sig-finance at stat.math.ethz.ch>
>>> Subject: Re: [R-SIG-Finance] Matlab vs. R performance/convenience
>>> benchmarking for quantitative analysis business
>>>
>>> Note that this benchmark appears to be using non-optimized BLAS libraries.
>>> Speed of matrix operations should be comparable to Matlab when using ATLAS,
>>> Intel mkl, vecLib, or Goto.  Loops are a different story.  Even with Ra and
>>> JIT, R is a far cry from Matlab.
>>>
>>> My personal take:
>>>
>>> Pro Matlab: compact and clean syntax, good IDE and tooling, more efficient
>>> (e.g. pass by reference, loop optimization), some very good toolboxes not
>>> available elsewhere (eg CVX, Murphy's BBN toolbox)
>>>
>>> Against Matlab: it is a pain to work with anything that doesn't look like a
>>> matrix; lags behind R in methods available, licensing costs
>>>
>>> Pro R: (much) broader selection of methods -- at least in machine learning,
>>> survival analysis, and finance; named arguments in functions, easier to work
>>> with mixed data (eg dataframes), better statistics-oriented graphics (e.g.
>>> Lattice, ggplot), better interactive graphics [for my taste] (rggobi,
>>> iplots), licensing costs
>>>
>>> Against R: performance issues previously discussed, pretty ugly code, lack
>>> of a good IDE -- all the ones I have tested lack an integrated debugger and
>>> profiler, steep learning curve.
>>>
>>>
>>>
>>> On Oct 6, 2010, at 9:08 AM,<Samuel.Meichtry at bkw-fmb.ch>   wrote:
>>>
>>>> Hello everyone,
>>>>
>>>> At the moment we are trying to decide which software to use for
>>> quantitative analysis and the following Link about performance doesn't
>>> really vote for R (http://mlg.eng.cam.ac.uk/dave/rmbenchmark.php).
>>>>
>>>> I am not against Matlab and cannot say anything about its convenience
>>> because I haven't worked with it for a long time. On the other hand Matlab
>>> is already heavily used in other departments in our company and we have to
>>> tell our project committee why we would like to use R instead.
>>>>
>>>> Now I am very interested to know why you are using R in your business and
>>> not Matlab?
>>>> Or do you use both of them?
>>>>
>>>> Thank you very much for your feedback!
>>>>
>>>>
>>>> Kind regards,
>>>> Samuel Meichtry
>>>>
>>>>
>>>> ____________________________________________
>>>> BKW FMB Energie AG
>>>> Energy Trading
>>>> Samuel Meichtry
>>>> Analyst Energy Trading
>>>> Tel +41 31 330 53 99
>>>> Fax +41 31 330 56 16
>>>> e-mail samuel.meichtry at bkw-fmb.ch
>>>>
>>>> *  Sie drucken dieses E-Mail nicht aus? Die Umwelt dankt!
>>>>
>>>>
>>>>
>>>>
>>>>     [[alternative HTML version deleted]]
>>>>
>>>> _______________________________________________
>>>> R-SIG-Finance at stat.math.ethz.ch mailing list
>>>> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
>>>> -- Subscriber-posting only. If you want to post, subscribe first.
>>>> -- Also note that this is not the r-help list where general R questions
>>> should go.
>>>
>>> _______________________________________________
>>> R-SIG-Finance at stat.math.ethz.ch mailing list
>>> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
>>> -- Subscriber-posting only. If you want to post, subscribe first.
>>> -- Also note that this is not the r-help list where general R questions
>>> should go.
>>>
>>> _______________________________________________
>>> R-SIG-Finance at stat.math.ethz.ch mailing list
>>> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
>>> -- Subscriber-posting only. If you want to post, subscribe first.
>>> -- Also note that this is not the r-help list where general R questions should go.
>>>
>>
>



More information about the R-SIG-Finance mailing list