[R] Benchmark

[Yoon, Hoon (CICG - NY Program Trading)]

Prof,

  The reason I rewrote the code was to compare it to work already done by
Stefan. 
  You are right, given that 0.63.0 NT version of R's status and many things
I have running. The results should NOT be taken literally (I think I missed
NOT part in the earlier letter) and interpreted with much caution.
  It also happens that no one can agree on any benchmark. This code just
happened to be around and much work has been done already. I have seen many
benchmark studies. I still have not seen one that people could agree on. Not
to start a flame war, but MS claims that VB is as fast as C++. Inprise
claims that Delphi is faster than VB. Java camp claims 1/2 the speed of C++,
when compiled native. VB claims that Java is slower by factor of teens. 
  Everyone has a benchmark to prove their point. Virtually any benchmark can
be stacked for one lang or another. I can show that Numeric Python is fast
as C, while Perl can show that it can cut files less than 1/2 the speed on
Python by using some tricks.
  Only thing that really matters is a "how fast would my application run."
Well, we can NOT write same application over and over again in every
possible lang choices. We have to take some sample measures with "Grain of
salt."

  I will probably say that "look at ANY benchmark with great skeptism."
Still, they can be useful in some narrow contexts.  

  The reason I provided full source is so any one can try this and modify it
without doing this work over again. Especially with my machine running 10
things at once. I would expect others to try it on their machine and
different platforms.
  Given that I have been using R for less than a week, I needed some guiding
post and wanted to share what I had.

>From http://www.informatik.uni-frankfurt.de/~stst/ncrunch.html#Speed
7. Speed comparison
The following speed comparison have been performed on a PENTIUM PC with 90
MHz and 96 MB RAM running under
Windows NT 4.0 with Windows versions of the programs (all timings are
displayed in seconds)
                                                  Gauss    Mac-syma Maple
Mathmatica Matlab MuPad OMtrx  Ox     Splus
400 x 400 random matrix^1000     16.476     136.747     > 1h      30.023
5.468  > 1h     0.400  1.011   1.260

On NT machines Splus ranks as close third and much faster than R for
generating Random Matrix. Comparatively speaking, it seems a Rocket compared
to many that takes over an hour and even compared to Gauss (which I have
been using for last 5 years).


	>Note that, having tried it, I do NOT believe `if I add something
	>complicated or/and loops that Splus will slow down to crawl'. Your
	>prejudices are showing!

I think no one will doubt that Splus is about the one of the worst languages
to do loops in. Back in Salomon Bros (before "Citi Traveling Barney
Brothers"), we used to joke about S in Splus really stands for Slow.
Everytime I have to anything remotely like a loop, I just abandoned Splus.
The reason I said that R might be faster on loops is the impression I gained
from various sources including CRAN and this e-mail list. Why just other
day:

>Problems which require lots of looping and cannot be re-coded as
vector/matrix
>operations are theoretically possible but practically infeasible in S,
unless
>they  are coded in C or Fortran and called from S. These problems seem to
run fine in R.
>
>Paul Gilbert

I really hope that Paul is right about this. I guess this was more a case of
"wishful thinking" than "prejudices." Acually, given that my speed results
seemed quite close to each other, I wanted to make sure that others know
that I am not contradicting what they have already stated.

I would like to see other people run this test on their machines and report
back. I also would like to see additional tests and make a better comparison
possible. Although I do not think there is real test that can be performed
that shows absolute advantage, we can look at the test and pick out the most
important feature for our particular need.

BTW: Given the nature of the trading floor, sometimes I have to write these
letters very quickly. Many times, errors will be creeping in (like not
adding NOT to: "I had many things running on the computer while trying to do
this, so I am ^^^
very confident about results" on earlier letter). My apologies in advance.

**************************************************************
S. Hoon Yoon                   (Quant)                    Merrill Lynch
Equity Trading 
yelled at yahoo.com hoon at bigfoot.com(w)
"Miracle is always only few standard deviations away, but so is
catastrophe."
* Expressed opinions are often my own, but NOT my employer's.
"I feel like a fugitive from the law of averages."    Maudin
**************************************************************

-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
Send "info", "help", or "[un]subscribe"
(in the "body", not the subject !)  To: r-help-request at stat.math.ethz.ch
_._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._