[R-SIG-Mac] Multiple cores on a Mac

[Joseph Kunkel]

Wow, thanks guys.  I am back to original speed with matrix operations as documented here.  As noted previously, this includes the 4 fold basic slowing of matrix operations on a single processor that occurred at the time of the change ~2012(?).  So I have approx 80/5 = 16 fold improvement!!!   I can do experiments and do a new experiment tomorrow based on the calculation!!!

Tests applied today II_5_2016:

*Test without vecLib option
 > source("http://www.bio.umass.edu/biology/kunkel/pub/R/CuriousResult.R")
 matrix multiplication      80.516   1.338   81.22 
 tcrossprod                 79.999   1.366  80.734 
 transposition and reuse    14.303   2.323   16.52 
 elementwise after reshape  11.088   1.776  12.774 
 columnwise sapply    22.781   9.631  32.137 
 for loop over columns      23.976   7.811  31.507 
 >q()
*apply unix adjustment # for vecLib use
> source("http://www.bio.umass.edu/biology/kunkel/pub/R/CuriousResult.R")
 matrix multiplication      19.595   1.188   5.389 
 tcrossprod                     19   1.175   4.702 
 transposition and reuse    14.113   2.416  16.432 
 elementwise after reshape  11.102   1.764  12.798 
 columnwise sapply    21.747   9.125  30.683 
 for loop over columns      23.243   6.458  29.515 

I have renewed faith!
Joe

> On Feb 5, 2016, at 10:20 AM, Bryan Hanson <hanson at depauw.edu> wrote:
> 
> On El Cap you can still pull off the symlink approach, as described here, without building R fresh:
> 
> http://blog.quadrivio.com/2015/06/improved-r-performance-with-openblas.html
> 
> The next time you launch R you will have the library you specified.
> 
> Bryan
> 
>> On Feb 5, 2016, at 10:03 AM, peter dalgaard <pdalgd at gmail.com> wrote:
>> 
>> Why me..?
>> 
>> Probably Simon and maybe Brian has the full recollection of the story, but as I understood it, once upon a time you could switch out the BLAS on CRAN R just by editing a symlink in the R installation. For some reason, that cannot work anymore (Apple considered symlinked libraries a security risk?). I think the current situation is that you can still do it, but you have to physically overwrite the default BLAS(? Simon will know.). 
>> 
>> At any rate, you can certainly still do a local compile with the Accelerate framework, once you get all the tools in place:
>> 
>>> M <- matrix(rnorm(1e8),10000)
>>> system.time(M %*% t(M))
>>  user  system elapsed 
>> 219.566   0.806  21.752 
>>> M <- crossprod(M)
>>> system.time(solve(M))
>>  user  system elapsed 
>> 310.874   1.477  29.998 
>> 
>> (To tell the truth, I actually don't have all the tools in place on that machine, so this was from a build of 3.2.1 patched)
>> 
>> -pd
>> 
>> On 05 Feb 2016, at 14:52 , Joseph Kunkel <joe at bio.umass.edu> wrote:
>> 
>>> To me there are big gorillas in the room and I need to know why I can not use them all.
>>> 
>>> • Testing for physical and logical cpus on Joe's MacBook Pro.
>>> Josephs-MacBook-Pro:~ josephgkunkel$ /usr/sbin/sysctl -n hw.physicalcpu
>>> 4
>>> Josephs-MacBook-Pro:~ josephgkunkel$ /usr/sbin/sysctl -n hw.logicalcpu
>>> 8
>>> 
>>> Prior to about 2012 my multicore Macs would use all (physical) cores automagically in R.  %*% was multicore automatically.
>>> 
>>> A 24 hour heavy matrix calculation would take a little over 6 hours on a 4 core Mac.
>>> 
>>> Then some problem with the BLAS library changed everything and colleague stats people and I got really mad that we could not do our calculations as fast in R.
>>> 
>>> Many work-around libraries were devised which did not seem to be that useful for freewielding matrix operations.
>>> 
>>> Then Revolution R seemed to solve the problem and patented(?) it.  … but not for Macs.
>>> 
>>> Recently they provided a free Mac version but using their R ‘open' version messes up my computer for updating the libraries I am addicted to using.
>>> 
>>> My question after this appologeticlay long narrative is:
>>> 
>>> Why has no satisfactory and transparent method for multicore use been available to CRAN R?
>>> 
>>> Secondarily,  how could our R open software system be hijacked by Revolution and now Microsoft?
>>> 
>>> I would be pleased to know that there are colleagues out there who are similarly hoping for an R core solution within CRAN.
>>> 
>>> I can do primitive matrix things faster with Julia, which is encouraging, but the libraries and flexability for me are not there yet.
>>> 
>>> Joe
>>> 
>>> -·.  .· ·.  .><((((º>·.  .· ·.  .><((((º>·.  .· ·.  .><((((º> .··.· >=-       =º}}}}}><
>>> Joseph G. Kunkel, Research Professor
>>> 112A Marine Science Center
>>> University of New England
>>> Biddeford ME 04005
>>> http://www.bio.umass.edu/biology/kunkel/
>>> 
>>> 
>>> 
>> 
>> -- 
>> Peter Dalgaard, Professor,
>> Center for Statistics, Copenhagen Business School
>> Solbjerg Plads 3, 2000 Frederiksberg, Denmark
>> Phone: (+45)38153501
>> Office: A 4.23
>> Email: pd.mes at cbs.dk  Priv: PDalgd at gmail.com
>> 
>> _______________________________________________
>> R-SIG-Mac mailing list
>> R-SIG-Mac at r-project.org
>> https://stat.ethz.ch/mailman/listinfo/r-sig-mac
>