[R] [ExternalEmail] Pearson Correlation Speed

[Nathan S. Watson-Haigh]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Nathan S. Watson-Haigh wrote:
> I'm trying to calculate Pearson correlation coefficients for a large
> matrix of size 18563 x 18563. The following function takes about XX
> minutes to complete, and I'd like to do this calculation about 15 times
> and so speed is some what of an issue.

Sorry, meant to fill in the blanks!

the following takes about 15 mins to complete:
corr <- abs(cor(dat, use="p"))

> 
> Does anyone have any suggestions on ways to speed this up? I'd wondered
> if using C++ code to do the calculations might speed things up, but I've
> never written any C/C++ code or attempted to use any within R.
> 
> I've seen some C++ code here:
> http://www.alglib.net/statistics/correlation.php
> 
> I wondered if anyone might be able to help me get this so it can run in
> R? I've tried the following:
> 1) download and unzipped
> http://www.alglib.net/translator/dl/statistics.correlation.cpp.zip
> 2) moved the contents of the libs dir into the parent dir alongside
> correlation.cpp (didn't know how to tell R where to look for C libraries)
> 3) Tried: "R CMD SHLIB correlation.cpp" and got the following as output:
> -- start output --
> icpc -I/tools/R/2.7.1/lib/R/include  -I/usr/local/include  -mp  -fpic
> -g -O2 -c correlation.cpp -o correlation.o
> ap.h(163): warning #858: type qualifier on return type is meaningless
>   const bool operator==(const complex& lhs, const complex& rhs);
>              ^
> 
> ap.h(164): warning #858: type qualifier on return type is meaningless
>   const bool operator!=(const complex& lhs, const complex& rhs);
>              ^
> 
> ap.h(179): warning #858: type qualifier on return type is meaningless
>   const double abscomplex(const complex &z);
>                ^
> 
> icpc -shared -L/usr/local/lib -o correlation.so correlation.o
> -- end output --
> 4) Now this doesn't look brilliant! Any thoughts? Also, I'm assuming I
> need to do some other work with the C++ code in order to allow me to use
> it from within my R scripts - any pointers on that?
> 
> Thanks for any input - I hope I just need a hand over the initial
> hurdles and then I can get onto that up-hill learning curve!!
> 
> Nathan
> 
> 

______________________________________________
R-help at r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.


- --
- --------------------------------------------------------
Dr. Nathan S. Watson-Haigh
OCE Post Doctoral Fellow
CSIRO Livestock Industries
Queensland Bioscience Precinct
St Lucia, QLD 4067
Australia

Tel: +61 (0)7 3214 2922
Fax: +61 (0)7 3214 2900
Web: http://www.csiro.au/people/Nathan.Watson-Haigh.html
- --------------------------------------------------------

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAklF8xAACgkQ9gTv6QYzVL4+wwCeL3jZVYi1VsCIQG/FQYpvcUPi
XCwAoKGAImMBJOLSOBELchL+LpKDnlTT
=LIiy
-----END PGP SIGNATURE-----