[R-SIG-Mac] using optimized BLAS [Was: problem installing rjags package]
Michael Braun
braunm at mit.edu
Wed Oct 13 16:25:49 CEST 2010
Simon:
Thanks for sending this over. I'd much rather stick with the CRAN binary if I can, but exploiting the capabilities of my machine as much as possible is important to me. There are 3 reasons I'm not using the precompiled Mac binary:
1. Linking to MKL Blas
2. Wanted to have a version of R where the linker lets me profile my precompiled code (you and I corresponded in a thread about this last year).
3. Compiling R for the newest Nehalem, and the vectorizing operations, using the -msse4.2 flag.
I've still never been able to get #2 to work, and it's really not a huge priority at the moment. #3 would be nice if it does let R milk all the power out of the new processor (I've never done a speed test on it). But if I understand correctly, would I now be able to use the CRAN R binary and still choose my own BLAS? Whether I choose MKL or ATLAS, I'm sorry that I don't really understand what you are talking about when you say :
"MKL is picking them at run time so you don't have to point it to the machine-specific libraries. You can simply create one library (ld -dylib) re-exporting exactly what you would have done when linking it in R (i.e. use -reexport-l instead of -l)."
I didn't mean pointing to the machine-specific libraries. In the MKL Users Manual (July 2010 version, Chapter 5), there are different options depending on if you want to link static or dynamic, sequential or threaded, ILP or LP, FORTRAN interface or not, etc. That;s why my Makevars file looks like: BLAS_LIBS=-lmkl_intel_lp64 -lmkl_sequential -lmkl_core -lpthread (and incidentally, adding -lpthread is mentioned in the Intel manual, even for the sequential version.)
Since I'm learning this all as I go, could you possibly provide a cookbook recipe of exactly what I would need to do. I'm just not sure what I should be re-exporting. Is this a new script file? Or something I need to compile?
Another issue I've considered when choosing the MKL BLAS: sometimes I'm calling BLAS routines from sequential code, in which case the threaded BLAS makes sense. But sometimes I call from threaded code, either using the plyr, doMC, foreach, and /or multicore R packages, or using OpenMP. In those cases, I've been setting MKL_NUM_THREADS (separately from OMP_NUM_THREADS) as needed. I have not found a comparable way to do this in the ATLAS manual.
Thanks,
Michael
On Oct 12, 2010, at 11:44 AM, Simon Urbanek wrote:
>
> On Oct 12, 2010, at 11:19 AM, Michael Braun wrote:
>
>> Thanks, Simon:
>>
>> I do have a Nehalem Mac Pro, and I've found MKL to be much faster than vecLib.
>
> Yes, because vecLib has a serious bug and doesn't spawn threads on Nehalem, but ATLAS is faster than MKL on Nehalem Mac Pro -- here are the tcrossprod timings (on 2.66GHz Mac Pro):
>
> vecLib 6.43
> ATLAS serial 4.80
> MKL serial 4.30
> MKL parallel 0.90
> ATLAS parallel 0.71
>
>
>> I saw in one of the manuals that one could point R to another BLAS. But I wasn't sure what to do with MKL because of their "layering" approach to linking. So if I point libRblas to something else, how to I tell the link which of the multiple libraries I need?
>>
>
> MKL is picking them at run time so you don't have to point it to the machine-specific libraries. You can simply create one library (ld -dylib) re-exporting exactly what you would have done when linking it in R (i.e. use -reexport-l instead of -l).
>
> For ATLAS it's much easier since you can compile it statically with PIC and create one library -- e.g. see
> http://r.research.att.com/libatlas-MacPro.dylib
> (Note that one is 10.6 only but includes both 32-bit and 64-bit ATLAS 3.9.25)
>
> Cheers,
> Simon
>
>
>> And since we are now straying away from the original topic about JAGS, let me know if you prefer that I start another thread. I do try to follow the posting guide as best I can.
>>
>> Michael
>>
>>
>>
>>
>>
>> On Oct 12, 2010, at 9:10 AM, Simon Urbanek wrote:
>>
>>> Michael,
>>>
>>> On Oct 9, 2010, at 5:56 PM, Michael Braun wrote:
>>>
>>>> Hi!
>>>>
>>>> I am trying to install rjags on my Mac Pro running OSX 10.6.4, and R 2.11.1 (which I compiled myself so I could link to the Intel MKL blas, which I prefer to other options).
>>>
>>>
>>> Just FWIW: CRAN R comes with shared BLAS option enabled and allows easy plug-in of MKL BLAS without the need to re-compile (just use the libRblas.dylib symlink). Note that MKL BLAS is often slower than ATLAS so if your main goal is speed I suggest using ATLAS (or vecLib unless you have Nehalem-type Mac Pro) - it is also easier to handle (both from technical and licensing point of view).
>>>
>>> Cheers,
>>> Simon
>>>
>>>
>>>
>>>> I was successfully able to install JAGS 2.1.0 through both the binary installer, and compiling from source (I get the same problem either way).
>>>>
>>>> But I would like to use the rjags package. When I do
>>>>
>>>> install.packages("rjags")
>>>>
>>>>
>>>> I get the folliowing output:
>>>>
>>>> * installing *source* package ‘rjags’ ...
>>>> checking for prefix by checking for jags... /usr/local/bin/jags
>>>> checking for g++... g++
>>>> checking for C++ compiler default output file name... a.out
>>>> checking whether the C++ compiler works... yes
>>>> checking whether we are cross compiling... no
>>>> checking for suffix of executables...
>>>> checking for suffix of object files... o
>>>> checking whether we are using the GNU C++ compiler... yes
>>>> checking whether g++ accepts -g... yes
>>>> checking how to run the C++ preprocessor... g++ -E
>>>> checking for grep that handles long lines and -e... /usr/bin/grep
>>>> checking for egrep... /usr/bin/grep -E
>>>> checking for ANSI C header files... yes
>>>> checking for sys/types.h... yes
>>>> checking for sys/stat.h... yes
>>>> checking for stdlib.h... yes
>>>> checking for string.h... yes
>>>> checking for memory.h... yes
>>>> checking for strings.h... yes
>>>> checking for inttypes.h... yes
>>>> checking for stdint.h... yes
>>>> checking for unistd.h... yes
>>>> checking Console.h usability... yes
>>>> checking Console.h presence... yes
>>>> checking for Console.h... yes
>>>> checking for gcc... gcc
>>>> checking whether we are using the GNU C compiler... yes
>>>> checking whether gcc accepts -g... yes
>>>> checking for gcc option to accept ISO C89... none needed
>>>> checking for jags_version in -ljags... yes
>>>> configure: creating ./config.status
>>>> config.status: creating src/Makevars
>>>> configure: creating ./config.status
>>>> config.status: creating src/Makevars
>>>> config.status: creating R/unix/zzz.R
>>>> ** libs
>>>> *** arch - x86_64
>>>> g++ -arch x86_64 -I/Library/Frameworks/R.framework/Resources/include -I/Library/Frameworks/R.framework/Resources/include/x86_64 -I/usr/local/include/JAGS -I/opt/intel/Compiler/11.1/089/Frameworks/mkl/Headers -I/Install_Files/MKL_gfortran_interface/include/em64t/lp64 -fPIC -m64 -msse4.2 -g -O2 -c jags.cc -o jags.o
>>>> g++ -arch x86_64 -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/opt/intel/Compiler/11.1/089/Frameworks/mkl/Libraries/em64t -L/Users/braunm/Install_Files/MKL_gfortran_interface/lib/em64t -o rjags.so jags.o -lgsl -lmkl_intel_lp64 -lmkl_sequential -lmkl_core -lpthread -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation
>>>> installing to /Users/braunm/.R/Library/rjags/libs/x86_64
>>>> ** R
>>>> ** data
>>>> ** preparing package for lazy loading
>>>> ** help
>>>> *** installing help indices
>>>> ** building package indices ...
>>>> Error : .onLoad failed in loadNamespace() for 'rjags', details:
>>>> call: dyn.load(file, DLLpath = DLLpath, ...)
>>>> error: unable to load shared library '/Users/braunm/.R/Library/rjags/libs/x86_64/rjags.so':
>>>> dlopen(/Users/braunm/.R/Library/rjags/libs/x86_64/rjags.so, 10): Symbol not found: _JAGS_NA
>>>> Referenced from: /Users/braunm/.R/Library/rjags/libs/x86_64/rjags.so
>>>> Expected in: flat namespace
>>>> in /Users/braunm/.R/Library/rjags/libs/x86_64/rjags.so
>>>> ERROR: installing package indices failed
>>>> * removing ‘/Users/braunm/.R/Library/rjags’
>>>>
>>>>
>>>> However, when I go to look at the symbols in the jags library, I see
>>>>
>>>> braunm4: /usr/local/lib $ nm libjags.dylib |grep JAGS
>>>> 0000000000098650 S _JAGS_NA
>>>> 0000000000110338 S _JAGS_NAN
>>>> 0000000000110348 S _JAGS_NEGINF
>>>> 0000000000110340 S _JAGS_POSINF
>>>> 000000000000133c t __GLOBAL__I_JAGS_NA
>>>>
>>>> and when I try
>>>>
>>>> braunm4: /usr/local/lib $ nm libjags.a |grep JAGS
>>>> U _JAGS_NA
>>>> U _JAGS_NEGINF
>>>> U _JAGS_NA
>>>> U _JAGS_NA
>>>> 0000000000000060 S _JAGS_NA
>>>> 00000000000000b8 B _JAGS_NAN
>>>> 00000000000000c8 B _JAGS_NEGINF
>>>> 00000000000000c0 B _JAGS_POSINF
>>>> U _JAGS_NA
>>>> U _JAGS_NEGINF
>>>> U _JAGS_POSINF
>>>> U _JAGS_NEGINF
>>>> U _JAGS_POSINF
>>>> U _JAGS_NEGINF
>>>> U _JAGS_NA
>>>> U _JAGS_NEGINF
>>>> U _JAGS_NEGINF
>>>> U _JAGS_NA
>>>> U _JAGS_NEGINF
>>>> U _JAGS_POSINF
>>>>
>>>>
>>>> So the jags libraries (both static and dynamic) are in the standard place, and it looks like the symbols rjags.so needs are there as well. So I'm not sure what the rjags installer is looking for. Is there perhaps an environment variable that I am missing?
>>>>
>>>> Here is my sessionInfo():
>>>>
>>>>> sessionInfo()
>>>> R version 2.11.1 (2010-05-31)
>>>> x86_64-apple-darwin10.4.2
>>>>
>>>> locale:
>>>> [1] en_US.UTF-8/en_US.UTF-8/C/C/en_US.UTF-8/en_US.UTF-8
>>>>
>>>> attached base packages:
>>>> [1] stats grDevices datasets graphics utils methods base
>>>>
>>>> other attached packages:
>>>> [1] xtable_1.5-6 doMC_1.2.1 multicore_0.1-3 foreach_1.3.0 codetools_0.2-2 iterators_1.0.3 coda_0.13-5 matrixcalc_1.0-1 lattice_0.18-8 reshape_0.8.3 plyr_1.2.1
>>>> [12] mvtnorm_0.9-92
>>>>
>>>> loaded via a namespace (and not attached):
>>>> [1] grid_2.11.1 tcltk_2.11.1 tools_2.11.1
>>>>
>>>>
>>>> Thanks in advance for your help.
>>>>
>>>> Michael
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> -------------------------------------------
>>>> Michael Braun
>>>> Homer A. Burnell (1928) Career Development Professor,
>>>> and Assistant Professor of Management Science (Marketing Group)
>>>> MIT Sloan School of Management
>>>> 100 Main St.., E62-535
>>>> Cambridge, MA 02139
>>>> braunm at mit.edu
>>>> 617-253-3436
>>>>
>>>> _______________________________________________
>>>> R-SIG-Mac mailing list
>>>> R-SIG-Mac at stat.math.ethz.ch
>>>> https://stat.ethz.ch/mailman/listinfo/r-sig-mac
>>>>
>>>>
>>>
>>
>> -------------------------------------------
>> Michael Braun
>> Homer A. Burnell (1928) Career Development Professor,
>> and Assistant Professor of Management Science (Marketing Group)
>> MIT Sloan School of Management
>> 100 Main St.., E62-535
>> Cambridge, MA 02139
>> braunm at mit.edu
>> 617-253-3436
>>
>>
>>
>
-------------------------------------------
Michael Braun
Homer A. Burnell (1928) Career Development Professor,
and Assistant Professor of Management Science (Marketing Group)
MIT Sloan School of Management
100 Main St.., E62-535
Cambridge, MA 02139
braunm at mit.edu
617-253-3436
More information about the R-SIG-Mac
mailing list