[R-sig-hpc] Getting multithreaded R working (solution)

[Jonathan Greenberg]

Dirk:

Have you seen whether there is any significant performance boost in a
multithreaded system between GotoBLAS2 and Intel MKL?  I'm curious if
I should be spending time trying to get Intel MKL configured if I
already have a working GotoBLAS2...

--j

On Thu, Jul 29, 2010 at 3:29 PM, Jonathan Greenberg
<greenberg at ucdavis.edu> wrote:
> I really like that all we have to do is "update-alternatives --install
> /usr/lib/libblas.so.3gf" with your version -- gotoblas2 was a breeze
> to get working this technique.  Mostly with MKL I'm just unclear on
> what specific file I need to link against once its been installed.
> Actually, along these lines, does LAPACK work independently from the
> BLAS, e.g. will the "default" LAPACK (sudo apt-get install
> liblapack-dev) use whichever BLAS happens to be linked up, or do we
> need a different LAPACK (update-alternatives --config liblapack.so)
> for each particularly BLAS we are using (e.g. use the GotoBLAS2 lapack
> with GotoBLAS2, use Intel MKL's LAPACK, with Intel's BLAS, etc...)
>
> And unfortunately no revolution-mkl for debian yet :(  The intel
> install is easy tho, just run the install script and its all good.  I
> did move it out of /opt tho and into /usr/lib to live with the rest of
> the libraries on my system!
>
> --j
>
> On Thu, Jul 29, 2010 at 3:16 PM, Dirk Eddelbuettel <edd at debian.org> wrote:
>>
>> On 29 July 2010 at 23:46, Stefan Theussl wrote:
>> | [...]
>> |  > P.S. If anyone has *easy* instructions for getting atlas working as a
>> |  > multithreaded library or how do drop in the Intel MKL (I couldn't
>> |  > figure out which *.so would be the correct one) please let me know and
>> |  > I'll post them as well.
>> |
>> | I did get MKL working with R via the following trick:
>> |
>> | 1) export
>> | LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/intel/intel-mkl-9.0.018/lib/em64t
>> |
>> | 2) then configure with:
>> | "--with-blas=-L/opt/intel/intel-mkl-9.0.018/lib/em64t/ -lmkl -lguide
>> | -lpthread"
>> |
>> | Back then I used MKL version 9.0.018. I don't know if this still works
>> | with more recent MKL or R.
>>
>> Step 2) should not be necessary.  The BLAS is a binary interface; you can
>> "simply" drop the right ones in.  By having ld.so "find" the MKL ones before
>> the system ones, we made the MKL be used in the Revolution R release with
>> Ubuntu 9.10.  That still works for -- so no rebuilding of R needed here.
>>
>>
>> On 29 July 2010 at 14:56, Jonathan Greenberg wrote:
>> | Have you (or anyone) gotten Intel MKL working via the "plugin"
>> | approach that Dirk has setup for the debian packages, where you just
>> | point:
>>
>>
>> I have :)
>>
>> I have a proof-of-concept shell script that takes advantage of the Debian /
>> Ubuntu package management system and does the following for benchmarks:
>>
>>    i)    removes all accelerates blas and runs with 'refblas'
>>
>>    ii)   installs Atlas and runs with 'atlas'
>>
>>    iii)  removes Atlas, install MKL (via aforementioned Revo package) and
>>          runs with 'mkl'
>>
>>    iv)   remove MKL and install goto (using .deb package built with the neat
>>          goto-helper package from the ISM in Japan) and run 'goto'
>>
>> plus for kicks
>>
>>    v)    install gputools and runs with 'gpu'
>>
>> I showed first results in the 'Intro to HPC' tutorial at useR and want to do
>> some more work on this.  In fact, the gputools authors and I have a mostly
>> empty project on r-forge for this but not had time to do much...
>>
>>
>>
>> --
>> Dirk Eddelbuettel | edd at debian.org | http://dirk.eddelbuettel.com
>>
>