[R-sig-Debian] GotoBLAS2 breaks lapack

[Dirk Eddelbuettel]

On 1 March 2011 at 18:32, Paul Johnson wrote:
| On Sun, Feb 27, 2011 at 10:16 PM, Dirk Eddelbuettel <edd at debian.org> wrote:
| >
| > Hi Paul,
| >
| > On 27 February 2011 at 21:54, Paul Johnson wrote:
| > | Suppose the administrator has the Atlas blas installed, as well as the
| > | revolution-mkl, and then also the GOTOblas.  How does R--as you
| > | package it--select which one to use?
| >
| > These two or three distinct issues:
| >
| 
| Yes, that's what I intended to ask. I think you mistake me for
| somebody who doesn't understand that there are several options in
| system administration.  I just wanted to know how you choose among
| several installed BLAS. From your comments below, I gather you don't
| choose, rather you uninstall the other deb packages entirely?

But that is a special case (my gcbd project / benchmarking investigation),
which made the discussion needlessly more complicated.

For everything else, the system has sane defaults: Atlas optimised wins over
Atlas base which wins over Reference BLAS.  This has worked this way of over
half a decade.  [ I think the gotoblas2 helper plays along and the wins over
Atlas optimised --- in any case, I also built a local Atlas39 package which
was not part of the scheme. So I had to do something else. ]
 
| > i)   The package system has a mechanism to express and honour preferences,
| >     that is how Atlas is chose in case it as well as reference Blas are
| >     installed. Same for the different 'vi' commands from vim, nvi, ... or
| >     emacsen
| >
| 
| Yes, that's the "fiddling sym links in /etc/alternatives" approach
| that I mentioned in my previous email. Whether it is debian scripts or
| me typing at the keyboard, it is still just a directory full of
| symbolic links.  And I don't necessarily trust a deb package to do
| that any more than I trust myself.

It is simple yet robust way of enforcing preference reliably, through
upgrades, downgrades, etc... while allowing user overrides.  It works, for
dozen of package sets. 

You are free to call it whatever names you think are appropriate, we will
just carry on using it _as it works_.

As for the trusting: I would trust a well-maintained, known Debian package
almost always more than a local part-time sysadmin like you or me (as we have
other jobs).  This is actually key: the packages are put together by _people
who know what they are doing_.  Sylvestre is very knowledgeable about Blas,
Atlas, ... and I know a thing or two about how to configure R right.  Again,
you are free to whatever local hacks you prefer.  Choice is good.  But then I
keep wondering why I bother explaining this to you as you seemingly prefer
your local setup anyway. 

| > ii)  MKL is different as Revo made it appear in a directory only R uses.
| >     So different mechanism.
| >
| Right.  As you mention, it puts itself in the special place that only
| R uses.  R ignores Goto BLAS2 as long as the mkl blas is sitting
| there. Hence it appears absolutely necessary to fiddle the symbolic
| links to stop R from using mkl. Unless you just remove revolution-mkl
| package altogether.

Please understand that MKL was a one-off package put together at the request
of Revo and part of exactly one Ubuntu release.  It does not enter the Atlas
optimised / Atlas base / RefBLAS comparison above.  Nor does it relate to
Goto.  It was a one-off, ok?  So we cannot drag it into this discussion.  It
is also no longer maintained by anyone as far as I know.
 
| > iii) Goto is not formally packaged and part of the distro. I think the helper
| >     package is so good that is does the preferences scheme too but I'd have
| >     to check. For my gcbd project I scripted it such that exactly one is
| >     present.
| 
| Yes, but we were talking about Goto as you described it packaged, not
| in the abstract.
| 
| In case anybody wonders, here's how gotoblas2-helper handles it.
| 
| gotoblas2-helper manipulates the ld search path. It adds this file
| "00_gotoblas2.conf", hoping that it is the first in the alphabetical
| order among blas-related files in /etc/ld.so.conf.d.
| 
| $ cat /etc/ld.so.conf.d/00_gotoblas2.conf
| /usr/lib/gotoblas2
| 
| That's a safe gamble, until somebody packages up atlas and puts in
| 00_atlas.conf. Oops :)

Which is why that ld.so conf file approach is NOT part of Debian Policy, and
gotoblas2-helper is not an official package.  See above.

Sylvestre has started on Goto packaging for Debian. One day this will be
integrated.  You could actually help him....

| > | Following the R Install and Admin manual,  Section A.3.1, I've built R
| >
| > That has *nothing* to do or say on Debian package and hence no bearing on
| > your question.
| 
| It bears on my case, because I control which BLAS libraries are used
| by changing the symbolic links, in the way that the R manual
| describes.  Without uninstalling the deb packages, I don't see how it
| can be done.
| 
| So, in summary, I was just curious to know how people that use your R
| packages manage to choose among the several BLAS.

MKL, gotoblas2 etc are not part of a common installation.  Normal users take
what is in the distro: Atlas.  

Dirk

-- 
Dirk Eddelbuettel | edd at debian.org | http://dirk.eddelbuettel.com