[Rd] Re: [R] LD_LIBRARY_PATH is harmfull

[Prof Brian Ripley]

On Mon, 21 Feb 2005, Alexander Klimov wrote:

> Hi.
>
> On Mon, 21 Feb 2005, Prof Brian Ripley wrote:
>>> Consider the following situation: Solaris 8, /usr/local has gcc 2.95,
>>> my home has gcc 3.4.3, my gcc is first on PATH, LD_LIBRARY_PATH is
>>> unset (everything is perfect since I always use -R). In fact, programs
>>> compiled with my gcc do not work if LD_LIBRARY_PATH is set to
>>> something which has /usr/local/lib before home/lib, because it
>>> overrides stored path (-R) and I got
>>>
>>>   libgcc_s.so.1 (GCC_3.3) =>       (version not found)
>>
>> I don't believe that happens unless `home/lib' is not in the library paths
>> at all.
>
> Notice that I do have libgcc_s in /usr/local/lib, but it is for
> gcc2.95, so ldd said `(version not found)' because there is a library
> but with wrong version -- I guess other libraries simply are not
> considered at all.

Something is very strange: notice you say you have 3.4.3 and it is looking 
for GCC_3.3!  I get

         libgcc_s.so.1 =>         /usr/local/lib/libgcc_s.so.1

and no version number.

>> I've checked man ld and man ld.so.1 on Solaris 8, and neither appear
>> to mention that -R is ignored if LD_LIBRARY_PATH is set.  I think
>> that is a (well-buried) Solaris-specific gotcha.
>
> The standard meaning of LD_LIBRARY_PATH is to be considered *before*
> any other path.

Yes.  That is not what you say happens for you, though, rather you are 
sayin *instead of*.

>>> I found setting of LD_LIBRARY_PATH in bin/R (how /usr/local/lib get
>>> into it at all and especially before PREFIX/lib??? -- there was no
>>> LD_LIBRARY_PATH during configure!) and fixed it (although I spent
>>> quite a while editing lib/R/bin/R and wondering an abscence of
>>> any effect :-)
>>
>> Please take a look at e.g. config.site, which explains this under LDFLAGS,
>> (as does the R-admin manual which INSTALL asks you to read).
>
> Note that I do not need any *additional* flags because
> -L<home>/lib and -R<home>/lib are already in my specs file.

You need something to find R's shared libraries, e.g. libRlapack.

>>> After all the troubles I manage to load an extension, but, frankly, I
>>> think there were too many problems. It would be very nice if R-project
>>> reject the idea of using LD_LIBRARY_PATH because its setting is
>>> considered harmfull and leads to too many problems:
>>> http://www.linuxmafia.com/faq/Admin/ld-lib-path.html
>>
>> [Which is Solaris specific, despite the site name.]
>
> AFAIK LD_LIBRARY_PATH has the same semantic on Linux and every other
> recent *nix: it overrides system-wide defaults (e.g., crle on solaris
> and ldconfig on linux) as well as embeded paths; -R or -rpath are also
> common among *nixes.

The issue is if they are ignored, not if they are overridden.

>
>> Unfortunately, the alternatives lead to even more problems, and this is
>> the first report we have had for years of a problem
>
> Note that I do not have a problem with R installation, but I was
> troubled to load an extension (gmp in my case).
>
>> (which can be solved on reading the documentation)
>
> Could you, please, enlight me what page of R-admin.pdf I missed?

No, as I don't have your layout.  But it _is_ there:

If you have libraries and header files, e.g., for @acronym{GNU}
readline, in non-system directories, use the variables @code{LDFLAGS}
(for libraries, using @samp{-L} flags to be passed to the linker) and
@code{CPPFLAGS} (for header files, using @samp{-I} flags to be passed to
the C/C++ preprocessors), respectively, to specify these locations.
These default to @file{/usr/local/lib} and @file{/usr/local/include} to
catch the most common cases.

[You clearly do so have.]

...

Library paths specified as @option{-L/lib/path} in @code{LDFLAGS} are
collected together and prepended to @env{LD_LIBRARY_PATH} (or your
system's equivalent), so there should be no need for @option{-R} or
@option{-rpath} flags.

and in config.site.

>> As the R-admin manual points out, we regularly test on Solaris 8 and
>> give an example there of setting LDFLAGS under the Solaris section.
>
> B.7.3 (p.22) says nothing about my case (32-bit solaris and gcc >3.3)

It gives an example of setting LDFLAGS: you cannot expect every user's 
setup to be mentioned.

>> Use of -R is harmful for sure!  It stops R being relocatable (so it
>> either could not be tested before being installed or it would not
>> run after installation),
>
> Testing before installtion is exactly the purpose of LD_LIBRARY_PATH:
> you compile with -R for final installation and use
> LD_LIBRARY_PATH=<build-dir>/lib during testing. Alternatively you can
> use $ORIGIN so that libraries are searched in the places relative to
> executable itself.
>
>> and it is not at all portable.
>
> And could you name at least a single modern platform which supports
> shared libraries and does not support an equivalent of -R for linking?

Windows (probably the most-used platform for R).

I actually said -R is not portable, and many platforms do not have that.

> BTW, LD_LIBRARY_PATH also has different names on different *nix
> (e.g., SHLIB_PATH on HP-UX, although AFAIK HP-UX also supports
> LD_LIBRARY_PATH)
>
>> Maybe one day when we have libtool tamed we will be able to use the
>> multiple equivalents of -R or LD_RUN_PATH in a portable-enough way.
> Nice to know that such plans exist :-)

-- 
Brian D. Ripley,                  ripley at stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272866 (PA)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595