[R-sig-hpc] Rmpi and shared libraries

[vlagani at ics.forth.gr]

Dear All,

I am trying to install Rmpi on a Cray XT4 system with a Unicos/lc OS.

As simple user, I have to install Rmpi under a local directory; R  
2.9.0 is already available for all the users.

Well, the installation of the package was pretty simple. I used the command:

R CMD INSTALL --configure-args="--with-mpi=<path_to_mpi>   
CC=path_to_CC" -l <mydirectory> Rmpi_0.5-7.tar.gz

and I got a very clean installation, without errors.

However, I checked the Rmpi.so library:

> R CMD ldd Rmpi.so
libmpich.so.1.1 => not found
libutil.so.1 => /lib64/libutil.so.1 (0x00002afcda620000)
libpthread.so.0 => <path_to_R>/R/lib/libpthread.so.0 (0x00002afcda724000)
libc.so.6 => <path_to_R>/R/lib/libc.so.6 (0x00002afcda83b000)
/lib64/ld-linux-x86-64.so.2 (0x0000555555554000)


After reading some r-sig-hpc posts, I easily solved this problem:

> LD_LIBRARY_PATH=<path_to_mpi>/lib:$LD_LIBRARY_PATH
> R CMD ldd Rmpi.so
libmpich.so.1.1 => <path_to_mpi>/lib/libmpich.so.1.1 (0x00002b5a65272000)
libutil.so.1 => /lib64/libutil.so.1 (0x00002b5a65550000)
libpthread.so.0 => <path_to_R>/R/lib/libpthread.so.0 (0x00002b5a65654000)
libc.so.6 => <path_to_R>/R/lib/libc.so.6 (0x00002b5a6576b000)
libportals.so.1 => not found
libpmi.so => not found
libalpslli.so.0 => not found
libalpsutil.so.0 => not found
librt.so.1 => /lib64/librt.so.1 (0x00002b5a6599e000)
/lib64/ld-linux-x86-64.so.2 (0x0000555555554000)


After a while, I recognized that libportals.so.1, libpmi.so, etc. were  
under a non-standard directory. So I modified the path in order to have:

LD_LIBRARY_PATH=<path_to_missing_libs>:<path_to_mpi>/lib:$LD_LIBRARY_PATH

After this modification "R CMD ldd Rmpi.so" was able to find all the  
libraries.

Unfortunately, when I try to run a simple Rmpi script, I receive an error.

The script (rscript.R) is just one line:

> library(Rmpi, lib.loc='<mydirectory>');


I launch rscript.R with the following command:

aprun -n 1 -N 1 R CMD BATCH rscript.R

and when the job is finished this is the output (rscript.Rout):

library(Rmpi, lib.loc='<mydirectory>')
Error in dyn.load(file, DLLpath = DLLpath, ...) :
   unable to load shared library '<mydirectory>/Rmpi/libs/Rmpi.so':
   <path_to_missing_libs>/libpmi.so: undefined symbol: ptl_err_str
Error in library(Rmpi, lib.loc = "<mydirectory>") :
   .First.lib failed for 'Rmpi'
Error in dyn.unload(file.path(libpath, "libs", paste("Rmpi",  
.Platform$dynlib.ext,  :
   dynamic/shared library '/<mydirectory>/Rmpi/libs/Rmpi.so' was not loaded
Execution halted

So, if you are still reading, thanks for your attention: I know that  
this is a very long email!

Well, at this point I would like to ask:

1) Why I was able to compile and link Rmpi.so if libportals, libpmi,  
etc. were not under standard directories?

2) When I call "R CMD ldd Rmpi.so" all the libs are in the form:
lib => <somewhere>lib
except /lib64/ld-linux-x86-64.so.2. Any explanations?

3) Maybe the problem is that I have to compile the missing libraries  
(libportals, libpmi, etc) by myself. The ones that I am using were  
compiled in order to be used by OpenFOAM, so maybe they were compiled  
with some particular options. Where can I find the source of this  
libraries?

4) The administrator of the system suggested to me to link Rmpi.so  
only with static libraries. The reasons are inherent to the  
architecture of the supercomputer, and to be honest, I don't  
understand them.
I mean, I should force gcc to link Rmpi.so to libmpich.a, libpmi.a,  
etc., rather than to libmpich.so, libpmi.so, etc. I guess that I  
should play a little bit with the "configure" file of the Rmpi  
package, but I am quite a linux newbie and I  am not so confident with  
shell scripting.

Well, that's all! Thanks again for reading. And thanks in advance for  
any answer!

Ciao,

Vincenzo Lagani