[R-sig-hpc] Setting up Rmpi, snow, snowfall on hpc

Dirk Eddelbuettel edd at debian.org
Tue Mar 15 15:59:08 CET 2011


On 15 March 2011 at 10:43, Brian Smith wrote:
| Hi,
| 
| Sorry, if this appears to be a naive question! I was trying to set up a
| parallel R working environment on my account on hpc (running SGE). I tried
| to install Rmpi, but wasn't successful. What do I need to get this going ?
| 
| My attempt so far (sessionInfo, and OS details) are below:
| 
| OS on hpc:
| Linux version 2.6.9-55.ELsmp (mockbuild at builder6.centos.org) (gcc version
| 3.4.6 20060404 (Red Hat 3.4.6-8)) #1 SMP
| 
| > install.packages("Rmpi",
| dependencies=TRUE)
| also installing the dependency ‘rsprng’
| 
| trying URL '
| http://www.ibiblio.org/pub/languages/R/CRAN/src/contrib/rsprng_1.0.tar.gz'
| Content type 'application/x-gzip' length 35916 bytes (35 Kb)
| opened URL
| ==================================================
| downloaded 35 Kb
| 
| trying URL '
| http://www.ibiblio.org/pub/languages/R/CRAN/src/contrib/Rmpi_0.5-9.tar.gz'
| Content type 'application/x-gzip' length 87953 bytes (85 Kb)
| opened URL
| ==================================================
| downloaded 85 Kb
| 
| * installing *source* package ‘rsprng’ ...
| checking for gcc... gcc
| checking for C compiler default output file name... a.out
| checking whether the C compiler works... yes
| checking whether we are cross compiling... no
| checking for suffix of executables...
| checking for suffix of object files... o
| checking whether we are using the GNU C compiler... yes
| checking whether gcc accepts -g... yes
| checking for gcc option to accept ANSI C... none needed
| Try to find sprng.h ...
| checking how to run the C preprocessor... gcc -E
| checking for egrep... grep -E
| checking for ANSI C header files... yes
| checking for sys/types.h... yes
| checking for sys/stat.h... yes
| checking for stdlib.h... yes
| checking for string.h... yes
| checking for memory.h... yes
| checking for strings.h... yes
| checking for inttypes.h... yes
| checking for stdint.h... yes
| checking for unistd.h... yes
| checking sprng.h usability... no
| checking sprng.h presence... no
| checking for sprng.h... no
| Cannot find sprng 2.0 header file.

The rsprng package needs the SPRNG library. Which is a bit of pain to install
build. You may consider rlecuyer as an alternative parallel RNG.

| ERROR: configuration failed for package ‘rsprng’
| * removing ‘/home/bs/R_home/R-2.11.1/library/rsprng’
| * installing *source* package ‘Rmpi’ ...
| checking for gcc... gcc -std=gnu99
| checking for C compiler default output file name... a.out
| checking whether the C compiler works... yes
| checking whether we are cross compiling... no
| checking for suffix of executables...
| checking for suffix of object files... o
| checking whether we are using the GNU C compiler... yes
| checking whether gcc -std=gnu99 accepts -g... yes
| checking for gcc -std=gnu99 option to accept ISO C89... none needed
| I am here /usr/lib/lam and it is LAM
| Trying to find mpi.h ...
| Found in /usr/lib/lam/include
| Trying to find libmpi.so or libmpich.a ...
| Found libmpi in /usr/lib/lam/lib
| Try to find liblam.so ...
| Found liblam in /usr/lib/lam/lib
| checking for openpty in -lutil... yes
| checking for main in -lpthread... yes
| configure: creating ./config.status
| config.status: creating src/Makevars
| ** libs
| gcc -std=gnu99 -I/home/bs/R_home/R-2.11.1/include -DPACKAGE_NAME=\"\"
| -DPACKAGE_TARNAME=\"\" -DPACKAGE_VERSION=\"\" -DPACKAGE_STRING=\"\"
| -DPACKAGE_BUGREPORT=\"\" -I/usr/lib/lam/include  -DMPI2 -DLAM
| -I/usr/local/include    -fpic  -g -O2 -c RegQuery.c -o RegQuery.o
| gcc -std=gnu99 -I/home/bs/R_home/R-2.11.1/include -DPACKAGE_NAME=\"\"
| -DPACKAGE_TARNAME=\"\" -DPACKAGE_VERSION=\"\" -DPACKAGE_STRING=\"\"
| -DPACKAGE_BUGREPORT=\"\" -I/usr/lib/lam/include  -DMPI2 -DLAM
| -I/usr/local/include    -fpic  -g -O2 -c Rmpi.c -o Rmpi.o
| gcc -std=gnu99 -I/home/bs/R_home/R-2.11.1/include -DPACKAGE_NAME=\"\"
| -DPACKAGE_TARNAME=\"\" -DPACKAGE_VERSION=\"\" -DPACKAGE_STRING=\"\"
| -DPACKAGE_BUGREPORT=\"\" -I/usr/lib/lam/include  -DMPI2 -DLAM
| -I/usr/local/include    -fpic  -g -O2 -c conversion.c -o conversion.o
| gcc -std=gnu99 -I/home/bs/R_home/R-2.11.1/include -DPACKAGE_NAME=\"\"
| -DPACKAGE_TARNAME=\"\" -DPACKAGE_VERSION=\"\" -DPACKAGE_STRING=\"\"
| -DPACKAGE_BUGREPORT=\"\" -I/usr/lib/lam/include  -DMPI2 -DLAM
| -I/usr/local/include    -fpic  -g -O2 -c internal.c -o internal.o
| gcc -std=gnu99 -shared -L/usr/local/lib64 -o Rmpi.so RegQuery.o Rmpi.o
| conversion.o internal.o -L/usr/lib/lam/lib -lmpi -llam -lutil -lpthread
| /usr/bin/ld: skipping incompatible /usr/lib/lam/lib/libmpi.so when searching
| for -lmpi
| /usr/bin/ld: skipping incompatible /usr/lib/lam/lib/libmpi.a when searching
| for -lmpi
| /usr/bin/ld: cannot find -lmpi
| collect2: ld returned 1 exit status
| make: *** [Rmpi.so] Error 1
| ERROR: compilation failed for package ‘Rmpi’
| * removing ‘/home/bs/R_home/R-2.11.1/library/Rmpi’
| 
| The downloaded packages are in
|         ‘/tmp/RtmpShmM5e/downloaded_packages’
| Updating HTML index of packages in '.Library'
| Warning messages:
| 1: In install.packages("Rmpi", dependencies = TRUE) :
|   installation of package 'rsprng' had non-zero exit status
| 2: In install.packages("Rmpi", dependencies = TRUE) :
|   installation of package 'Rmpi' had non-zero exit status

There is a very detailed README with installation notes in the Rmpi sources.
Try to see if you can get that to work.  Your system did find LAM, however
most folks these days recommend going with the Open MPI which has replaced LAM.

I don't have a RH box so I can't help you with any other details.

Good luck,  Dirk

| >
| 
| 
| 
| 
| > sessionInfo()
| R version 2.11.1 (2010-05-31)
| x86_64-unknown-linux-gnu
| 
| locale:
|  [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C
|  [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8
|  [5] LC_MONETARY=C              LC_MESSAGES=en_US.UTF-8
|  [7] LC_PAPER=en_US.UTF-8       LC_NAME=C
|  [9] LC_ADDRESS=C               LC_TELEPHONE=C
| [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
| 
| attached base packages:
| [1] stats     graphics  grDevices utils     datasets  methods   base
| 
| loaded via a namespace (and not attached):
| [1] tools_2.11.1
| >
| 
| ================
| 
| Any help would be highly appreciated!
| 
| thanks!
| 
| 	[[alternative HTML version deleted]]
| 
| 
| ----------------------------------------------------------------------
| _______________________________________________
| R-sig-hpc mailing list
| R-sig-hpc at r-project.org
| https://stat.ethz.ch/mailman/listinfo/r-sig-hpc

-- 
Dirk Eddelbuettel | edd at debian.org | http://dirk.eddelbuettel.com



More information about the R-sig-hpc mailing list