[R-sig-hpc] Rmpi on NetBSD with OpenMPI
Kevin.Buckley at ecs.vuw.ac.nz
Kevin.Buckley at ecs.vuw.ac.nz
Mon Mar 29 02:19:28 CEST 2010
Hi there,
I have recently compiled Rmpi 0.5-8 against the OpenMPI 1.4
installation on the School's NetBSD workstations.
As someone who spent some time getting OpenMPI to run upon
NetBSD, I am aware that it's not a common combination, so
I am posting here more in hope that someone will have seen
similar errors and has any pointers, than expecting an actual
solution for our platform and MPI flavour combo.
Basically, when I come to load the Rmpi library
> library(Rmpi, lib.loc="/local/scratch/kevin/Pkgs/R/")
I get a swathe of OpenMPI errors (attached below)
I have tried invoking R using orterun instead of mpirun,
as suggested in the thread from this list from Feb 2009
entitled,
Rmpi with Open MPI on Debian
as, despite the Debian slant, that actually seemed to touch upon
generic OpenMPI issues.
For background:
When I compiled the Rmpi.so, I had to do so by overiding the
compiler and linker via a ~/.R/Makevars file as there seemed
no other way to get the OpenMPI wrappers passed to the R build
process on the command line.
That issue was touched upon in another thread on this list from
Jan 2010
R CMD INSTALL configure.args and CC customization
So, with an ~/.R/Makevars that has
CC=mpicc
SHLIB_LD=mpicc
I do an
R CMD INSTALL --library=/local/scratch/kevin/Pkgs/R Rmpi_0.5-8.tar.gz \
--configure-args=--with-Rmpi-type=OPENMPI \
--configure-args=--with-Rmpi-libpath=/usr/pkg/lib/openmpi \
--configure-args=--with-Rmpi-include=/usr/pkg/include
and see
* installing *source* package 'Rmpi' ...
checking for openpty in -lutil... yes
checking for main in -lpthread... yes
configure: creating ./config.status
config.status: creating src/Makevars
** libs
mpicc -I/usr/pkg/lib/R/include -DPACKAGE_NAME=\"\" -DPACKAGE_TARNAME=\"\"
-DPACKAGE_VERSION=\"\" -DPACKAGE_STRING=\"\" -DPACKAGE_BUGREPORT=\"\"
-I/usr/pkg/include -DMPI2 -DOPENMPI -I/usr/include -I/usr/pkg/include
-I/usr/pkg/include/freetype2 -fpic -O2 -march=pentiumpro
-I/usr/include -I/usr/pkg/include -I/usr/pkg/include/freetype2 -c
RegQuery.c -o RegQuery.o
mpicc -I/usr/pkg/lib/R/include -DPACKAGE_NAME=\"\" -DPACKAGE_TARNAME=\"\"
-DPACKAGE_VERSION=\"\" -DPACKAGE_STRING=\"\" -DPACKAGE_BUGREPORT=\"\"
-I/usr/pkg/include -DMPI2 -DOPENMPI -I/usr/include -I/usr/pkg/include
-I/usr/pkg/include/freetype2 -fpic -O2 -march=pentiumpro
-I/usr/include -I/usr/pkg/include -I/usr/pkg/include/freetype2 -c Rmpi.c
-o Rmpi.o
mpicc -I/usr/pkg/lib/R/include -DPACKAGE_NAME=\"\" -DPACKAGE_TARNAME=\"\"
-DPACKAGE_VERSION=\"\" -DPACKAGE_STRING=\"\" -DPACKAGE_BUGREPORT=\"\"
-I/usr/pkg/include -DMPI2 -DOPENMPI -I/usr/include -I/usr/pkg/include
-I/usr/pkg/include/freetype2 -fpic -O2 -march=pentiumpro
-I/usr/include -I/usr/pkg/include -I/usr/pkg/include/freetype2 -c
conversion.c -o conversion.o
mpicc -I/usr/pkg/lib/R/include -DPACKAGE_NAME=\"\" -DPACKAGE_TARNAME=\"\"
-DPACKAGE_VERSION=\"\" -DPACKAGE_STRING=\"\" -DPACKAGE_BUGREPORT=\"\"
-I/usr/pkg/include -DMPI2 -DOPENMPI -I/usr/include -I/usr/pkg/include
-I/usr/pkg/include/freetype2 -fpic -O2 -march=pentiumpro
-I/usr/include -I/usr/pkg/include -I/usr/pkg/include/freetype2 -c
internal.c -o internal.o
mpicc -shared -L/usr/lib -Wl,-R/usr/lib -L/usr/pkg/lib -Wl,-R/usr/pkg/lib
-o Rmpi.so RegQuery.o Rmpi.o conversion.o internal.o
-L/usr/pkg/lib/openmpi -lmpi -lutil -lpthread -Wl,-R/usr/pkg/lib/R/lib
-L/usr/pkg/lib/R/lib -lR
** R
** demo
** inst
** preparing package for lazy loading
** help
*** installing help indices
** building package indices ...
* DONE (Rmpi)
The shared library itself shows that it's aware of the OpenMPI stuff
$ldd /local/scratch/kevin/Pkgs/R/Rmpi/libs/Rmpi.so
/local/scratch/kevin/Pkgs/R/Rmpi/libs/Rmpi.so:
-lc.12 => /usr/lib/libc.so.12
-lltdl.7 => /usr/pkg/lib/libltdl.so.7
-lutil.7 => /usr/lib/libutil.so.7
-lm.0 => /usr/lib/libm.so.0
-lpthread.0 => /usr/lib/libpthread.so.0
-lopen-pal.0 => /usr/pkg/lib/libopen-pal.so.0
-lopen-rte.0 => /usr/pkg/lib/libopen-rte.so.0
-lmpi.0 => /usr/pkg/lib/libmpi.so.0
-lblas.1 => /usr/pkg/lib/libblas.so.1
-lintl.0 => /usr/lib/libintl.so.0
-ltermcap.0 => /usr/lib/libtermcap.so.0
-lreadline.6 => /usr/pkg/lib/libreadline.so.6
-lpcre.0 => /usr/pkg/lib/libpcre.so.0
-llzma.0 => /usr/pkg/lib/liblzma.so.0
-lbz2.1 => /usr/lib/libbz2.so.1
-lz.1 => /usr/lib/libz.so.1
-liconv.2 => /usr/pkg/lib/libiconv.so.2
-lR => /usr/pkg/lib/R/lib/libR.so
however after either invoking R as
orterun -n 1 R --no-save
and trying to load the library at the R prompt, or setting
R_PROFILE to load it automatically
via the following block:
if (!invisible(library(Rmpi,logical.return = TRUE,
lib.loc="/local/scratch/kevin/Pkgs/R/"))){
warning("Rmpi cannot be loaded")
q(save = "no")
}
$ R_PROFILE=/local/scratch/kevin/Pkgs/R/Rmpi/Rprofile orterun -n 1 R
--no-save
I see the same errors from OpenMPI
[europa.ecs.vuw.ac.nz:09687] mca: base: component_find: unable to open
/usr/pkg/lib/openmpi/mca_carto_auto_detect: perhaps a missing symbol, or
compiled for a different version of Open MPI? (ignored)
[europa.ecs.vuw.ac.nz:09687] mca: base: component_find: unable to open
/usr/pkg/lib/openmpi/mca_carto_file: perhaps a missing symbol, or compiled
for a different version of Open MPI? (ignored)
[europa.ecs.vuw.ac.nz:09687] mca: base: component_find: unable to open
/usr/pkg/lib/openmpi/mca_ess_env: perhaps a missing symbol, or compiled
for a different version of Open MPI? (ignored)
--------------------------------------------------------------------------
A requested component was not found, or was unable to be opened. This
means that this component is either not installed or is unable to be
used on your system (e.g., sometimes this means that shared libraries
that the component requires are unable to be found/loaded). Note that
Open MPI stopped checking at the first component that it did not find.
Host: europa.ecs.vuw.ac.nz
Framework: ess
Component: env
--------------------------------------------------------------------------
[europa.ecs.vuw.ac.nz:09687] [[INVALID],INVALID] ORTE_ERROR_LOG: Error in
file runtime/orte_init.c at line 120
--------------------------------------------------------------------------
It looks like orte_init failed for some reason; your parallel process is
likely to abort. There are many reasons that a parallel process can
fail during orte_init; some of which are due to configuration or
environment problems. This failure appears to be an internal failure;
here's some additional information (which may only be relevant to an
Open MPI developer):
orte_ess_base_open failed
--> Returned value Error (-1) instead of ORTE_SUCCESS
--------------------------------------------------------------------------
--------------------------------------------------------------------------
It looks like MPI_INIT failed for some reason; your parallel process is
likely to abort. There are many reasons that a parallel process can
fail during MPI_INIT; some of which are due to configuration or environment
problems. This failure appears to be an internal failure; here's some
additional information (which may only be relevant to an Open MPI
developer):
ompi_mpi_init: orte_init failed
--> Returned "Error" (-1) instead of "Success" (0)
--------------------------------------------------------------------------
*** An error occurred in MPI_Init
*** before MPI was initialized
*** MPI_ERRORS_ARE_FATAL (your MPI job will now abort)
[europa.ecs.vuw.ac.nz:9687] Abort before MPI_INIT completed successfully;
not able to guarantee that all other processes were killed!
If I just do an orterun of R and dont use Rmpi, things seem to work.
Oh yeah, I also tried submitting a job into our SGE grid, in case
OpenMPI gets a different set up when running within SGE as opposed
to on the desktop but see the same errors.
As I say, if anyone has seen similar errors upon their platform,
I'd be interested to hear of your experiences.
Kevin
--
Kevin M. Buckley Room: CO327
School of Engineering and Phone: +64 4 463 5971
Computer Science
Victoria University of Wellington
New Zealand
More information about the R-sig-hpc
mailing list