
Thanks for your guidance, Dirk.

In regard to a) and b), I am shaky on using compilers, but it seems that Rmpi is working based on a Hello World script I used.

> mpi.spawn.Rslaves(nslaves=8)
	8 slaves are spawned successfully. 0 failed.
master (rank 0, comm 1) of size 9 is running on: john-desktop 
slave1 (rank 1, comm 1) of size 9 is running on: john-desktop 
slave2 (rank 2, comm 1) of size 9 is running on: john-desktop 
slave3 (rank 3, comm 1) of size 9 is running on: john-desktop 
slave4 (rank 4, comm 1) of size 9 is running on: john-desktop 
slave5 (rank 5, comm 1) of size 9 is running on: john-desktop 
slave6 (rank 6, comm 1) of size 9 is running on: john-desktop 
slave7 (rank 7, comm 1) of size 9 is running on: john-desktop 
slave8 (rank 8, comm 1) of size 9 is running on: john-desktop 
> mpi.remote.exec(paste("I am",mpi.comm.rank(),"of",mpi.comm.size()))
$slave1
[1] "I am 1 of 9"

$slave2
[1] "I am 2 of 9"

$slave3
[1] "I am 3 of 9"

and so on, all the way through "I am 8 of 9".  I am using only my own desktop, which is a Cori-i7, and not networking.  When I script mpi.spawn.Rslaves(), I get only 1 slave.

In regard to c: OK, I'll try to give the big picture, but I hope you can read through my weak computer vocabulary.  SWAT, the Soil and Water Assessment Tool, is a hydrologic model that simulates streamflow based on parameters such as soil permeability, type of vegetative cover, etc.  The SWAT executable consists of a compilation of over 300 Fortran source code files, and normally reads parameter values from ASCII files.  As the parameter values represent data obtained through geographic information systems, there may be hundreds to tens of thousands of such files, depending on the resolution of the information and the size of the basin.  If we want to change the parameter values thousands of times as part of an uncertainty analysis, then it becomes very advantageous to avoid rewriting all these files repeatedly, and to directly feed the parameter values to SWAT through R.  We modified the SWAT code so that it can receive these changes "externally" rather than look to the ASCII files.  Now, the file "swat2009_i64_calibrate.so" is meant to be the "interface" between the parameter values in R and the SWAT executable.  This has actually been done before by others with SWAT with CentOS as the operating system, and it is from them that I obtained makefiles.  (I'm using Ubuntu 10.04 because it was recommended to me as being more compatible with my Core-i7.)

In regard to d: My intention is to use dyn.load() to link swat2009_i64_calibrate.so to R.  To me, it does seem that I should not have to load Rmpi in order to use dyn.load(), since dyn.load() is part of the base package.  But...well, I guess I'm unclear on how Rmpi and dyn.load() are related. 

In regard to e: libifport.so.5 is part of or at least closely associated with the Intel Fortran compiler I am using.  I find it in opt/intel/lib/ia32/, but when I click to try to open it I get "Could not create the archive.  Archive type not supported."

In regard to your not getting the attachment:  I'm not sure why it got clipped, but below is copying and pasting of the file, less most of the lengthy list of source code files.  (This is the makefile I used to generate swat2009_i64_calibrate.so.)

# SWAT MAKEFILE FOR INTEL COMPILER
#
F95=ifort
FFLAGS= -c -g -fpe:0 -align sequence -fPIC -threads -fltconsistency -real-size 64 -debug all
LDFLAGS= -g -shared -fPIC -shared-intel -heap-arrays

SRC= main.autocal.calibrate.f\
     freearray_09.f\
     addh.f\

 (many source file names omitted for brevity in this e-mail)

     zero2.f\
     zeroini.f
MOD=parm.mod
EXE=swat2009
OBJ=$(SRC:.f=.o)
all: $(OBJ) 
ifort $(LDFLAGS) -g -o swat2009_i64_calibrate.so $(OBJ) 
clean:
	rm -f *.o parm.mod swat2009
$(MOD): modparm.f
	$(F95) $(FFLAGS) $<
$(OBJ): $(MOD)
$(EXE): $(OBJ) modparm.o
	$(F95) $(LDFLAGS) $(OBJ) -o $@
	## debug
	#$(F95) $(LDFLAGS) $(OBJ) $@
.f.o: modparm.f
	$(F95) $(FFLAGS) $< -o $@


Thank you, Dirk, and to anyone else who has read this far.  I much appreciate any help, as I have been struggling with this off and on for a couple of months now.

John Joseph


-----Original Message-----
From: Dirk Eddelbuettel [mailto:edd@debian.org]
Sent: Mon 11/29/2010 8:14 PM
To: John Joseph
Cc: Kevin.Buckley@ecs.vuw.ac.nz; r-sig-hpc@r-project.org
Subject: Re: [R-sig-hpc] trouble installing Rmpi
 

On 29 November 2010 at 19:46, John Joseph wrote:
| 
| Thanks, Kevin and Dirk, for the help you've given me.  I uninstalled
| (COMPLETELY unistalled) all of R, and then re-installed Rmpi, using the simple
| command Dirk had recommended: sudo apt-get install r-cran-rmpi.I am no longer
| getting errors when entering the script "library(Rmpi)", but I'm still having

At this point I strongly recommend two things:

  a) Ascertain that Open MPI works, either just on localhost or between two
     or more machines.

     There are numerous 'hello World' example for C and/or C++ floating
     around, try those.  [ This assumes some familiarity with compilers etc;
     if unsure skip the step. ]

  b) Ascertain that Rmpi work by running a helloWorld example in R.

Example for a) and b) are for example in the 'Intro to HPC with R' slides
from the different tutorials I have given on this.

| trouble associated with dyn.load().  I haven't found clarity by reading the
| documentation.  Maybe someone can at least help me focus my search among what
| I've looked at already: makefile flags/options, installation of the fortran
| compiler software (Intel Composer, latest edition), R manuals.
| Here's the problem I'm having:
| 
| > library(Rmpi)
| > library(snow)
| > setwd("/home/john/LREWI15_TIO")
| > dyn.load("./swat2009_i64_calibrate.so")
| Error in dyn.load("./swat2009_i64_calibrate.so") :
|   unable to load shared library '/home/john/LREWI15_TIO/./
| swat2009_i64_calibrate.so':
|   libifport.so.5: cannot open shared object file: No such file or directory
| >

I am confused:

   c) What is this program?  

   d) Why do you load Rmpi and then compile code? The idea of Rmpi is to give
      you MPI idioms _in R_ without the need to compile anything more.

   e) What is libifport ?
 
| I've also attached the makefile I used to generate swat2009_i64_calibrate.so,
| if it helps.

I saw no attachment.

Good luck,  Dirk

 
| Any guidance would be much appreciated. 
| 
| John Joseph
| 
| P.S.  Thanks for the added clarity regarding what worked for you for Rmpi.  If
| you think the instructions are still relevant to my new situation, I would be
| happy to still try to work them out.  But I have a little fear of digging
| myself into a deeper hole, so I'll need to ask you a couple of things first.
| 
| -----Original Message-----
| From: r-sig-hpc-bounces@r-project.org on behalf of Kevin.Buckley@ecs.vuw.ac.nz
| Sent: Sun 11/21/2010 4:11 PM
| To: r-sig-hpc@r-project.org
| Subject: Re: [R-sig-hpc] trouble installing Rmpi
| 
| >
| > Thanks for your help, Kevin.
| >
| > These instructions look very helpful, and I've begun studying them.
| > I'm such a newbie that I understand little of it.  I'm learning a
| > little by reading the documentation, but I'm just not getting all of
| > it.  It seems that the idea is to make a file called "Makevars" that
| > is stored in certain directories.  I did the first three lines, but
| > can't find where Makevars went...Well, I did find a Makevars and
| > Makevars.win file several directories deep into the R directory that
| > was already there, but I don't know if these two files were created
| > by the 3 lines I wrote or were already there...Anyway, as you can
| > see, I'm struggling with the basics.
| >
| > Another thing is that when I go to etc/profile.d I do not find
| > openmpi.sh within it.
| >
| > In any case, I appreciate this help you gave.
| >
| > John Joseph
| 
| hey John,
| 
| it seems as though I forgot to copy the list into my "advice/experience"
| so am now doing so (see below for original)
| 
| In respect of the issues you raise, the /etc/profile.d/openmpi.sh
| is a LOCAL thing, which merely gives the users, at login time, the
| correct environment (paths, etch) for OpenMPI.
| 
| Chances are that that's being handled for you in some way because
| you installed OpenMPI from a package.
| 
| The inclusion of that line there was just me running things explicitly
| as a "hedge" against a user who might not have had such an environment
| set up. Apologies, I should have tailored the commands to remove the
| "local" bits.
| 
| As to the R Makevars file I referred to, again the syntax of my
| suggestions is creating it as though it was not there (I should
| have, of course, suggested checking to see if the user account
| being used for the installation already had such a file as my
| "instructions" trample all over it).
| 
| That ties in with the note about removing the Makevars file
| afterwards: if you already had one and moved it out of the
| way, or otherwise appended the variables that Rmpi seemed to
| need, you'ld want to simply remove them or replace the old file,
| rather than bin it.
| 
| But basically, you won't see that file ~/.R/Makevars (you might
| not even see the directory ~/.R) unless it had already been created
| beneath the home directory of the user doing the installtion.
| 
| Kevin
| 
| PS
| 
| Just in case it is something you have not seen before, the ~ (tilde)
| character at the start of the path to the Makevears referred to
| is a shorthand for the home directory of the user, in case the user
| doing the install.
| 
| 
| > -----Original Message-----
| > From: Kevin.Buckley@ecs.vuw.ac.nz
| > Sent: Thu 11/18/2010 3:35 PM
| > To: John Joseph
| > Subject: Re: [R-sig-hpc] trouble installing Rmpi
| >
| >
| > Not directly addressing your install methodology issues, John, but
| >
| >  "the way it worked for me"
| >
| > was to install from outside of R itself, but still using R, so as
| > to have some control over the paths to our OpenMPI installtion.
| >
| > Note the use of the R "Makevars" file to override some
| > of the defaults R might be picking up if left to its own
| > devices:
| >
| > The "mpicc" is the OpenMPI-aware C compiler wrapper
| >
| > You might need to tweak the paths for Ubuntu's OpenMPI package
| > setup.
| >
| > This was for Rmpi_0.5-8.tar.bz2
| >
| > mkdir ~/.R
| > echo CC=mpicc > ~/.R/Makevars
| > echo SHLIB_LD=mpicc >> ~/.R/Makevars
| > . /etc/profile.d/openmpi.sh
| >
| > /opt/R/bin/R CMD INSTALL \
| >  --library=/opt/Rlibs \
| >  /path/to/Rmpi_0.5-8.tar.bz2 \
| >  --configure-args=--with-Rmpi-type=OPENMPI  \
| >  --configure-args=--with-Rmpi-libpath=/opt/openmpi/1.4.2/lib \
| >  --configure-args=--with-Rmpi-include=/opt/openmpi/1.4.2/include
| >
| > Oh yeah, once you have built the package, (assuming you do!),
| > remove the file
| >
| >  ~/.R/Makevars
| >
| > unless you want all future R packages you build by hand, to be built
| > against OpenMPI that is !
| >
| > Kevin
| 
| --
| Kevin M. Buckley                                  Room:  CO327
| School of Engineering and                         Phone: +64 4 463 5971
|  Computer Science
| Victoria University of Wellington
| New Zealand
| 
| _______________________________________________
| R-sig-hpc mailing list
| R-sig-hpc@r-project.org
| https://stat.ethz.ch/mailman/listinfo/r-sig-hpc
| 
| 
| 
| 
| 
| 
| xMakefile.calibrate.gmk, Makefile.ca [Click mouse-2 to save to a file]
| _______________________________________________
| R-sig-hpc mailing list
| R-sig-hpc@r-project.org
| https://stat.ethz.ch/mailman/listinfo/r-sig-hpc

-- 
Dirk Eddelbuettel | edd@debian.org | http://dirk.eddelbuettel.com











	[[alternative HTML version deleted]]

