Dear List,

I have a rather curious problem when running Rmpi scripts under SGE. I 
submit the following R-script (an example from Stephen's doMPI package) to 
the grid engine:

# This example comes from an old version of the Wikipedia article on
# bootstrapping, which actually included R code.  The task being executed
# in the foreach loop executes very quickly, but it does a lot of trials.
# That makes it a good example of the need for the chunkSize option.

suppressMessages(library(doMPI))

# Create and register an MPI cluster
cl <- startMPIcluster()
registerDoMPI(cl)

# Initialize variables
x <- iris[which(iris[,5] != "setosa"), c(1,5)]
trials <- 100000

# Define chunkSize so that each cluster worker gets a single "task chunk"
chunkSize <- ceiling(trials / getDoParWorkers())

cat("Workers:", getDoParWorkers(), "ChunkSize:", chunkSize, "\n")

mpiopts <- list(chunkSize=chunkSize)

# Perform the bootstrapping in parallel
r <- foreach(icount(trials), .combine='cbind', .options.mpi=mpiopts) 
%dopar% {
  ind <- sample(100, 100, replace=TRUE)
  result1 <- glm(x[ind,2]~x[ind,1], family=binomial(logit))
  structure(coefficients(result1), names=NULL)
}

# save the resulting matrix
save(r, file="bootMPI_r_test.Rdata")


The R-Script is called via orterun from a bash script:

$> qsub -pe pe_r  9 bootMPI.sh

#!/bin/sh
# Export all environment variables
#$ -V
# Set this job name
#$ -N mpiSGEbootMole_3
# Use current working directory
#$ -cwd
# Join stdout and stder into one file
#$ -j y
#$ -q mole
# The mpirun command; note the lack of host names.
echo $NSLOTS
/programs/openmpi-1.3.3/bin/orterun -hostfile sge_ompi_nodes -n $NSLOTS 
--mca pls_gridengine_verbose 1 Rscript bootMPIsmall.R


This produces the following output:


/bin/sh: error importing function definition for `mc'
/bin/sh: mc: line 1: syntax error: unexpected end of file
/bin/sh: error importing function definition for `mc'
9
error: commlib error: can't connect to service (Connection refused)
error: executing task of job 2469452 failed: failed sending task to 
execd@node49.cluster: can't find connection
--------------------------------------------------------------------------
A daemon (pid 3504) died unexpectedly with status 1 while attempting
to launch so we are aborting.

There may be more information reported by the environment (see above).

This may be because the daemon was unable to find all the needed shared
libraries on the remote node. You may set your LD_LIBRARY_PATH to have the
location of the shared libraries on the remote nodes and this will
automatically be forwarded to the remote nodes.
--------------------------------------------------------------------------
--------------------------------------------------------------------------
orterun noticed that the job aborted, but has no info as to the process
that caused that situation.
--------------------------------------------------------------------------
error: commlib error: can't connect to service (Connection refused)
error: executing task of job 2469452 failed: failed sending task to 
execd@node43.cluster: can't find connection
--------------------------------------------------------------------------
orterun was unable to cleanly terminate the daemons on the nodes shown
below. Additional manual cleanup may be required - please refer to
the "orte-clean" tool for assistance.
--------------------------------------------------------------------------
        node49.cluster - daemon did not report back when launched
        node45.cluster - daemon did not report back when launched
        node43.cluster - daemon did not report back when launched
        node35.cluster - daemon did not report back when launched


What's totally strange: If  I submit the bash script via

$> qsub -pe pe_r  4 bootMPI.sh

so that the whole job can run on a single node, the job runs smoothly 
without errors. 
 I can rsh and ssh without password from all nodes to all nodes.

Our System administrator is at a loss. Does anybody have an idea what 
might be the problem?

Thanks a lot 

Sebastian 
	[[alternative HTML version deleted]]

