[R] Running R-program as queue jobs

[Prof Brian Ripley]

On 13 Nov 2003, Peter Dalgaard wrote:

> Jason Turner <jasont at indigoindustrial.co.nz> writes:
> 
> > Philipp Pagel wrote:
> > > 	Hi!
> > > On Thu, Nov 13, 2003 at 09:59:48AM +0000, Arne Gjuvsland wrote:
> > >
> > >>I have a problem with running my R programs as queue jobs. When I try
> > >>to submit a batch file to the queue with qsub I get the following error
> > >>message:
> > >>____________________________________________________________
> > >>
> > >>/home/gjuvslan/kluster/R-1.7.1/bin/R.bin: error while loading shared
> > >> libraries: libpcre.so.0: cannot load shared object file: No such
> > >> file or directory
> > >>____________________________________________________________
> > >>
> > >>When executed from the command prompt the batch file does its job.
> > > Did you run the script on the same machine in both cases? I got
> > > burnt a
> > > couple of times with different machines running different versions of
> > > the OS, non-identical versions of shared libraries etc...
> > >
> > 
> > In addition to Philipp's good sugestion, I've been burnt on the same
> > machine, but with the batch job running as a different user.  When
> > environment variables are needed, or private libraries need to be
> > loaded, things go bad very quickly.
> 
> Also, environment variables may be set differently between interactive
> and batch shells. E.g. my crontab file looks like this
> 
> CVS_RSH=ssh
> 35 0 * * *  $HOME/scripts/r-bugs-commit > /dev/null
> ...
> 
> for a reason. In Arne's case, I'd suspect the setting of
> LD_LIBRARY_PATH. 

(The R script will include in R_LD_LIBRARY_PATH the settings used at
configure time, so we do try to workaround that one.  For example my 
64-bit Solaris build has sparcv9 library paths in.)

As an aside, from the next non-patch release PCRE, BZLIB and ZLIB will be
statically linked into R in a vanilla configuration to help avoid such
problems.  (For some reason my RH8.0 system has a dynamic pcreposix but 
only a static pcre, although the current PCRE default build makes both 
versions of each.)

-- 
Brian D. Ripley,                  ripley at stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272866 (PA)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595