[R] Running R-program as queue jobs - problem solved

[Arne Gjuvsland]

At 11:56 13.11.2003 +0000, Prof Brian Ripley wrote:
>On 13 Nov 2003, Peter Dalgaard wrote:
>
>> Jason Turner <jasont at indigoindustrial.co.nz> writes:
>> 
>> > Philipp Pagel wrote:
>> > > 	Hi!
>> > > On Thu, Nov 13, 2003 at 09:59:48AM +0000, Arne Gjuvsland wrote:
>> > >
>> > >>I have a problem with running my R programs as queue jobs. When I try
>> > >>to submit a batch file to the queue with qsub I get the following error
>> > >>message:
>> > >>____________________________________________________________
>> > >>
>> > >>/home/gjuvslan/kluster/R-1.7.1/bin/R.bin: error while loading shared
>> > >> libraries: libpcre.so.0: cannot load shared object file: No such
>> > >> file or directory
>> > >>____________________________________________________________
>> > >>
>> > >>When executed from the command prompt the batch file does its job.
>> > > Did you run the script on the same machine in both cases? I got
>> > > burnt a
>> > > couple of times with different machines running different versions of
>> > > the OS, non-identical versions of shared libraries etc...
>> > >
>> > 
>> > In addition to Philipp's good sugestion, I've been burnt on the same
>> > machine, but with the batch job running as a different user.  When
>> > environment variables are needed, or private libraries need to be
>> > loaded, things go bad very quickly.
>> 
>> Also, environment variables may be set differently between interactive
>> and batch shells. E.g. my crontab file looks like this
>> 
>> CVS_RSH=ssh
>> 35 0 * * *  $HOME/scripts/r-bugs-commit > /dev/null
>> ...
>> 
>> for a reason. In Arne's case, I'd suspect the setting of
>> LD_LIBRARY_PATH. 
>
>(The R script will include in R_LD_LIBRARY_PATH the settings used at
>configure time, so we do try to workaround that one.  For example my 
>64-bit Solaris build has sparcv9 library paths in.)
>
>As an aside, from the next non-patch release PCRE, BZLIB and ZLIB will be
>statically linked into R in a vanilla configuration to help avoid such
>problems.  (For some reason my RH8.0 system has a dynamic pcreposix but 
>only a static pcre, although the current PCRE default build makes both 
>versions of each.)
>
>-- 
>Brian D. Ripley,                  ripley at stats.ox.ac.uk
>Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
>University of Oxford,             Tel:  +44 1865 272861 (self)
>1 South Parks Road,                     +44 1865 272866 (PA)
>Oxford OX1 3TG, UK                Fax:  +44 1865 272595
>
>
Thanks for helpful advice, the R_LD_LIBRARY_PATH in the R script did not
contain
the path of the library. Now the script is in the queue, waiting for some
quality CPU-time.

Arne