[R-sig-hpc] Loading rgdal as PBS script in R
Ei-ji Nakama
nakama at ki.rim.or.jp
Tue May 10 08:27:16 CEST 2016
Hi, suddenly remember...
1) When you build in intel Compiler, did you properly configured AR,LD, etc.?
> source /SOFT/ics_2013.1.046/icsxe/2013.1.046/ictvars.sh intel64
cf. https://software.intel.com/en-us/articles/build-r-301-with-intel-c-compiler-and-intel-mkl-on-linux
intel obj link in the of gnu-ar and gnu-ld to fail(but build time no
error report).
not good mixed tool-chain. please all re-build with IntelCompiler
(many librarys) or
not use IntelCompiler.
2) maybe start process in rsh(or ssh)?...# not under PBS process.
> mpirun -machinefile $PBS_NODEFILE -np 24 R CMD BATCH *test.r *
cf. https://wiki.mpich.org/mpich/index.php/Using_the_Hydra_Process_Manager#Resource_Managers_and_Launchers
CAUTION : I do not know all the combinations on cluster...
2016-05-09 21:07 GMT+09:00 Hariom Singh <hariom.cs1 at gmail.com>:
> Hey all,
>
> I thought I'd share this since we spent quite some time with it and it
> might be relevant to some of you. Or at least amusing, possibly.
>
> It is stated that, I am getting difficulty in loading rgdal R library while
> using PBS script for parallel computing environment....
>
> First of all I would like to share my relevant files....Please check the
> error..
>
> *##### test.sh pbs script file for running R program*
>
> #!/usr/bin/env bash
>
> #PBS -N snow_1
> #PBS -m abe
> #PBS -q medium
> #PBS -l nodes=2:ppn=12
> #PBS -V
>
> cd $PBS_O_WORKDIR
>
>
> cp /UHOME/hariodce/R/x86_64-unknown-linux-gnu-library/3.2/Rmpi/Rprofile
> ./.Rprofile
>
> cat $PBS_NODEFILE > pbsnodes
>
> mpirun -machinefile $PBS_NODEFILE -np 24 R CMD BATCH *test.r *
>
> *#### test.r ... test R file where rgdal is not loaded........*
>
> # Load the R MPI package if it is not already loaded.
>
> if (is.loaded("mpi_initialize"))
>
> {
>
> library("Rmpi")
> *library("rgdal")......*
>
> }
> # In case R exits unexpectedly, have it automatically clean up
> # resources taken up by Rmpi (slaves, memory, etc...)
> .Last <- function(){
> if (is.loaded("mpi_initialize")){
> if (mpi.comm.size(1) > 0){
> print("Please use mpi.close.Rslaves() to close slaves.")
> mpi.close.Rslaves()
> }
> print("Please use mpi.quit() to quit R")
> .Call("mpi_finalize")
>
> }
> }
> # Tell all slaves to return a message identifying themselves
> mpi.remote.exec(paste("I am",
> mpi.comm.rank(),"of",mpi.comm.size(),system("hostname",intern=T)))
>
> # Tell all slaves to close down, and exit the program
>
> mpi.close.Rslaves()
>
> *output file......*
> ..................
> Type 'demo()' for some demos, 'help()' for on-line help, or
> 'help.start()' for an HTML browser interface to help.
> Type 'q()' to quit R.
>
> master (rank 0 , comm 1) of size 24 is running on: cn05
> slave1 (rank 1 , comm 1) of size 24 is running on: cn05
> slave2 (rank 2 , comm 1) of size 24 is running on: cn05
> slave3 (rank 3 , comm 1) of size 24 is running on: cn05
> ... ... ...
> slave22 (rank 22, comm 1) of size 24 is running on: cn07
> slave23 (rank 23, comm 1) of size 24 is running on: cn07
> [Previously saved workspace restored]
>
>> # Load the R MPI package if it is not already loaded.
>>
>> if (is.loaded("mpi_initialize"))
> +
> + {
> +
> + library("Rmpi")
> + library("rgdal")
> +
> + } ########error...*.it always loaded without pbs scipt like
> library(rgdal) but here it is not loaded.....so what is the problem in
> loading.......Plz help me out.*
>
>
>
>
> *Loading required package: spError in dyn.load(file, DLLpath = DLLpath,
> ...) : unable to load shared object
> '/SOFT/R320/lib64/R/library/rgdal/libs/rgdal.so': libgdal.so.20: cannot
> open shared object file: No such file or directory*
> Error: package or namespace load failed for ‘rgdal’
>>
>>
>>
>> # In case R exits unexpectedly, have it automatically clean up
>> # resources taken up by Rmpi (slaves, memory, etc...)
>> .Last <- function(){
> + if (is.loaded("mpi_initialize")){
> + if (mpi.comm.size(1) > 0){
> + print("Please use mpi.close.Rslaves() to close slaves.")
> + mpi.close.Rslaves()
> + }
> + print("Please use mpi.quit() to quit R")
> + .Call("mpi_finalize")
> +
> + }
> + }
>> # Tell all slaves to return a message identifying themselves
>> mpi.remote.exec(paste("I am",
> mpi.comm.rank(),"of",mpi.comm.size(),system("hostname",intern=T)))
> $slave1
> [1] "I am 1 of 24 cn05"
>
> $slave2
> [1] "I am 2 of 24 cn05"
>
> .......................
> .............................
>
>
> *# .bashrc *
>
> # Source global definitions
> if [ -f /etc/bashrc ]; then
> . /etc/bashrc
> fi
>
> export R_LIBS_USER=/SOFT/R320/lib64/R/library
>
>
> *############## .bash_profile*
>
> # Get the aliases and functions
> if [ -f ~/.bashrc ]; then
> . ~/.bashrc
> fi
>
> # User specific environment and startup programs
>
> PATH=$PATH:$HOME/bin
>
> export PATH
> source /SOFT/ics_2013.1.046/icsxe/2013.1.046/ictvars.sh intel64
> export JAVA_HOME=/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.45.x86_64
> export PATH=/SOFT/R320/bin/:/SOFT/mpich-3.1.1/bin:$PATH
> export
> LD_LIBRARY_PATH=/SOFT/R320/lib64/:/SOFT/mpich-3.1.1/lib:$LD_LIBRARY_PATH
> export LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH
>
> Please help me. I would highly appreciate for your time and concern.
>
> Thanks in anticipation.
>
> Hariom Singh
>
> Ph.D research scholar , IIT Roorkee.
>
> [[alternative HTML version deleted]]
>
> _______________________________________________
> R-sig-hpc mailing list
> R-sig-hpc at r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-hpc
--
Best Regards,
--
Eiji NAKAMA <nakama (a) ki.rim.or.jp>
"\u4e2d\u9593\u6804\u6cbb" <nakama (a) ki.rim.or.jp>
More information about the R-sig-hpc
mailing list