[R-sig-hpc] cannot install ncdf4_1.6.1 in R-3.0.0 on Cray XE6 HERMIT (R-sig-hpc Digest, Vol 56, Issue 4)
Pragnesh Patel
pragnesh777 at gmail.com
Tue May 7 15:35:42 CEST 2013
Hi Martin,
It looks like you need to recompile hdf5 and netcdf with
"--enable-shared" option (either in your home directory or lustre
directory(I assume you use lustre file system)). Once you have those,
you should be able to install pbdNCDF4 and ncdf4 packages.
Please let me know, if you need detail installation instructions of
hdf5, netcdf, ncdf4 and pbdNCDF4 on Cray XE6 machine.
(TO run: you need to copy all *.so files(from hdf5 and netcdf lib
directory) to lustre file system so compute nodes can access those).
Thanks
Pragnesh
On 5/5/13, r-sig-hpc-request at r-project.org
<r-sig-hpc-request at r-project.org> wrote:
> Send R-sig-hpc mailing list submissions to
> r-sig-hpc at r-project.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
> https://stat.ethz.ch/mailman/listinfo/r-sig-hpc
> or, via email, send a message with subject or body 'help' to
> r-sig-hpc-request at r-project.org
>
> You can reach the person managing the list at
> r-sig-hpc-owner at r-project.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of R-sig-hpc digest..."
>
>
> Today's Topics:
>
> 1. cannot install ncdf4_1.6.1 in R-3.0.0 on Cray XE6 HERMIT
> (Martin Ivanov)
> 2. Re: Trouble compiling R-3.0.0 an a Linux Cluster (Martin
> Ivanov) (Martin Ivanov)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Sat, 04 May 2013 15:25:53 +0200
> From: Martin Ivanov <martin.ivanov at ifg.uni-tuebingen.de>
> To: "R-sig-hpc at r-project.org" <r-sig-hpc at r-project.org>
> Subject: [R-sig-hpc] cannot install ncdf4_1.6.1 in R-3.0.0 on Cray XE6
> HERMIT
> Message-ID: <51850C61.10201 at ifg.uni-tuebingen.de>
> Content-Type: text/plain; charset=UTF-8; format=flowed
>
> Dear All,
>
> I am working on Cray xe6 (HLRS) HERMIT. Recently I had a problem
> compiling R-3.0.0.
> Finally the problem is solved, I had to use PrgEnv-gnu/4.1.40 together
> with craype-target-native, and also supply
> LDFLAGS="-L$PREFIX/lib64 -Wl,-rpath -Wl,$PREFIX/lib64
> -L/univ_2/ws3/ws/ipmiva-WRF_331_CORDEX-0/build/tmp/R-3.0.0/lib",
> where PREFIX=/univ_2/ws3/ws/ipmiva-WRF_331_CORDEX-0/system/usr.
>
> Now that I have R, I also need to work with netcdf data files, so I need
> to use the netcdf library. I want
> netcdf-4.1.1 and hdf5-1.8.5. I see both are available as modules, so I
> loaded hdf5-parallel/1.8.5.0 and
> netcdf-hdf5parallel/4.1.1.0. To use netcdf from within R I have to
> compile the ncdf4_1.6.1.tar.gz package.
> And here is my problem. This package uses the nc-config file to tune the
> compilation environment. However,
> the
> /opt/cray/netcdf-hdf5parallel/4.1.1.0/netcdf-hdf5parallel-gnu/bin/nc-config
>
> contains obsolete information:
>
> prefix=/tmp/ulib/netcdf/4.1.1.0/netcdf-hdf5parallel/netcdf-hdf5parallel-gnu
> exec_prefix=${prefix}
> libdir=${exec_prefix}/lib
> includedir=${prefix}/include
> cc="cc"
> cxx="CC"
> fc="ftn"
> cflags=" -I${includedir}"
> fflags="-g -O2 -I${includedir}"
> libs="-L${libdir} -lnetcdf
> -L/tmp/ulib/hdf5/1.8.5.0/hdf5-parallel/hdf5-parallel-gnu/lib -lhdf5_hl
> -lhdf5 -L/usr/lib64/lib -lz -lm "
> flibs="-L${libdir} -lnetcdff -lnetcdf
> -L/tmp/ulib/hdf5/1.8.5.0/hdf5-parallel/hdf5-parallel-gnu/lib -lhdf5_hl
> -lhdf5 -L/usr/lib64/lib -lz -lm "
>
> the prefix as well as
> /tmp/ulib/hdf5/1.8.5.0/hdf5-parallel/hdf5-parallel-gnu do not exist any
> more. That is why I copied the nc-config to
> my home directory and modified it with the correct paths like this:
>
> prefix=/opt/cray/netcdf-hdf5parallel/4.1.1.0/netcdf-hdf5parallel-gnu
> exec_prefix=${prefix}
> libdir=${exec_prefix}/lib
> includedir=${prefix}/include
>
> cc="cc"
> cxx="CC"
> fc="ftn"
> cflags=" -I${includedir}"
> fflags="-g -O2 -I${includedir}"
> libs="-L${libdir} -lnetcdf
> -L/opt/cray/hdf5-parallel/1.8.5.0/hdf5-parallel-gnu/lib -lhdf5_hl
> -lhdf5 -L/usr/lib64/lib -lz -lm "
> flibs="-L${libdir} -lnetcdff -lnetcdf
> -L/opt/cray/hdf5-parallel/1.8.5.0/hdf5-parallel-gnu/lib -lhdf5_hl
> -lhdf5 -L/usr/lib64/lib -lz -lm "
>
> Compilation failed with the message that mpi.h is not found. So I
> modified the cflags like this:
>
> cflags=" -I${includedir}
> -I/opt/cray/mpt/5.6.4/gni/mpich2-cray64/74/include" and now the
> compilation fails with this message:
>
> ********************** Results of ncdf4 package configure
> *******************
>
> netCDF v4 CPP flags =
> -I/opt/cray/netcdf-hdf5parallel/4.1.1.0/netcdf-hdf5parallel-gnu/include
> -I/opt/cray/mpt/5.6.4/gni/mpich2-cray/74/include
> netCDF v4 LD flags =
> -Wl,-rpath,/opt/cray/netcdf-hdf5parallel/4.1.1.0/netcdf-hdf5parallel-gnu/lib
>
> -Wl,-rpath,/opt/cray/hdf5-parallel/1.8.5.0/hdf5-parallel-gnu/lib
> -Wl,-rpath,/usr/lib64/lib
> -L/opt/cray/netcdf-hdf5parallel/4.1.1.0/netcdf-hdf5parallel-gnu/lib
> -lnetcdf -L/opt/cray/hdf5-parallel/1.8.5.0/hdf5-parallel-gnu/lib
> -lhdf5_hl -lhdf5 -L/usr/lib64/lib -lz -lm
> netCDF v4 runtime path =
> -Wl,-rpath,/opt/cray/netcdf-hdf5parallel/4.1.1.0/netcdf-hdf5parallel-gnu/lib
>
> -Wl,-rpath,/opt/cray/hdf5-parallel/1.8.5.0/hdf5-parallel-gnu/lib
> -Wl,-rpath,/usr/lib64/lib
>
> ******************************************************************************
>
> ** libs
> gcc -std=gnu99
> -I/univ_2/ws3/ws/ipmiva-WRF_331_CORDEX-0/system/usr/include -DNDEBUG
> -I/opt/cray/netcdf-hdf5parallel/4.1.1.0/netcdf-hdf5parallel-gnu/include
> -I/opt/cray/mpt/5.6.4/gni/mpich2-cray/74/include -I/usr/local/include
> -fpic -g -O2 -c ncdf.c -o ncdf.o
> gcc -std=gnu99
> -I/univ_2/ws3/ws/ipmiva-WRF_331_CORDEX-0/system/usr/include -DNDEBUG
> -I/opt/cray/netcdf-hdf5parallel/4.1.1.0/netcdf-hdf5parallel-gnu/include
> -I/opt/cray/mpt/5.6.4/gni/mpich2-cray/74/include -I/usr/local/include
> -fpic -g -O2 -c ncdf2.c -o ncdf2.o
> gcc -std=gnu99
> -I/univ_2/ws3/ws/ipmiva-WRF_331_CORDEX-0/system/usr/include -DNDEBUG
> -I/opt/cray/netcdf-hdf5parallel/4.1.1.0/netcdf-hdf5parallel-gnu/include
> -I/opt/cray/mpt/5.6.4/gni/mpich2-cray/74/include -I/usr/local/include
> -fpic -g -O2 -c ncdf3.c -o ncdf3.o
> gcc -std=gnu99
> -I/univ_2/ws3/ws/ipmiva-WRF_331_CORDEX-0/system/usr/include -DNDEBUG
> -I/opt/cray/netcdf-hdf5parallel/4.1.1.0/netcdf-hdf5parallel-gnu/include
> -I/opt/cray/mpt/5.6.4/gni/mpich2-cray/74/include -I/usr/local/include
> -fpic -g -O2 -c src_ncdf4.c -o src_ncdf4.o
> gcc -std=gnu99 -shared
> -L/univ_2/ws3/ws/ipmiva-WRF_331_CORDEX-0/system/usr/lib64 -Wl,-rpath
> -Wl,/univ_2/ws3/ws/ipmiva-WRF_331_CORDEX-0/system/usr/lib64
> -L/univ_2/ws3/ws/ipmiva-WRF_331_CORDEX-0/build/tmp/R-3.0.0/lib -o
> ncdf4.so ncdf.o ncdf2.o ncdf3.o src_ncdf4.o
> -Wl,-rpath,/opt/cray/netcdf-hdf5parallel/4.1.1.0/netcdf-hdf5parallel-gnu/lib
>
> -Wl,-rpath,/opt/cray/hdf5-parallel/1.8.5.0/hdf5-parallel-gnu/lib
> -Wl,-rpath,/usr/lib64/lib
> -L/opt/cray/netcdf-hdf5parallel/4.1.1.0/netcdf-hdf5parallel-gnu/lib
> -lnetcdf -L/opt/cray/hdf5-parallel/1.8.5.0/hdf5-parallel-gnu/lib
> -lhdf5_hl -lhdf5 -L/usr/lib64/lib -lz -lm
> -L/univ_2/ws3/ws/ipmiva-WRF_331_CORDEX-0/system/usr/lib64/R/lib -lR
> /usr/bin/ld:
> /opt/cray/netcdf-hdf5parallel/4.1.1.0/netcdf-hdf5parallel-gnu/lib/libnetcdf.a(nc4attr.o):
>
> relocation R_X86_64_32 against `.rodata' can not be used when making a
> shared object; recompile with -fPIC
> /opt/cray/netcdf-hdf5parallel/4.1.1.0/netcdf-hdf5parallel-gnu/lib/libnetcdf.a:
>
> could not read symbols: Bad value
> collect2: error: ld returned 1 exit status
> make: *** [ncdf4.so] Error 1
> ERROR: compilation failed for package ?ncdf4?
>
>
> I have tried instead -I/opt/cray/mpt/5.6.4/gni/mpich2-cray64/74/include
> to also use /opt/cray/mpt/5.6.4/gni/mpich2-gnu/47/include and
> /opt/cray/mpt/5.6.4/gni/mpich2-cray/74/include, but I always get the
> same error message. I have found this:
> http://www.gentoo.org/proj/en/base/amd64/howtos/?part=1&chap=3
> on this topic, but I get confused, because as one can see -fpic is
> already used as a flag.
>
> By the way my environment is as follows:
> ipmiva at eslogin001:/univ_2/ws3/ws/ipmiva-WRF_331_CORDEX-0> module list
> Currently Loaded Modulefiles:
> 1) modules/3.2.6.7 14)
> ugni/4.0-1.0401.5928.9.5.gem 27) lbcd/2.1-1.0401.35360.1.2.gem
> 2) xtpe-network-gemini 15)
> pmi/4.0.1-1.0000.9421.73.3.gem 28)
> nodehealth/5.0-1.0401.38460.12.18.gem
> 3) xtpe-interlagos 16)
> dmapp/3.2.1-1.0401.5983.4.5.gem 29) pdsh/2.26-1.0401.37449.1.1.gem
> 4) cray-mpich2/5.6.4 17)
> gni-headers/2.1-1.0401.5675.4.4.gem 30)
> shared-root/1.0-1.0401.37253.3.50.gem
> 5) eswrap/1.0.9 18)
> xpmem/0.1-2.0401.36790.4.3.gem 31)
> switch/1.0-1.0401.36779.2.72.gem
> 6) torque/2.5.9 19)
> job/1.5.5-0.1_2.0401.35380.1.10.gem 32) xe-sysroot/4.1.40
> 7) moab/6.1.5.s1992 20)
> csa/3.0.0-1_2.0401.37452.4.50.gem 33) atp/1.6.2
> 8) system/ws_tools 21)
> dvs/1.8.6_0.9.0-1.0401.1401.1.120 34) PrgEnv-gnu/4.1.40
> 9) system/hlrs-defaults 22)
> rca/1.0.0-2.0401.38656.2.2.gem 35) craype-target-native
> 10) xt-asyncpe/5.19 23)
> audit/1.0.0-1.0401.37969.2.32.gem 36) hdf5-parallel/1.8.5.0
> 11) gcc/4.7.2 24)
> ccm/2.2.0-1.0401.37254.2.142 37) netcdf-hdf5parallel/4.1.1.0
> 12) xt-libsci/12.0.01 25)
> configuration/1.0-1.0401.35391.1.2.gem
> 13) udreg/2.3.2-1.0401.5929.3.3.gem 26)
> hosts/1.0-1.0401.35364.1.115.gem
>
>
> Any suggestions will be appreciated.
>
> Best regards,
>
> Martin
>
>
>
>
>
> --
> Dr. Martin Ivanov
> Eberhard-Karls-Universit?t T?bingen
> Mathematisch-Naturwissenschaftliche Fakult?t
> Fachbereich Geowissenschaften
> Water & Earth System Science (WESS)
> H?lderlinstra?e 12, 72074 T?bingen, Deutschland
> Tel. +4970712974213
>
>
>
> ------------------------------
>
> Message: 2
> Date: Sat, 04 May 2013 15:30:40 +0200
> From: Martin Ivanov <martin.ivanov at ifg.uni-tuebingen.de>
> To: Rodney Sparapani <rsparapa at mcw.edu>
> Cc: Martin Ivanov
> <public-martin.ivanov-qUHYnPI9w1ZN1tXVncjlcWfrygkm6VTR at plane.gmane.org>,
> r-sig-hpc at r-project.org,
> public-r-sig-hpc-0bNBQ1PAWB4BXFe83j6qeQ at plane.gmane.org
> Subject: Re: [R-sig-hpc] Trouble compiling R-3.0.0 an a Linux Cluster
> (Martin Ivanov)
> Message-ID: <51850D80.5060606 at ifg.uni-tuebingen.de>
> Content-Type: text/plain; charset=UTF-8; format=flowed
>
> Dear Dr Sparapani,
>
>
> Thank You very much for Your suggestion, it worked like a charm. Finally
> I have R-3.0.0 on Cray XE6.
> Now I have another problem: I cannot install ncdf4, which I have
> described in another thread, called
> "cannot install ncdf4_1.6.1 in R-3.0.0 on Cray XE6 HERMIT".
>
> Best regards,
> Martin
>
> On 04/30/2013 06:26 PM, Rodney Sparapani wrote:
>> On 04/29/2013 04:54 PM, Pragnesh Patel wrote:
>>> export LD_LIBRARY_PATH=$R_WORK_HOME/lib64/R/lib:$LD_LIBRARY_PATH
>>
>> Good catch. But, instead of fooling around with LD_LIBRARY_PATH,
>> which can cause all kinds of headaches, you might be better off
>> trying something like...
>>
>> LDFLAGS="-L$PREFIX/lib${LIBDIRSUFFIX} -Wl,-rpath \
>> -Wl,$PREFIX/lib${LIBDIRSUFFIX}"
>>
>> Rodney
>>
>> _______________________________________________
>> R-sig-hpc mailing list
>> R-sig-hpc at r-project.org
>> https://stat.ethz.ch/mailman/listinfo/r-sig-hpc
>
>
> --
> Dr. Martin Ivanov
> Eberhard-Karls-Universit?t T?bingen
> Mathematisch-Naturwissenschaftliche Fakult?t
> Fachbereich Geowissenschaften
> Water & Earth System Science (WESS)
> H?lderlinstra?e 12, 72074 T?bingen, Deutschland
> Tel. +4970712974213
>
>
>
> ------------------------------
>
> _______________________________________________
> R-sig-hpc mailing list
> R-sig-hpc at r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-hpc
>
>
> End of R-sig-hpc Digest, Vol 56, Issue 4
> ****************************************
>
--
pragnesh patel
Graduate Student | Computer Science
Indiana University Bloomington
More information about the R-sig-hpc
mailing list