[R-sig-hpc] wrestling to compile sprng as RPM for a Rocks Cluster. Long rambling whining request for help :)

Thu Jan 28 09:09:00 CET 2010

Paul -

As per Hao's comment, it is far better to use the rlecuyer package --
sprng is theoretically not as well justifiable as an RNG.

On Mon, Jan 18, 2010 at 9:16 PM, Hao Yu <hyu at stats.uwo.ca> wrote:
> Paul,
>
> rsprng is suggested (not required) in Rmpi. I am using rlecuyer. rlecuyer
> can be simply installed in R as
> install.packages("rlecuyer")
>
> To use rlecuyer in Rmpi, run mpi.setup.rngstream(). Otherwise use
> mpi.setup.sprng() for SRRNG.
>
> Hao
>
>
> Paul Johnson wrote:
>> Anybody have sprng (scalable parallel random number generator) RPM and
>> SRPM files for RedHat/Centos with MPI?
>>
>> There are various posts about troubles compiling it, but I don't find
>> details on the fixes.
>>
>> Many R packages for HPC require Rmpi, Rmpi requires rsprng, which in
>> turn requires the C++/Fortran library SPRNG (capitalized? or not?).
>>
>> SPRNG is here, http://sprng.fsu.edu/. AFAICT, it is now in version 4.
>> The source (dated 6/7/2007) is available here:
>>
>> http://sprng.fsu.edu/Version4.0/sprng4.tar.gz
>>
>> I've been trying to compile that on Centos so I can build an RPM for a
>> Cluster.
>>
>> There are a number of wrinkles with this compile. It has a standard
>> GNU INSTALL file, which is encouraging.  But
>>
>> 1. The c++ code in the sprng4 package is not consistent with the
>> guidelines of the current gcc c++ compiler.  There are quite a few
>> little changes to make, like changing
>>
>> #include <iostream.h>
>>
>> to
>>
>> #include <iostream>
>>
>> I am attaching a patch file which includes the changes  that are, as
>> far as I can tell, MINIMALLY necessary to get sprng4 to compile,
>> except for the EXAMPLES directory.
>>
>> In the sprng4 source directory, this patch requires -p2, as in
>>
>> patch -p2 < sprng-4.patch
>>
>> 2. The build in the EXAMPLES directory  has some big problems. Even
>> after applying that patch, the "make" fails with errors in [1].
>>
>> The make will complete, however, if we delete the EXAMPLES directory
>> from the Makefile in the top level. That is, change:
>>
>> SUBDIRS = SRC lib check TESTS include EXAMPLES .
>>
>> to
>>
>> SUBDIRS = SRC lib check TESTS include .
>>
>> After applying the sprng-4.patch (attached), running the configure,
>> and then making that one change, then the make will finish.
>>
>> 3. The "make install" step does not copy the include files into the
>> install directory. I mean, if you run "ls" in the install folder,
>> there is "bin", "lib" and "share", but no "include".
>>
>>
>> I found many hints about the required fixes in the packaging efforts
>> of Eugeny A. Rostovtsev (REAL) <real at altlinux.org> in the ALTLINUX
>> distribution.  The SRPM file supplies "sprng4.tar.gz", but it is not
>> the original source code package from the SPRNG authors.  The
>> packaging is a little atypical for the RPM framework, as one is
>> supposed to supply the original "pristine" source and a patch.
>> Nevertheless, it is very informative. The patch I attach here includes
>> the changes incorporated in the ALTLINUX package (after some cleanup).
>> Most changes are C++ cleanup.
>>
>> In the SPEC file that Mr. Rostovtsev uses in ALTLINUX, he deals with
>> the "lack of installed include files" problem by copying the *.h files
>> "by hand" after running "make install". For your reference, I'm
>> attaching that SPEC file so you can see for yourself. The SPEC file
>> assumes the MPI implementation has an executable file called
>> "mpivars.sh", and there is no such file in OpenMPI 1.3.
>>
>> In Debian/Ubuntu, I see Dirk Eddelbuettel supplies libsprng-2.  I'm trying
>> to see how that compiles. The source code on that one is pretty
>> old.  If I can't figure out sprng4, I
>> will go back and work more on sprng-2 following Dirk's lead.
>>
>> I find also Nakama's mention of troubles compiling sprng4 and some
>> proposed patches:
>>
>> http://prs.ism.ac.jp/~nakama/mascagni/
>>
>> He proposes patches and fixes, but I can't make them compile AT ALL.
>> I notice those patches were proposed here about 1.5 years ago.
>>
>> If you have ideas, I'd be glad to hear them. In the interim, I'm going
>> to study how Dirk compiled & installed SPRNG2 to see if I can make an RPM
>> that way.
>>
>> pj
>>
>> [1]
>> mpif77 -Wl,-R/usr/local/libexec/openmpi/lib -DHAVE_CONFIG_H -I. -I.
>> -I../.. -I../../include   -I../../include -DSPRNG_MPI -g -O2 -c -o
>> sprngf_mpi_tmp-sprngf_mpi.o `test -f 'sprngf_mpi.F' || echo
>> './'`sprngf_mpi.F
>> mpif77 -Wl,-R/usr/local/libexec/openmpi/lib  -g -O2   -o
>> sprngf_mpi.tmp -I/usr/lib/openmpi/include
>> -I/usr/lib/openmpi/include/openmpi -pthread
>> -Wl,-R/usr/local/libexec/openmpi/lib --showme:link
>> sprngf_mpi_tmp-sprngf_mpi.o -L../../lib -lsprng -lm -lstdc++ -lm
>> -pthread -Wl,-R/usr/local/libexec/openmpi/lib -g -O2 -o sprngf_mpi.tmp
>> -I/usr/lib/openmpi/include -I/usr/lib/openmpi/include/openmpi -pthread
>> -Wl,-R/usr/local/libexec/openmpi/lib sprngf_mpi_tmp-sprngf_mpi.o
>> -L../../lib -lsprng -lm -lstdc++ -lm -L/usr/lib/openmpi/lib -lmpi_f77
>> -lmpi -lopen-rte -lopen-pal -ldl -Wl,--export-dynamic -lnsl -lutil -lm
>> -ldl
>> mpif77 -Wl,-R/usr/local/libexec/openmpi/lib -DHAVE_CONFIG_H -I. -I.
>> -I../.. -I../../include   -I../../include -DSPRNG_MPI -g -O2 -c -o
>> sprngf_simple_mpi_tmp-sprngf-simple_mpi.o `test -f
>> 'sprngf-simple_mpi.F' || echo './'`sprngf-simple_mpi.F
>> mpif77 -Wl,-R/usr/local/libexec/openmpi/lib  -g -O2   -o
>> sprngf-simple_mpi.tmp -I/usr/lib/openmpi/include
>> -I/usr/lib/openmpi/include/openmpi -pthread
>> -Wl,-R/usr/local/libexec/openmpi/lib --showme:link
>> sprngf_simple_mpi_tmp-sprngf-simple_mpi.o -L../../lib -lsprng -lm
>> -lstdc++ -lm
>> -pthread -Wl,-R/usr/local/libexec/openmpi/lib -g -O2 -o
>> sprngf-simple_mpi.tmp -I/usr/lib/openmpi/include
>> -I/usr/lib/openmpi/include/openmpi -pthread
>> -Wl,-R/usr/local/libexec/openmpi/lib
>> sprngf_simple_mpi_tmp-sprngf-simple_mpi.o -L../../lib -lsprng -lm
>> -lstdc++ -lm -L/usr/lib/openmpi/lib -lmpi_f77 -lmpi -lopen-rte
>> -lopen-pal -ldl -Wl,--export-dynamic -lnsl -lutil -lm -ldl
>> mv: cannot stat `sprng-simple_mpi.tmp': No such file or directory
>> make[4]: *** [sprng_mpi_test] Error 1
>> make[4]: Leaving directory
>> `/home/pauljohn/LinuxDownloads/src/sprngrpm/sprng-4.0/EXAMPLES/mpisprng'
>> make[3]: *** [all-recursive] Error 1
>> make[3]: Leaving directory
>> `/home/pauljohn/LinuxDownloads/src/sprngrpm/sprng-4.0/EXAMPLES/mpisprng'
>> make[2]: *** [all-recursive] Error 1
>> make[2]: Leaving directory
>> `/home/pauljohn/LinuxDownloads/src/sprngrpm/sprng-4.0/EXAMPLES'
>> make[1]: *** [all-recursive] Error 1
>> make[1]: Leaving directory
>> `/home/pauljohn/LinuxDownloads/src/sprngrpm/sprng-4.0'
>> make: *** [all] Error 2
>>
>>
>>
>> --
>> Paul E. Johnson
>> Professor, Political Science
>> 1541 Lilac Lane, Room 504
>> University of Kansas
>> _______________________________________________
>> R-sig-hpc mailing list
>> R-sig-hpc at r-project.org
>> https://stat.ethz.ch/mailman/listinfo/r-sig-hpc
>>
>
>
> --
> Department of Statistics & Actuarial Sciences
> Fax Phone#:(519)-661-3813
> The University of Western Ontario
> Office Phone#:(519)-661-3622
> London, Ontario N6A 5B7
> http://www.stats.uwo.ca/faculty/yu
>
> _______________________________________________
> R-sig-hpc mailing list
> R-sig-hpc at r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-hpc
>

-- 
best,
-tony

blindglobe at gmail.com
Muttenz, Switzerland.
"Commit early,commit often, and commit in a repository from which we
can easily roll-back your mistakes" (AJR, 4Jan05).

Drink Coffee:  Do stupid things faster with more energy!