[Rd] AIX testers needed
Jason Barnhart
jasoncbarnhart at msn.com
Wed May 30 02:13:31 CEST 2007
Ei-ji,
Thank you very much for your reply; your suggestion resolved the
issue.
Follow up (for others):
1) In addition to setting the LDR_CNTRL environment variable
at run-time, R can be compiled by adding '-Wl,-
bmaxdata:0xn0000000' to LD_FLAGS (where 'n' is the number of
256MB increments of RAM requested) to engage large program
support (using GCC). This will support up to 2GB of RAM.
2) Setting LD_FLAGS set to '-Wl,-bmaxdata:0xD0000000/DSA' I
was able to compile R and use 3.0+ GB of RAM in a 32-bit
build. AIX documentation states this will support 3.5 GB of
RAM.
3) Under a 64-bit build, R could use all available RAM on the
server. One caveat here: requesting more RAM than the
system has available forces the OS to kill R without warning.
This could probably be prevented by using the -bmaxdata
option, but I have not tested this hypothesis.
4) AIX system documentation for 'Large Program Support' can
be found at
http://publib.boulder.ibm.com/infocenter/pseries/v5r3/index.jsp
for AIX 5.3 under 'Programming for AIX', then 'General
Programming Concepts: Writing and Debugging Programs'
----- Original Message -----
From: "Ei-ji Nakama" <nakama at ki.rim.or.jp>
To: "Jason Barnhart" <jasoncbarnhart at msn.com>
Cc: <r-devel at stat.math.ethz.ch>
Sent: Monday, May 21, 2007 7:32 PM
Subject: Re: [Rd] AIX testers needed
> Hi.
>
> I was not careful to 32bit very much.
> You can expand memory for data to 2G by LDR_CNTRL.
> It is as follows.
>
> LDR_CNTRL="MAXDATA=0x10000000" 1 256 MB
> LDR_CNTRL="MAXDATA=0x20000000" 2 512 MB
> LDR_CNTRL="MAXDATA=0x30000000" 3 768 MB
> LDR_CNTRL="MAXDATA=0x40000000" 4 1024 MB
> LDR_CNTRL="MAXDATA=0x50000000" 5 1280 MB
> LDR_CNTRL="MAXDATA=0x60000000" 6 1536 MB
> LDR_CNTRL="MAXDATA=0x70000000" 7 1792 MB
> LDR_CNTRL="MAXDATA=0x80000000" 8 2048 MB
>
> try.
> $ export LDR_CNTRL="MAXDATA=0x80000000"
> $ R -q
>> xx <- matrix(rep(1e+10,1e7),nrow=1e4,ncol=1e3)
>> tmp.df <- as.data.frame(cbind(xx,xx,xx))
>>
>
>
>
> 2007/5/22, Jason Barnhart <jasoncbarnhart at msn.com>:
>> Thank you for responding.
>>
>> I should have added -a on my ulimit command. Here are its results;
>> which I believe are not the limiting factor.
>>
>> %/ > ulimit -a
>> core file size (blocks, -c) 1048575
>> data seg size (kbytes, -d) unlimited
>> file size (blocks, -f) unlimited
>> max memory size (kbytes, -m) 32768
>> open files (-n) 2000
>> pipe size (512 bytes, -p) 64
>> stack size (kbytes, -s) hard
>> cpu time (seconds, -t) unlimited
>> max user processes (-u) 128
>> virtual memory (kbytes, -v) unlimited
>>
>> Here are my gcc-v results. They are very similiar.
>>
>> %/ > gcc -v
>> Using built-in specs.
>> Target: powerpc-ibm-aix5.3.0.0
>> Configured with: ../gcc-4.1.2/configure
>> --prefix=/opt/sas/msd/home/barnharj/usr/local
>> --program-suffix=-4.1.2
>> --enable-threads=aix
>> --enable-languages=c,c++,fortran
>> --disable-nls
>> --with-mpfr=/opt/sas/msd/home/barnharj/usr/local
>> --with-gmp=/opt/sas/msd/home/barnharj/usr/local
>> Thread model: aix
>> gcc version 4.1.2
>>
>> My results for gcc -print-multi-lib are indentical to yours.
>>
>> It should be noted that I did not attempt a 64 bit build. My
>> system
>> is not configured properly for that just yet.
>>
>> Via private correspondence someone also suggested the following
>> system("ulimit -d unlimited") during an R session. That failed as
>> did
>> issuing the command before launching R.
>>
>> I'll keep investigating.
>>
>> My call to configure is listed below.
>>
>> /configure --prefix=$HOME/usr/local --program-suffix=rc --with-readline=no
>> --with-x=no --enable-memory-profiling
>>
>>
>> ----- Original Message -----
>> From: "Ei-ji Nakama" <nakama at ki.rim.or.jp>
>> To: "Jason Barnhart" <jasoncbarnhart at msn.com>
>> Cc: <r-devel at stat.math.ethz.ch>
>> Sent: Saturday, May 19, 2007 6:53 PM
>> Subject: Re: [Rd] AIX testers needed
>>
>>
>> > Hi.
>> >
>> > system("ulimit") of AIX gives back file block size.
>> > A limit of memory is `ulimit -m'.
>> >
>> > I made gcc-4.1.2 and made R-2.5.0+patched, but the problem did
>> > not
>> > happen.
>> >
>> > $ gcc-4.1 -v
>> > Using built-in specs.
>> > Target: powerpc-ibm-aix5.2.0.0
>> > Configured with: ../configure --with-as=/usr/bin/as
>> > --with-ld=/usr/bin/ld --disable-nls --prefix=/usr/local1/gcc-4.1.2
>> > --enable-threads --host=powerpc-ibm-aix5.2.0.0 --program-suffix=-4.1
>> > --with-gmp=/usr/local1 --with-mpfr=/usr/local1
>> > --enable-languages=c,c++,f95
>> > Thread model: aix
>> > gcc version 4.1.2
>> >
>> > $ gcc-4.1 -print-multi-lib
>> > .;
>> > pthread;@pthread
>> > ppc64;@maix64
>> > pthread/ppc64;@pthread at maix64
>> >
>> > $ export OBJECT_MODE=64
>> > $ ./configure CC="gcc-4.1 -maix64" \
>> > F77="gfortran-4.1 -maix64" \
>> > CXX="g++-4.1 -maix64" \
>> > --enable-BLAS-shlib --without-iconv
>> >
>> >
>> > 2007/5/19, Jason Barnhart <jasoncbarnhart at msn.com>:
>> >> Per the request to test the latest tarball referenced below, I
>> >> have
>> >> built R on AIX 5.3. There is a memory issue, please see 3)
>> >> below.
>> >>
>> >> 1) Build with --enable-BLAS-shlib option. Builds and
>> >> passes "make check".
>> >>
>> >> 2) GNU libiconv was installed; R configured *without*
>> >> the --without-
>> >> iconv option. Builds and passes "make check."
>> >>
>> >> 3) Memory issue:
>> >> a) Although the server possesses 8GB of RAM and
>> >> system("ulimit") returns "unlimited" as its value, R
>> >> does
>> >> not
>> >> "have enough memory."
>> >>
>> >> b) This code works on R-2.4.0 on WinXP with 2GB of RAM,
>> >> but
>> >> fails on the AIX build. Example code:
>> >>
>> >> > xx <- matrix(rep(1e+10,1e7),nrow=1e4,ncol=1e3)
>> >> > tmp.df <- as.data.frame(cbind(xx,xx,xx))
>> >> Error: cannot allocate vector of size 228.9 Mb
>> >> > gc()
>> >> used (Mb) gc trigger (Mb) max used (Mb)
>> >> Ncells 233035 6.3 467875 12.5 350000 9.4
>> >> Vcells 10104141 77.1 31854441 243.1 30104289 229.7
>> >>
>> >> 4) Used gcc-4.1.2; I have not tried the native compiler.
>> >>
>> >> Questions:
>> >>
>> >> 1) Are there suggestions on how to diagnose and resolve the
>> >> memory
>> >> issue? I'm still building my tool chain and I am in the
>> >> midst
>> >> of
>> >> installing gdb.
>> >>
>> >> 2) Regarding the build process itself, is there more
>> >> documentation
>> >> or results that I should forward?
>> >>
>> >> Thanks,
>> >> -jason
>> >>
>> >>
>> >> > version _
>> >> platform powerpc-ibm-aix5.3.0.0
>> >> arch powerpc
>> >> os aix5.3.0.0
>> >> system powerpc, aix5.3.0.0
>> >> status RC
>> >> major 2
>> >> minor 5.0
>> >> year 2007
>> >> month 04
>> >> day 22
>> >> svn rev 41276
>> >> language R
>> >> version.string R version 2.5.0 RC (2007-04-22 r41276)
>> >>
>> >>
>> >> ----- Original Message -----
>> >> From: "Prof Brian Ripley" <ripley at stats.ox.ac.uk>
>> >> To: <R-devel at r-project.org>
>> >> Cc: "Sheth, Jagat K" <jagat.k.sheth at citigroup.com>; "kamil
>> >> Marcinkowski" <kamil at ualberta.ca>; "Tiong Lim"
>> >> <t.lim at auckland.ac.nz>;
>> >> "Eric Harley" <ericctharley at gmail.com>; "Matthew Beason"
>> >> <mbeason at harrahs.com>; <watplatt at us.ibm.com>
>> >> Sent: Wednesday, March 28, 2007 11:27 PM
>> >> Subject: [Rd] AIX testers needed
>> >>
>> >>
>> >> > With much thanks to Ei-ji Nakama, R 2.5.0 alpha supports
>> >> > building
>> >> > on
>> >> > AIX
>> >> > (at least AIX 5.2 on one system).
>> >> >
>> >> > Would anyone able to test this please get the latest tarball
>> >> > from
>> >> >
>> >> > http://cran.r-project.org/src/base-prerelease/R-latest.tar.gz
>> >> >
>> >> > and try installing (after reading the AIX notes in
>> >> > R-admin.html
>> >> > section
>> >> > C.9).
>> >> >
>> >> > In particular it would be very helpful to know if
>> >> >
>> >> > 1) --enable-BLAS-shlib works (it is the default everywhere
>> >> > except
>> >> > AIX and
>> >> > Darwin)
>> >> >
>> >> > 2) if people succeed in installing GNU libiconv and building
>> >> > without
>> >> > needing --without-iconv.
>> >> >
>> >> > I am Cc:ing all the people I tracked down who reported
>> >> > attempted
>> >> > AIX
>> >> > installations in 2006 in the hope that they may still be
>> >> > interested.
>> >> >
>> >> > Please report success and any hints or problems in the R-devel
>> >> > list.
>> >> >
>> >> > --
>> >> > Brian D. Ripley, ripley at stats.ox.ac.uk
>> >> > Professor of Applied Statistics,
>> >> > http://www.stats.ox.ac.uk/~ripley/
>> >> > University of Oxford, Tel: +44 1865 272861 (self)
>> >> > 1 South Parks Road, +44 1865 272866 (PA)
>> >> > Oxford OX1 3TG, UK Fax: +44 1865 272595
>> >> >
>> >> > ______________________________________________
>> >> > R-devel at r-project.org mailing list
>> >> > https://stat.ethz.ch/mailman/listinfo/r-devel
>> >> >
>> >>
>> >> ______________________________________________
>> >> R-devel at r-project.org mailing list
>> >> https://stat.ethz.ch/mailman/listinfo/r-devel
>> >>
>> >>
>> >>
>> >
>> >
>> > --
>> > EI-JI Nakama <nakama at ki.rim.or.jp>
>> > "\u4e2d\u9593\u6804\u6cbb" <nakama at ki.rim.or.jp>
>> >
>>
>>
>>
>>
>
>
> --
> EI-JI Nakama <nakama at ki.rim.or.jp>
> "\u4e2d\u9593\u6804\u6cbb" <nakama at ki.rim.or.jp>
>
More information about the R-devel
mailing list