[BioC] Memory and vector size management on linux 64bit

Fri Apr 16 21:54:57 CEST 2010

Hi Marco,

I just checked the page I gave you the link to and it appears that the
options there are either i386 or amd64. :-(  Maybe you'll need to
compile from the source (the r-sig-debian folks will have the answer
for sure).

As you probably noticed, what you want to achieve is
.Machine[["sizeof.pointer"]] == 8.

b

On Fri, Apr 16, 2010 at 8:04 PM, Manca Marco (PATH)
<m.manca at path.unimaas.nl> wrote:
>
> Dear Benilton,
>
> thank you for the prompt reply.
>
> I suspect you are right. R had just been installed from the Canonical repository on this machine and it was a 32bit edition.
>
> Thank you for your suggestion and for pointing out the 64bit repositories.
>
> Just for the sake of completeness at this point, since you alread solved the puzzle, I attach the output of the command you suggested:
>
>> .Machine[["sizeof.pointer"]]
> [1] 4
>
>
> All the best, Marco
>
>
> --
> Marco Manca, MD
> University of Maastricht
> Faculty of Health, Medicine and Life Sciences (FHML)
> Cardiovascular Research Institute (CARIM)
>
> Mailing address: PO Box 616, 6200 MD Maastricht (The Netherlands)
> Visiting address: Experimental Vascular Pathology group, Dept of Pathology - Room5.08,  Maastricht University Medical Center, P. Debeijelaan 25, 6229  HX Maastricht
>
> E-mail: m.manca at path.unimaas.nl
> Office telephone: +31(0)433874633
> Personal mobile: +31(0)626441205
> Twitter: @markomanka
>
>
> *********************************************************************************************************************
>
> This email and any files transmitted with it are confidential and solely for the use of the intended recipient.
>
> It may contain material protected by privacy or attorney-client privilege. If you are not the intended recipient or the person responsible for
>
> delivering to the intended recipient, be advised that you have received this email in error and that any use is STRICTLY PROHIBITED.
>
> If you have received this email in error please notify us by telephone on +31626441205 Dr Marco MANCA
>
> *********************************************************************************************************************
> ________________________________________
> Da: Benilton Carvalho [beniltoncarvalho at gmail.com]
> Inviato: venerdì 16 aprile 2010 18.47
> A: Manca Marco (PATH)
> Cc: bioconductor mailing list
> Oggetto: Re: [BioC] Memory and vector size management on linux 64bit
>
> what do you get when you run:
>
> .Machine[["sizeof.pointer"]]
>
> ?
>
> I'm really surprised by the "i486-pc-linux-gnu" bit from your
> sessionInfo()... I'd expect something like x86_64, if you were running
> a 64 bit version of R.
>
> Given that you're running Ubuntu, you're expected to follow the steps
> as described by the page below:
>
> http://cran.ma.imperial.ac.uk/bin/linux/ubuntu/
>
> b
>
> On Fri, Apr 16, 2010 at 5:30 PM, Manca Marco (PATH)
> <m.manca at path.unimaas.nl> wrote:
>>
>> Dear fellow bioconductors,
>>
>> good afternoon.
>>
>> I am encountering an error concerning the allocations of vectors bigger than a certain size. My problem is that said size is far below the amount of RAM I have installed on the system.
>>
>> Following some previous threads on the topic I have at first thought that my RAM could have been temporarily unavailable thus I have monitored the memory usage and the error message appeared when just 2,5Gb were in use out of the about 7 Gb available.
>>
>> The system is running linux Ubuntu Hardy Heron 64bit, and it sports a Xeon CPU and 8Gb of RAM.
>>
>> Following I am pasting my code and my sessionInfo()
>>
>> # Charging my microArray data (44 human whole genome by Illumina)
>>>library(lumi)
>>>fileName <- "MyExperiment_no_32_no_tech_duplos.csv";
>>>library(lumiHumanAll.db)
>>>Data.Immature <- lumiR(fileName, lib="lumiHumanAll.db");
>>>Data <- lumiExpresso(Data.Immature, QC.evaluation=TRUE);
>>
>> # Extract expression data from lumi object:
>>>Expression <- exprs(Data)
>>>names(Expression) <- Data at phenoData@data$sampleID
>>
>> # Calculate distance matrix for cluster analysis
>>>distance=dist(Expression, method="euclidean")
>> Error: cannot allocate vector of size 1.8 Gb
>>
>>> sessionInfo()
>> R version 2.10.1 (2009-12-14)
>> i486-pc-linux-gnu
>>
>> locale:
>>  [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C
>>  [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8
>>  [5] LC_MONETARY=C              LC_MESSAGES=en_US.UTF-8
>>  [7] LC_PAPER=en_US.UTF-8       LC_NAME=C
>>  [9] LC_ADDRESS=C               LC_TELEPHONE=C
>> [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
>>
>> attached base packages:
>> [1] stats     graphics  grDevices utils     datasets  methods   base
>>
>> other attached packages:
>>  [1] lumiHumanAll.db_1.8.1 org.Hs.eg.db_2.3.6    lumi_1.12.4
>>  [4] MASS_7.3-5            RSQLite_0.8-3         DBI_0.2-5
>>  [7] preprocessCore_1.8.0  mgcv_1.6-1            affy_1.24.2
>> [10] annotate_1.24.1       AnnotationDbi_1.8.1   Biobase_2.6.1
>>
>> loaded via a namespace (and not attached):
>> [1] affyio_1.14.0      grid_2.10.1        lattice_0.18-3     Matrix_0.999375-37
>> [5] nlme_3.1-96        tools_2.10.1       xtable_1.5-6
>>
>>
>>
>> Reading this page ( http://stat.ethz.ch/R-manual/R-patched/library/base/html/Memory.html ) I have come to hypothesize that there could be some vector size limit indipendent from the RAM availability, but I wonder whether it is true on a 64bit machine and how  could I circumvent  the problem:
>>
>> 1) any suggestion on how to modify the vector size threshold (if it exists)?
>>
>> 2) any suggestion on how to force the dist function to strip  down the matrix into smaller vectors?
>>
>>
>> Thank you in advance for your attention and for any feedback.
>>
>> All the best, Marco
>>
>>
>> --
>> Marco Manca, MD
>> University of Maastricht
>> Faculty of Health, Medicine and Life Sciences (FHML)
>> Cardiovascular Research Institute (CARIM)
>>
>> Mailing address: PO Box 616, 6200 MD Maastricht (The Netherlands)
>> Visiting address: Experimental Vascular Pathology group, Dept of Pathology - Room5.08,  Maastricht University Medical Center, P. Debeijelaan 25, 6229  HX Maastricht
>>
>> E-mail: m.manca at path.unimaas.nl
>> Office telephone: +31(0)433874633
>> Personal mobile: +31(0)626441205
>> Twitter: @markomanka
>>
>>
>> *********************************************************************************************************************
>>
>> This email and any files transmitted with it are confide...{{dropped:15}}
>>
>> _______________________________________________
>> Bioconductor mailing list
>> Bioconductor at stat.math.ethz.ch
>> https://stat.ethz.ch/mailman/listinfo/bioconductor
>> Search the archives: http://news.gmane.org/gmane.science.biology.informatics.conductor
>>
>