Restatement: Was Re: [BioC] Memory Requirments

[James MacDonald]

James W. MacDonald
UMCCC Microarray Core Facility
1500 E. Medical Center Drive
7410 CCGC
Ann Arbor MI 48109
734-647-5623

>>> "Michael Barnes" <Michael.Barnes at cchmc.org> 06/25/03 02:37PM >>>
>This is in response to the various answers I received to my questions
>below.  First, thank you all for your comments about duplicating work
>with the command I used.  Secondly, I am running Windows XP with R
>1.7.1.

>I should point out that according to the May 15th posting, Laurent
>Gautier's method does not run on Windows.

You are incorrect; Laurent contributed source code so it must be
compiled before use. This is not a trivial task on Windoze, so Laurent
was simply letting people know that if they didn't have the R tools,
they would be out of luck. However, with the correct tools it is
doable.


>1.  Concerning the error below, 83200kb is less than the allocated
1024
>Mb.  Does this mean that it requires 83200Kb for each chip (ie I need
>83200kb for each of 38 chips or a total of about 3000 Mb) or does
this
>mean that there are other memory usages that are causing this memory
>issue?  ie, is this due to the data processing for the analysis I
have
>tried to perform?

It does not mean that each chip requires 83 Mb. All it means is that R
was trying to allocate 83 Mb and ran out of memory. The memory
requirement is not linearly related to chip number, so you cannot simply
divide to get memory required per chip.

>2.  Sort of the reverse of that question, is there an estimate for
the
>amount of memory needed based on the number of chips being analyzed? 
I
>could go to another computer, buy more memory, or something else, but
>what do I need in order to be able to do the analysis that I want
here
>or I might chose to do in the future?  Is there some way to
approximate
>these requirements?  This is assuming I wanted to use rma (or an
>estimate for justRMA would be good too).  If I decide later that I
want
>to use one of the other analyses in Affy, is there some way, other
than
>trial and error, to guess/approximate the amount of memory I will
need?

Rafael requested a table of memory requirements, which I am working on.
However, ReadAffy gets really slow when the number of chips gets big, so
I only have info for justRMA. R ver is 1.7.1, affy ver is 1.3.0

# Chips	Max mem required	Final memory used (to hold
exprSet in memory)
59 U95A		310 Mb		212 Mb
76 U95A		529 Mb		338 Mb
96 U95A		582 Mb		370 Mb
46 U133A		345 Mb		301 Mb

This was all done on a 1.13 GHz PIII laptop running WinXP with 512 Mb
RAM (768 Mb page file). Needless to say, I was hitting the page file
really hard. Free memory dipped to under 1 Kb with the 96 chip run ;-D

I am testing ReadAffy/rma using a 2.4 GHz PIV Desktop running WinXP
with 2 Gb RAM. So far all I have is info for the 59 U95A chips:

#Chips	Max mem required	Final memory used (exprSet and
AffyBatch)
59 U95A		1.8 Gb		807 Mb


Jim