[BioC] Regarding error while normalizing using "loess" method

Francois Pepin fpepin at cs.mcgill.ca
Fri Oct 6 21:28:45 CEST 2006 

Hi Ganiraju,

how many arrays are you working with?

It seems to me that you just don't have enough RAM to do what you're
trying to do. This is exactly what the error message means. It has used
up all available memory and it's trying to access some more and failing.
Changing the memory.limit() above what you have available wouldn't help
for that. Even if you tell R it can 4Gb of RAM, your computer still only
has 1Gb.

The reason why it is be running forever would be because the computer is
heavily swapping.

The first solution would be to use a computer with more than 1Gb of RAM.
Windows is unfortunately severely limited in how it handles memory above
2Gb, so if you require more than that, it would be a good idea to use
Linux or Mac computers instead. If you need more than 4Gb, you would
need a 64bit-capable computer as well.

Francois

On Fri, 2006-10-06 at 15:00 -0400, Ganiraju Manyam wrote:
> hi James,
> 
> I tried debugging. But , I cant make any thing out of it.
> 
> When shut and open the R environment , the loess is running. I dont know
> even why it is running.. but happy.
> 
> And, I cannot really solve the memory problem. If I give the full dataset,
> it says there is problem in creating vector. Here is the set of commands.
> 
> **************************************************************************
> > memory.limit()
> [1] 1063641088
> > memory.limit(size = 4000)
> NULL
> > memory.limit()
> [1] 4194304000
> > ls()
> [1] "cel.data" "exp.data"
> > rm (list = ls())
> > ls()
> character(0)
> > cel.data <- ReadAffy()
> > memory.limit()
> [1] 4194304000
> > exp.data <- expresso(cel.data,bgcorrect.method="mas",normalize.method=
> "loess",pmcorrect.method="mas",summary.method="medianpolish",verbose=TRUE)
> background correction: mas
> normalization: loess
> PM/MM correction : mas
> expression values: medianpolish
> background correcting...Error: cannot allocate vector of size 201117 Kb
> >
> *************************************************************************
> 
> but the process ran overnight. I could not make out anything of it.
> Presently, I am working in a windows system with 1GB RAM.
> 
> Thank you,
> Ganiraju
> 
> 
> 
> On 10/3/06, James W. MacDonald <jmacdon at med.umich.edu> wrote:
> >
> > Ganiraju Manyam wrote:
> > > hi James,
> > > Thanks for the info.
> > >
> > > Here is the copy of console.
> > >
> > >
> > ************************************************************************************
> > >
> > >
> > >
> > >> sessionInfo()
> > >
> > > Version 2.3.1 (2006-06-01)
> > > i386-pc-mingw32
> > >
> > > attached base packages:
> > > [1] "tools"     "methods"   "stats"     "graphics"  "grDevices" "utils"
> > > [7] "datasets"  "base"
> > >
> > > other attached packages:
> > > hgu133plus2cdf           affy         affyio        Biobase
> > >      "1.12.0"       "1.10.0"        "1.0.0"       "1.10.1"
> > >
> > >> cel.data <- ReadAffy()
> > >> exp.data <- expresso(cel.data,bgcorrect.method = "mas",
> > >> normalize.method =
> > >
> > > "loess", pmcorrect.method = "mas", summary.method ="medianpolish")
> > > background correction: mas
> > > normalization: loess
> > > PM/MM correction : mas
> > > expression values: medianpolish
> > > background correcting...done.
> > > normalizing...Error in matrix(0, II, J) : non-numeric matrix extent
> >
> > That's odd. I can get this call to expresso() to work here with the same
> > packages. What happens if you do this:
> >
> > debug(normalize.loess)
> > cel.data <- ReadAffy()
> > exp.data <- expresso(cel.data,bgcorrect.method = "mas", normalize.method
> > = "loess", pmcorrect.method = "mas", summary.method ="medianpolish")
> >
> > Then when you enter the debugger, you type dim(mat) and then class(mat)?
> >
> > Best,
> >
> > Jim
> >
> >
> >
> > >> exp.data <- expresso(cel.data,bgcorrect.method = "mas",
> > >> normalize.method =
> > >
> > > "quantiles", pmcorrect.method = "mas", summary.method ="medianpolish")
> > > background correction: mas
> > > normalization: quantiles
> > > PM/MM correction : mas
> > > expression values: medianpolish
> > > background correcting...done.
> > > normalizing...done.
> > > 54675 ids to be processed
> > > |                    |
> > > |######
> > > ##############|
> > >
> > >>
> > >
> > >
> > ***********************************************************************************************
> > >
> > >
> > > The problem occurs only will "loess" method and is working fine when i
> > use
> > > "quantiles" or "constant" normalization.
> > >
> > > Thank you,
> > > Ganiraju
> > >
> > >
> > > On 10/2/06, James W. MacDonald <jmacdon at med.umich.edu> wrote:
> > >
> > >>
> > >> Hi Ganiraju,
> > >>
> > >> Ganiraju Manyam wrote:
> > >> > hi all,
> > >> >
> > >> > I am Ganiraju, a new member of the group. (and currently using the
> > affy
> > >> > package)
> > >> >
> > >> > I m trying to use "loess" method in expresso function to normalize
> > the
> > >> data.
> > >> >
> > >> > While running Im getting an error: "non-numeric matrix extent"
> > >> >
> > >> > I didnt really understand whether is some problem with ReadAffy(), or
> > >> > functinoality of "loess", whether it can be used on all kinds of
> > >> data or
> > >> > not.
> > >>
> > >> You really need to give more information than this. At a minimum, you
> > >> need to give the commands you ran along with the results (i.e., copying
> > >> and pasting from within R). You should also give the results that you
> > >> get when you run sessionInfo() after loading all the packages you are
> > >> using here.
> > >>
> > >>
> > >> >
> > >> > Also, Im runnig sort of memory. Is there any other way to create
> > >> virtual
> > >> > memory other than using "memory.limit."
> > >>
> > >> Assuming you are on windows, as AFAIK, this command doesn't exist for
> > >> *nix or MacOS, the short answer is no. You can set the command line
> > flag
> > >> --max-mem-size in the shortcut you use to launch R, setting this value
> > >> to the amount of RAM you have. If you still run out of memory, the next
> > >> step is to buy more RAM. R on windows will usually stop a computation
> > >> rather than using virtual memory (much to its credit IMO). Trying to do
> > >> any real computation using virtual memory is too painfully slow to be
> > >> recommended.
> > >>
> > >> Best,
> > >>
> > >> Jim
> > >>
> > >>
> > >> >
> > >> > Thanks in advance,
> > >> > Ganiraju
> > >> >
> > >> >       [[alternative HTML version deleted]]
> > >> >
> > >> > _______________________________________________
> > >> > Bioconductor mailing list
> > >> > Bioconductor at stat.math.ethz.ch
> > >> > https://stat.ethz.ch/mailman/listinfo/bioconductor
> > >> > Search the archives:
> > >> http://news.gmane.org/gmane.science.biology.informatics.conductor
> > >>
> > >>
> > >> --
> > >> James W. MacDonald, M.S.
> > >> Biostatistician
> > >> Affymetrix and cDNA Microarray Core
> > >> University of Michigan Cancer Center
> > >> 1500 E. Medical Center Drive
> > >> 7410 CCGC
> > >> Ann Arbor MI 48109
> > >> 734-647-5623
> > >>
> > >>
> > >> **********************************************************
> > >> Electronic Mail is not secure, may not be read every day, and should
> > not
> > >> be used for urgent or sensitive issues.
> > >>
> > >
> >
> >
> > --
> > James W. MacDonald, M.S.
> > Biostatistician
> > Affymetrix and cDNA Microarray Core
> > University of Michigan Cancer Center
> > 1500 E. Medical Center Drive
> > 7410 CCGC
> > Ann Arbor MI 48109
> > 734-647-5623
> >
> >
> > **********************************************************
> > Electronic Mail is not secure, may not be read every day, and should not
> > be used for urgent or sensitive issues.
> >
> 
> 	[[alternative HTML version deleted]]
> 
> _______________________________________________
> Bioconductor mailing list
> Bioconductor at stat.math.ethz.ch
> https://stat.ethz.ch/mailman/listinfo/bioconductor
> Search the archives: http://news.gmane.org/gmane.science.biology.informatics.conductor
>

More information about the Bioconductor mailing list